Passing Context with Temporal
Why do we need Context?
Within a program, context is a mechanism to pass some data without a strict definition or needing to tie it to a piece of code. In addition, context is usually passed to the explicit attributes as the first attribute. Usually, context is constructed as a key/value store, where the key and the value may be whatever data structure is possible in the programming language. For example, in Go, context accepts interface{} as the key and the value, so the data can be any structure or data type. In addition, context is used to cancel computations in case of a timeout.
Since we can pass any data with context, the question becomes, “Why not pass any input parameters in the single context parameter of a function?” Take the following code snippet as an example:
package main
import (
"fmt"
"context"
)
func main() {
ctx := context.WithValue(context.Background(), "arg1", 10)
ctx = context.WithValue(ctx, "arg2", 10)
fmt.Println(Summ(ctx))
}
func Summ(ctx context.Context) int {
arg1, ok1 := ctx.Value("arg1").(int)
arg2, ok2 := ctx.Value("arg2").(int)
if ok1 && ok2 {
return arg1 + arg2
}
return 0
}
It isn’t the best practice to do it this way because we lose the advantages of strong typing, i.e., there are no compile time checks for the function arguments. The code becomes much harder to read/understand and less predictable because the values are passed via a hidden context instead of explicit arguments.
Looking at this another way, we might ask why can’t we pass everything in the attributes for our functions? This way, we may have a lot of irrelevant information in the attributes, and we will have to change the function signature every time we add some key/value, even if it is not used in the function itself. So what are the criteria when passing a given piece of data to a method with the context or attributes?
When we develop microservices in Golang, we develop quite a clear understanding of the role of Context. But first, we need to see what is the Onion or DDD-centric architecture.
DDD Architecture
The onion or DDD (Domain Driven Design) architecture is about the dependencies among the components in the code. It considers the domain entities and the domain business scenarios at the center of the application, so they do not depend on any specific transport protocols, storage, or other implementation details.
The domain is the area that is going to be automated within the software we are going to create. It may be an ecommerce store, some help desk system, or larger banking system, etc.
What are the domain entities and scenarios? According to the work of Robert Martin, the domain entities and scenarios in a system is something that exists in the domain without the system and before the system was introduced. The business scenarios are very often just interfaces of the services that use the domain entities as the arguments.
Then we have the application layer that implements the domain (it will depend on it) and uses certain storage access (SQL/NoSQL databases, file storages) and computations in order to get the desired behavior.
The application does not depend on the transports or how the data is delivered precisely to the service. Always remember that this depends on the domain.
So the transport layer should be aware of the application (service layer), but not vice versa.
But what is the middleware? The middleware is common functionality that is not related to the business (domain) but to the functionality of the system itself. It may be authorization, logging of the request/response, taking metrics, tracing, error limiting, etc. The middleware does not care about the business data. It considers the business data as something general. We use go-kit for developing microservices, and go-kit allows for a clear separation of the layers.
The business data in go-kit is represented as a generic interface{} for the middleware endpoints. The interface can implement some methods for data extraction, but the middleware should not know about the real business contents of the request. The middleware should have some common data for processing. This is where we use the context.
Any transport like HTTP, gRPC, AMQP, or Temporal will all use the headers to pass the irrelevant to the business data information. The data in the headers can influence how the business data may be interpreted: a wrong jwt token in the headers may result in an authorization error, or a given process id in the headers may be used to filter out the logs.
If we do it this way, we can separate the business data and infrastructure data, placing the infrastructure data into the headers and then into the context. So the context is used by the middleware, and the business data is used by the application and the domain. The context is passed into the application, but the application should not extract any data from the context, so we don’t have any side effects. The application just passes the context through to some clients for databases, storage, or other service calls.
Thus there should be two types of mapping of the data: for the business data, taken from the transport payload and the context, taken from the headers. The business data mapping is specific to the concrete business scenario and always different. But the context mapping is the same for any endpoint of the application. The mapping of the context is usually done with transport options (go-kit), interceptors (gRPC), or context propagators (Temporal).
We usually pass the authorization data (tokens), the process id, and the tracing data inside the context. Let us see how it may be implemented with Temporal at the transport layer.
Temporal Context Propagators
I am going to use the repository developed for the Temporal Microservice blog post in order to demonstrate how the Temporal propagators work. I highly recommend that you get familiar with the Temporal Microservice blog post because I will use some ideas we discussed there. There is a special branch of the repository where we can see two context propagators for the strings and the secrets.
The string context propagator just passes the required strings from/into the headers, the context of the workflow, and the context of the activities.
The secret context propagator is used in order to keep tokens. Since Temporal saves the data with the headers in its database, we should encrypt it for security reasons.
The ContextPropagator interface has four functions in order to extract the data into the context from the headers and inject the data from the context into the headers
https://github.com/temporalio/sdk-go/blob/master/internal/headers.go#L46-L60
One pair of the functions (Inject and Extract) are working with the ordinary Golang context, and another pair (InjectFromWorkflow, ExtractToWorkflow) is working with the Temporal workflow context. The workflows have their own context because there should be no possibility to create a workflow context and erase all the technical Temporal context data from it.
In fact, it is very convenient to have a separate context for the workflow and a separate pair of functions for the context propagation. With regards to the secret propagation, we want to keep the secret encrypted inside the workflow context, and we are able to do that.
The custom context propagators should be added to the temporal client like this
https://github.com/guntenbein/temporal_microservices/blob/context-propagation/cmd/microservice_workflow/main.go#L48-L56
and that is all we have to do! The context values will be propagated into any activities, even if they are located in different services. Of course, the services should also have context propagators for the Temporal client.
Passing Context from HTTP Headers
The example project starts the Temporal workflow with the HTTP server. In this case, we need to pass the context in HTTP headers and then inject the headers into the context of the service that starts the workflow. Usually, it is done with a generic component used in any HTTP handler. We won’t go too deep into components in this post, and we just placed the injection part in the function of the controller
https://github.com/guntenbein/temporal_microservices/blob/context-propagation/controller/http.go#L49-L57
Token Encryption
In our sample, we used base64 encoding for the sake of simplicity
https://github.com/guntenbein/temporal_microservices/blob/context-propagation/context/propagators/crypto.go
Of course, it does not give us any level of security, and in a real application, we should use a more strong encryption technique like AES.
A developer should pay attention and use interfaces according to the dependency inversion principle. In fact, Golang’s duck typing lets us create interface contracts and then provide the implementations for the contracts. Within here, there is the encryption component interface used in the secret context propagator that can have any implementation:
https://github.com/guntenbein/temporal_microservices/blob/context-propagation/context/propagators/secret.go#L13-L16
Run Workflow with Context
Requirements needed to run the project:
Start by checking out the project using the following commands:
git clone https://github.com/guntenbein/temporal_microservices.git
Check out the branch with the context propagators:
git checkout context-propagation
Then, run the Temporal services by typing in the following command from the root of the project:
docker-compose up -d
When the Temporal service starts, run the microservices one-by-one:
go run cmd/microservice_square/main.go
go run cmd/microservice_volume/main.go
go run cmd/microservice_workflow/main.go
Then, run the workflow with the command:
curl --location --request POST 'localhost:8080' \
--header 'process-id: process-number-42' \
--header 'Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwczovL2Rldi1hcGkuaW50dXJuLmlvOjU2ODUvYXV0aC9hY2Nlc
3NfdG9rZW4iLCJpYXQiOjE2MDQ0MTE4NzMsImV4cCI6MTg2MTY5NTUxOCwibmJmIjoxNjA0NDExODczLCJqdGkiOiJGcWxIOXpwcFB
2V1N6bUFZIiwic3ViIjo0MzgzLCJ1aWQiOjQxMjYsInVpZF91dWlkIjoiZWZhZTQ0YTItN2IzNy00NjYxLWI5ZjItZjc5YzQzNzE0M
Tc4IiwiY29tcGFueV9pZCI6MjcxOSwiY29tcGFueV91dWlkIjoiM2U4YjM3NmEtOWI0MC00YzJmLWFhZTItODU1MTAwZTNiMmQ2Iiw
iY29tcGFueV9yb2xlIjoiU0VMTEVSIiwidXNlcl9yb2xlIjoiQURNSU4ifQ.eoHo5cnSslV3Xona4Ze-TbXXB-
JVcUhfij_BEdiBqTg' \
--header 'Content-Type: application/json' \
--data-raw '{
"BatchSize": 3,
"Parallelepipeds": [
{
"ID": "5fedcbf7901feb7213e84153",
"Length": 7584.6668,
"Width": 8551.7289,
"Height": 7911.1765
},
{
"ID": "5fedcbf755d18a8e807432d2",
"Length": 9854.9176,
"Width": 2333.052,
"Height": 9977.8465
},
{
"ID": "5fedcbf776f93aa072884a6e",
"Length": 6186.1635,
"Width": 7257.3111,
"Height": 744.9772
}
]
}'
After a while, you will see the response with the square and volume inside returned:
[
{
"ID": "5fedcbf7901feb7213e84153",
"Length": 7584.6668,
"Width": 8551.7289,
"Height": 7911.1765,
"BaseSquare": 64862014.27043052,
"Volume": 513134843038.89453
},
{
"ID": "5fedcbf755d18a8e807432d2",
"Length": 9854.9176,
"Width": 2333.052,
"Height": 9977.8465,
"BaseSquare": 22992035.216515202,
"Volume": 229410998112.98294
},
{
"ID": "5fedcbf776f93aa072884a6e",
"Length": 6186.1635,
"Width": 7257.3111,
"Height": 744.9772,
"BaseSquare": 44894913.034964845,
"Volume": 33445686607.031612
}
]
You can see the result of the workflow execution at the local Temporal UI server. If we open the last workflow execution result and go to History -> JSON tab, we see that the context values were added to the headers of the activity for the square and volume calculation:
We can see that the JWT token value is base64 encrypted.
See Context in the Activities
It’s important to confirm that the context appears with the activities executed by the workflow. In this case, let’s print the values from the context at the Square microservice, CalculateRectangleSquare activity. As we discussed earlier, the services don’t work with the context, and it is not best practice to extract some context values from the service directly. Please note, we can encapsulate it in a ContextRegistrar component under interface.
https://github.com/guntenbein/temporal_microservices/blob/context-propagation/domain/square/service.go#L33-L35
And provide the component implementation in the “context” package
https://github.com/guntenbein/temporal_microservices/blob/context-propagation/context/registrar.go
This way, the service doesn’t work with the context but instead passes it.
If we run the workflow again, we will see the following in the log:
Unit Tests
Of course, generic components like context propagators need unit tests. It is quite simple to test the functions for injecting/extracting the ordinary Golang context. We just create the mock for the header’s keeper and test how we pass/get the values directly from the headers:
https://github.com/guntenbein/temporal_microservices/blob/context-propagation/context/propagators/stubs.go
But it is more difficult to test the same thing for the workflow context because you can’t create it inside the test. Thus, it becomes more tricky: we need to have a mock workflow and test the InjectFromWorkflow, ExtractToWorkflow function there:
https://github.com/guntenbein/temporal_microservices/blob/context-propagation/context/propagators/secret_test.go#L75-L126
To start, I wrote tests for version v0.29.0 of the Temporal SDK, but Temporal is evolving quickly, and at the time of this post, the version is v1.3.0. Within this version, you can use context propagators for the unit tests of a workflow–so I decided to make the test with this feature in mind. I cannot say that this version is more simple and it doesn’t let me test some details like what exactly we have inside the headers after the Inject functions. And it is super important for the secret propagator that we are 100% sure that the headers will keep using encrypted tokens. In any case, it is nice to see how Temporal is evolving quickly, and the team responds to the questions and proposals from the community.
Conclusions
Context and transport headers are a very convenient way to pass the information unrelated to the business data but are still required by the infrastructure like authorization, process id, tracing, etc.
Temporal provides a very nice way to make custom context propagators and propagate the context across the whole workflow with many different services running on different hosts.
One of the most common examples for passing data with context is request tracing. We will cover how you can do this in a future article.