Temporal Universal Starter

As we described in our previous post about Temporal Microservices, Temporal can be used effectively as the microservices orchestration engine and will allow an engineer to construct quite complex behavior workflows with retries, backpressure and fault resistance. We can define different activities and use them as components for the workflows. In this case, the microservice orchestration becomes the real composition of the activities. In addition, the activity is the simplest block for any workflow. Often, an activity is not expected to be called separately, but only in relation to a workflow.

Any workflow needs a starter – a piece of code that should send a request to Temporal to start the workflow with a given piece of data in the input. But what if we want to run a separate activity? The activity cannot be started separately from a workflow. Thus, if we want to test the Get Balance activity from the picture above, then:

• We should execute the whole Complete Orders Workflow.
• We can create a small separate workflow, including the starter for the workflow, and only then can we use that starter in order to execute the single activity.

This can become more complex, especially if we need to repeat it for every activity. When developers construct a workflow, they should have the option to start an activity separately. Otherwise, the testing of the activities will become a true pain point. Can we have a universal activity or even a workflow starter? The standard Temporal SDK or UI does not support this, but it can be easily implemented. Below we provide a simple example for how it can be done in the https://github.com/guntenbein/temporal_starter repository. Let’s take a look at how to best implement this in your next project.

#Universal Starter Implementation

Any Temporal activity or workflow operates with interfaces as arguments. So any activity or workflow’s argument is just a Golang interface{}. It does not matter what data you pass. Temporal will provide the data to the activity or the workflow, independently of the content. This data abstraction is quite powerful and convenient because the activities can run on different services and in different languages. This data abstraction allows us to have an HTTP endpoint that provides any structure as the activity request and use the data for the activity input. This is the structure of the data we pass in the HTTP POST method body:

https://github.com/guntenbein/temporal_starter/blob/main/controller/start_activity.go#L14-L18
https://github.com/guntenbein/temporal_starter/blob/main/controller/start_activity.go#L14-L18	

We have a generic payload, the task queue and the activity name. This information is enough to start the activity using Temporal. All we need is a simple workflow in order to run the generic activity:

https://github.com/guntenbein/temporal_starter/blob/main/workflow/workflow.go				

And to start the workflow inside the HTTP handler:

https://github.com/guntenbein/temporal_starter/blob/main/controller/start_activity.go#L55-L71	

Temporal passes the arguments to the activities as the serialized JSON strings. Thus, it does not matter what the structure is of the activity’s real argument. The real structures may be different, but they should result in the same JSON.

#Running the Universal Starter

Prerequisites to run the project include:

• Installing Docker
• Installing Go

#Run Activities

First, we need to have some activities bootstrapped to Temporal to run them in a genetic way. We are going to use the same project from the post we linked to above called Temporal Microservices:

git clone https://github.com/guntenbein/temporal_microservices.git		

Check out the branch with the context propagators because we also are going to propagate the context with the context propagators, as was described in this post about [Context Propagators]:

git checkout context-propagation	

Then, we will run the Temporal services by typing the following command from the root of the project:

docker-compose up -d				

When the Temporal service starts, run the microservices for volume and square activities only, without the workflow for their invocation:

go run cmd/microservice_square/main.go

go run cmd/microservice_volume/main.go

Clone the repository with the starter implementation:

git clone https://github.com/guntenbein/temporal_starter.git				

Then, go to the root folder and run the application:

go run cmd/main.go				

Run the following request to execute the Square activity:

curl --location --request POST 'localhost:8081/activity/' \
--header 'process-id: Good Process!!!' \
--header 'Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwczovL2Rldi1hcGkuaW50dXJuLmlvOjU2ODUvYXV0aC9hY2Nlc
3NfdG9rZW4iLCJpYXQiOjE2MDQ0MTE4NzMsImV4cCI6MTg2MTY5NTUxOCwibmJmIjoxNjA0NDExODczLCJqdGkiOiJGcWxIOXpwcFB
2V1N6bUFZIiwic3ViIjo0MzgzLCJ1aWQiOjQxMjYsInVpZF91dWlkIjoiZWZhZTQ0YTItN2IzNy00NjYxLWI5ZjItZjc5YzQzNzE0M
Tc4IiwiY29tcGFueV9pZCI6MjcxOSwiY29tcGFueV91dWlkIjoiM2U4YjM3NmEtOWI0MC00YzJmLWFhZTItODU1MTAwZTNiMmQ2Iiw
iY29tcGFueV9yb2xlIjoiU0VMTEVSIiwidXNlcl9yb2xlIjoiQURNSU4ifQ.eoHo5cnSslV3Xona4Ze-TbXXB-
JVcUhfij_BEdiBqTg' \
--header 'Content-Type: application/json' \
--data-raw '{
 "ActivityName": "CalculateRectangleSquare",
 "QueueName": "SquareActivityQueue",
 "ActivityRequest": {
   "Rectangles": [
     {
       "ID": "5fedcbf7901feb7213e84153",
       "Length": 7584.6668,
       "Width": 8551.7289,
       "Height": 7911.1765
     },
     {
       "ID": "5fedcbf755d18a8e807432d2",
       "Length": 9854.9176,
       "Width": 2333.052,
       "Height": 9977.8465
     },
     {
       "ID": "5fedcbf776f93aa072884a6e",
       "Length": 6186.1635,
       "Width": 7257.3111,
       "Height": 744.9772
     },
     {
       "ID": "5fedcbf7cab168b5fa23e0bf",
       "Length": 1487.9815,
       "Width": 6904.6383,
       "Height": 6917.9239
     },
     {
       "ID": "5fedcbf7f1d0f9b6c8d94478",
       "Length": 9579.1483,
       "Width": 3908.5532,
       "Height": 1622.9292
     },
     {
       "ID": "5fedcbf7fd6de933e37141c6",
       "Length": 6060.2393,
       "Width": 5232.1464,
       "Height": 5528.2147
     },
     {
       "ID": "5fedcbf7a27adeb782670e09",
       "Length": 7608.5178,
       "Width": 3490.3491,
       "Height": 6064.8596
     },
     {
       "ID": "5fedcbf74cc4c73c8bdb6652",
       "Length": 6061.7923,
       "Width": 8985.7511,
       "Height": 7535.418
     },
     {
       "ID": "5fedcbf7b89726935be95418",
       "Length": 8633.0144,
       "Width": 4433.371,
       "Height": 2310.1432
     },
     {
       "ID": "5fedcbf7b55873c13a3d09c6",
       "Length": 6324.4951,
       "Width": 2566.1975,
       "Height": 7536.6964
     }
   ]
 }
}'					

And we should get this response:

{
   "Squares": {
       "5fedcbf74cc4c73c8bdb6652": 54469756.827696525,
       "5fedcbf755d18a8e807432d2": 22992035.216515202,
       "5fedcbf776f93aa072884a6e": 44894913.034964845,
       "5fedcbf7901feb7213e84153": 64862014.27043052,
       "5fedcbf7a27adeb782670e09": 26556383.255563978,
       "5fedcbf7b55873c13a3d09c6": 16229903.514382252,
       "5fedcbf7b89726935be95418": 38273355.6835424,
       "5fedcbf7cab168b5fa23e0bf": 10273974.05459145,
       "5fedcbf7f1d0f9b6c8d94478": 37440610.74123956,
       "5fedcbf7fd6de933e37141c6": 31708059.23663352
   }
}

Also, we will be able to see the execution of the workflow cover within the activity of Temporal’s UI:

If we take a look at the Square activity output, we will see that the context values are propagated normally so the activity has all the necessary context:

#Conclusions

The generic way for using Temporal keeps the activity’s arguments and allows us to run any activity or even any workflow. Thus, a developer or QA specialist can test activities or child workflows separately, without writing a starter and workflow cover for every single activity. This simplifies a lot the development process and allows for more easy manual testing of any separate activity. There are some negative side effects of the general, untyped passing of the arguments to the activities: - One could mistakenly provide a wrong argument (activity request) and may not see an error when compiling the code. One would only see it when they run it and it may take more time to detect errors.

Typed structures can use the power of the protocol buffers binary serialization, and the data will take less space than the JSON serialization used for the generic arguments.