-
Notifications
You must be signed in to change notification settings - Fork 4
Shepherd Architecture[Part#2]
devsda edited this page May 29, 2019
·
4 revisions
Shepherd is a highly scalable workflow management Engine to create, schedule and monitor workflows programmatically in any language. It can support conditional as well as un-conditional workflows. It is a Software as a service, where clients can register themselves, create multiple endpoints, and execute workflows in any programming language, or can use multiple programming languages in a single workflow.
- This is the very first step. This is the API through which client can register himself in Shepherd.
- While registering, client has to pass “clientName”. [Currently, skipping security discussion. Will prioritise in V2 cut.]
- This API helps to create new endpoint. There can be multiple endpoints against single client. For example, for client -> “data_platform”, possible endpoints -> “data_platform_beta”, “data_platform_staging”, “data_platform_prod”, “hitesh_dev_data_platform”, etc.
- To setup endpoint, client has to share workflow graph (in xml format), and corresponding endpoint details (in son format).
- This API helps to execute workflow.
- For executing workflow, client has to pass clientName, endpoint, initial payload. Shepherd will return objectId (Identifier of workflow in Shepherd).
- Parameters to call, clientName, endpoint, and initial payload.
- This helps to resume workflow from where it failed last time.
- When Shepherd gets request to execute workflow, it generates objectId for the same. Also, it creates execution_id, which is the actual running instance of given objectId (client request).
- One ObjectId can contain multiple executionIds. It depends upon how many times client requested for restart operation.
- Parameters : ObjectId
- This API helps to restart workflow, This means, it kills the current running instance of given objectId, if there is any and start a new running instance.
- By this operation, it increases the total execution counts by 1 for given object id.
- Parameters : ObjectId
- This API helps to kill the workflow. This means, it kills the current running instance of given objectId, if there is any.
- Parameters : ObjectId
- This API helps to get status of given workflow. Possible status -> PROCESSING, COMPLETED, FAILED
- Parameters : Object
- [ Cosmos DB’s maximum document size] https://docs.microsoft.com/en-gb/azure/cosmos-db/sql-api-resources#documents