diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md new file mode 100644 index 00000000000..e64d49003db --- /dev/null +++ b/ARCHITECTURE.md @@ -0,0 +1,157 @@ +# High-level view + +At the highest level, the components of a system using Temporal fall into two categories: + +- **User-hosted processes** + + - The user's application communicates with the Temporal server using one of the [Temporal SDKs](https://docs.temporal.io/dev-guide). + - The user runs Temporal [Worker](https://docs.temporal.io/workers) processes. These also use one of the Temporal SDKs and host the user's [Workflow](https://docs.temporal.io/workflows) and [Activity](https://docs.temporal.io/activities) code. + +- **Temporal Server**
+ Users can host and operate the Temporal server and its database themselves, or use [Temporal Cloud](https://temporal.io/cloud). + + +image + +# Workflow lifecycle + +Below we follow a typical sequence of events in the execution of the following very simple workflow: + +``` +myWorkflow() { + result = callActivity(myActivity) + return result +} +``` + +--- + +**1. The User Application uses a Temporal SDK to send a `StartWorkflowExecution` request to the Frontend service.** + +```mermaid +sequenceDiagram +User Application->>Frontend: StartWorkflowExecution +Frontend->> History: StartWorkflowExecution +History ->> Persistence: CreateWorkflowExecution +Persistence ->> Persistence: Persist MutableState and history tasks +Persistence ->> History: Create Succeed +History->>Frontend: Start Succeed +Frontend->>User Application: Start Succeed +loop QueueProcessor + History->>Persistence: GetHistoryTasks + History->>History: ProcessTask + History->>Matching: AddWorkflowTask +end +``` + +- The Frontend Service uses a History Service client to call the [`StartWorkflow` handler](https://github.com/temporalio/temporal/blob/ef49189005b5323c532264287af6c08a447aab8a/service/history/api/startworkflow/api.go#L157). +- This initializes history events with a `WorkflowExecutionStarted` event, and persists new mutable state and a history task (transfer task). +- A [queue processor](https://github.com/temporalio/temporal/blob/ef49189005b5323c532264287af6c08a447aab8a/service/history/history_engine.go#L303) goroutine runs for every history task queue. +- The [transfer task queue processor](https://github.com/temporalio/temporal/blob/ef49189005b5323c532264287af6c08a447aab8a/service/history/queues/queue_immediate.go#L150) adds a Workflow Task in the appropriate task queue in the Matching Service. + +--- + +**2. The Worker dequeues the Workflow Task, advances the workflow execution, and becomes blocked on the Activity call.** + +```mermaid +sequenceDiagram +Worker->>Frontend: PollWorkflowTask +Frontend->>Matching: PollWorkflowTask +History->>Matching: AddWorkflowTask +Matching->>History: RecordWorkflowTaskStarted +History->>Persistence: UpdateWorkflowExecution +Persistence->>Persistence: Update MutableState & Add timeout timer +Persistence->>History: Update Succeed +History->>Matching: Record Succeed +Matching->>Frontend: WorkflowTask +Frontend->>Persistence: GetHistoryEvents +Persistence->>Frontend: History Events +Frontend->>Worker: WorkflowTask +loop Replayer + Worker->>Worker: ProcessEvent +end +``` + +--- + +**3. The Worker sends a `ScheduleActivityTask`; an Activity task is added in the Matching service.** + +```mermaid +sequenceDiagram +Worker ->> Frontend: RespondWorkflowTaskCompleted(ScheduleActivityTask) +Frontend->> History: RespondWorkflowTaskCompleted(ScheduleActivityTask) +History ->> Persistence: UpdateWorkflowExecution +Persistence ->> Persistence: Persist MutableState and history tasks +Persistence ->> History: Update Succeed +History->>Frontend: Respond Succeed +Frontend->>Worker: Respond Succeed +loop QueueProcessor + History->>Persistence: GetHistoryTasks + History->>History: ProcessTask + History->>Matching: AddActivityTask +end +``` + +--- + +**4. The Worker dequeues the Activity task** + +```mermaid +sequenceDiagram +title: Activity task start +Worker->>Frontend: PollActivityTask +Frontend->>Matching: PollActivityTask +History->>Matching: AddActivityTask +Matching->>History: RecordActivityStarted +History->>Persistence: UpdateWorkflowExecution +Persistence->>Persistence: Update MutableState, add timeout timer +Persistence->>History: Update succeed +History->>Matching: Record Succeed +Matching->>Frontend: ActivityTask +Frontend->>Worker: ActivityTask +``` + +--- + +**4. The Worker sends `RespondActivityCompleted` to the History service; a Workflow Task is added to the Matching service** + +```mermaid +sequenceDiagram +SDK->>Frontend: RespondActivityCompleted +Frontend->>History: RespondActivityCompleted +History->>Persistence: UpdateWorkflowExecution +Persistence->>Persistence: Update MutableState & add transfer task +Persistence->>History: Update Succeed +History->>Frontend: Respond Succeed +Frontend->>SDK: Respond Succeed +loop QueueProcessor + History->>Persistence: GetHistoryTasks + History->>History: ProcessTask + History->>Matching: AddWorkflowTask +end +``` + +--- + +**5. The Worker dequeues the Workflow Task, advances the workflow, and finds that it has reached its end** + +\ + +--- + +**6. The Worker sends `RespondWorkflowTaskCompleted` to the History Service** + +```mermaid +sequenceDiagram +SDK->>Frontend: RespondWorkflowTaskCompleted +Frontend->>History: RespondWorkflowTaskCompleted +History->>Persistence: UpdateWorkflowExecution +Persistence->>Persistence: Update MutableState & add tasks (visibility, tiered storage, retention etc) +Persistence->>History: Update Succeed +History->>Frontend: Respond Succeed +Frontend->>SDK: Respond Succeed +loop QueueProcessor + History->>Persistence: GetHistoryTasks + History->>History: ProcessTask (Update visibility, Upload to S3, Delete data etc) +end +``` diff --git a/README.md b/README.md index 46d2b3caea8..b0f813f4c7e 100644 --- a/README.md +++ b/README.md @@ -6,18 +6,32 @@ [go-report-image]: https://goreportcard.com/badge/github.com/temporalio/temporal [go-report-url]: https://goreportcard.com/report/github.com/temporalio/temporal -# Temporal +# Temporal -Temporal is a microservice orchestration platform which enables developers to build scalable applications without sacrificing productivity or reliability. -Temporal server executes units of application logic, Workflows, in a resilient manner that automatically handles intermittent failures, and retries failed operations. +Temporal is a durable execution and service orchestration platform that enables developers to build scalable applications without sacrificing productivity or reliability. +The Temporal server executes units of application logic called Workflows in a resilient manner that automatically handles intermittent failures, and retries failed operations. -Temporal is a mature technology, a fork of Uber's Cadence. -Temporal is being developed by [Temporal Technologies](https://temporal.io/), a startup by the creators of Cadence. +Temporal is a mature technology that originated as a fork of Uber's Cadence. +It is developed by [Temporal Technologies](https://temporal.io/), a startup by the creators of Cadence. -[![image](https://user-images.githubusercontent.com/12602502/136433917-98abe0d7-4f81-4f97-9b11-62b331c76608.png)](http://www.youtube.com/watch?v=f-18XztyN6c "Temporal") +[![image](https://user-images.githubusercontent.com/12602502/136433917-98abe0d7-4f81-4f97-9b11-62b331c76608.png)](http://www.youtube.com/watch?v=f-18XztyN6c 'Temporal') + +# High-level view + +At the highest level, the components of a system using Temporal fall into two categories: + +- **User-hosted processes** + + - The user's application communicates with the Temporal server using one of the [Temporal SDKs](https://docs.temporal.io/dev-guide). + - The user runs Temporal [Worker](https://docs.temporal.io/workers) processes. These also use one of the Temporal SDKs and host the user's [Workflow](https://docs.temporal.io/workflows) and [Activity](https://docs.temporal.io/activities) code. + +- **Temporal Server**
+ Users can host and operate the Temporal server and its database themselves, or use [Temporal Cloud](https://temporal.io/cloud). Learn more about Temporal at [docs.temporal.io](https://docs.temporal.io). +For description of Temporal internals see [ARCHITECTURE.md](./ARCHITECTURE.md) + ## Getting Started ### Download and Start Temporal Server Locally