From e1e34c0ab9f70b1ca86de1b4e954f05bfd9307e5 Mon Sep 17 00:00:00 2001 From: Paul Cornell Date: Tue, 26 Nov 2024 13:10:40 -0800 Subject: [PATCH] Platform: DAG options --- platform/workflows.mdx | 37 ++++++++++++++++++++++++++++++++----- 1 file changed, 32 insertions(+), 5 deletions(-) diff --git a/platform/workflows.mdx b/platform/workflows.mdx index 1962ae72..18b807db 100644 --- a/platform/workflows.mdx +++ b/platform/workflows.mdx @@ -245,12 +245,39 @@ There are two ways to create a custom workflow: - Checking this box reprocesses all documents in the source location on every workflow run. - Unchecking this box causes only new documents that are added to the source location since the last workflow run to be processed on future runs. Previously processed documents are not processed again, even if those documents' contents change. -8. In the pipeline designer, click the **Source** node. In the **Source** pane, select the source location. Then click **Save**. +8. The workflow begins with the following layout: + + + ```mermaid + flowchart LR + Source-->Partitioner-->Destination + ``` + + The following workflow layouts are also valid: + + ```mermaid + flowchart LR + Source-->Partitioner-->Chunker-->Destination + ``` + ```mermaid + flowchart LR + Source-->Partitioner-->Chunker-->Embedder-->Destination + ``` + ```mermaid + flowchart LR + Source-->Partitioner-->Enrichment-->Chunker-->Destination + ``` + ```mermaid + flowchart LR + Source-->Partitioner-->Enrichment-->Chunker-->Embedder-->Destination + ``` + +9. In the pipeline designer, click the **Source** node. In the **Source** pane, select the source location. Then click **Save**. ![Workflow designer](/img/platform/Workflow-Designer.png) -9. Click the **Destination** node. In the **Destination** pane, select the destination location. Then click **Save**. -10. As needed, add more nodes by clicking the plus icon (recommended) or **Add Node** button: +10. Click the **Destination** node. In the **Destination** pane, select the destination location. Then click **Save**. +11. As needed, add more nodes by clicking the plus icon (recommended) or **Add Node** button: ![Add node to workflow](/img/platform/Workflow-Add-Node.png) @@ -269,8 +296,8 @@ There are two ways to create a custom workflow: To delete a node, click that node, and then click the trash can icon above it. -11. Click **Save**. -12. If you did not set the workflow to run on a schedule, you can [run the worklow](#edit-delete-or-run-a-workflow) now. +12. Click **Save**. +13. If you did not set the workflow to run on a schedule, you can [run the worklow](#edit-delete-or-run-a-workflow) now. #### Custom workflow node types