Skip to content

Comments

feat: add WorkflowResult#76

Merged
HairlessVillager merged 1 commit intoFellouAI:developfrom
HairlessVillager:feature/workflow-result
Feb 27, 2025
Merged

feat: add WorkflowResult#76
HairlessVillager merged 1 commit intoFellouAI:developfrom
HairlessVillager:feature/workflow-result

Conversation

@HairlessVillager
Copy link
Contributor

@HairlessVillager HairlessVillager commented Feb 26, 2025

注意:这个 PR 修改了 Eko.execute 的签名,请下游调用者注意适配。

这个 PR 增加了 Workflow Result 的功能,让工作流结束时返回一个结果,包括以下字段:

  1. isSuccessful:指示工作流最终是否执行成功;
  2. summary:对工作流本身的简短的总结;
  3. payload:工作流的产物,目前支持纯文本产物,同时预留了其他格式产物的类型,当工作流需要返回结果时(例如“给我生成一个述职报告”)非空,当工作流不需要返回结果时(例如“帮我打开 OpenAI 的官网”)为空,

以下是实现思路:

  1. 新增WorkflowResult类型,涵盖以上字段;
  2. 新增DocumentAgentToolInput工具,让它实现文书 Agent 工作,负责生成纯文本产物,通过修改工具的 description 来提示模型在合适的时候调用;
  3. 修改SummaryWorkflow工具,新增输入参数isSuccessful,并微调 prompt;
  4. 新增关于context.variables的写入逻辑:DocumentAgentToolInput工具会把返回值写入workflow_transcript键中,SummaryWorkflow工具会把返回值写入workflow_is_successfulworkflow_summary中;
  5. 新增关于context.variables的读取逻辑:WorkflowImpl.execute方法会读取workflow_is_successful, workflow_summary, workflow_transcript的值并组装成WorkflowResult类型的值。

以下是主要的修改:

  1. Eko.execute方法和WorkflowImpl.execute方法的返回类型修改为Promise<WorkflowResult>,不再返回Promise<NodeOutput[]>
  2. 修改了生成工作流时的 system prompt(见createWorkflowPrompts()),使其不再工作流最后加入一个 Workflow Summary 的节点,而且不允许使用summary_workflow工具;
  3. WorkflowGenerator.doGenerateWorkflow方法中,把一个 Workflow Summary 的节点通过硬编码的形式加入工作流;
  4. 新增接口/类型:WorkflowResult, WorkflowTranscript, WorkflowArtifact, DocumentAgentToolInput, DocumentAgentToolOutput
  5. 修改接口:SummaryWorkflowInput新增了一个isSuccessful: boolean的字段;
  6. 新增DocumentAgentTool工具,用来生成工作流产物(纯文本),通过详尽的参数利用大模型生成长文本;

你可以通过以下方式来验证这个 PR:

  1. 构造一个需要输出长文本的任务 prompt(例如Search Sam Altman's information on wikipedia and summarize it, you should only search 1 result)并运行,期望行为是工作流返回值为 WorkflowResult 类型,且isSuccessfultruesummary为对工作流状态的简短总结,payload 为对 Sam Altman 的总结;
  2. 构造一个不需要输出、只有操作的任务 prompt(例如Open the official website of OpenAI for me)并运行,期望行为是工作流返回值为 WorkflowResult 类型,且isSuccessfultruesummary为对工作流状态的简短总结,payload 为空。

NOTE: This PR modifies the signature of Eko.execute. Please pay attention to compatibility for downstream callers.

This PR enhances the functionality of Workflow Result, enabling it to return a result at the end of a workflow. The result includes the following fields:

  1. isSuccessful: Indicates whether the workflow executed successfully in the end;
  2. summary: A brief summary of the workflow itself;
  3. payload: The output of the workflow. Currently, it supports plain text output and reserves types for outputs in other formats. The payload is non-empty when the workflow needs to return a result (e.g., "Generate a performance report for me") and empty when the workflow does not need to return a result (e.g., "Open the official website of OpenAI for me").

Here is the implementation plan:

  1. Add a new WorkflowResult type, encompassing the above fields;
  2. Introduce the DocumentAgentToolInput tool to handle document-related tasks and generate plain text outputs. The tool's description will be modified to prompt the model to invoke it at appropriate times;
  3. Modify the SummaryWorkflow tool by adding an input parameter isSuccessful and slightly adjusting the prompt;
  4. Add logic for writing to context.variables: The DocumentAgentToolInput tool writes the return value to the workflow_transcript key, while the SummaryWorkflow tool writes the return values to workflow_is_successful and workflow_summary;
  5. Add logic for reading from context.variables: The WorkflowImpl.execute method reads the values of workflow_is_successful, workflow_summary, and workflow_transcript and assembles them into a WorkflowResult type value.

Here are the main changes:

  1. The return types of the Eko.execute and WorkflowImpl.execute methods have been changed to Promise<WorkflowResult>, instead of Promise<NodeOutput[]>;
  2. The system prompt when generating a workflow (see createWorkflowPrompts()) has been modified to no longer add a Workflow Summary node at the end of the workflow and to disallow the use of the summary_workflow tool;
  3. In the WorkflowGenerator.doGenerateWorkflow method, a Workflow Summary node is hard-coded into the workflow;
  4. New interfaces/types added: WorkflowResult, WorkflowTranscript, WorkflowArtifact, DocumentAgentToolInput, DocumentAgentToolOutput;
  5. Interface modification: SummaryWorkflowInput has a new field isSuccessful: boolean;
  6. Introduced the DocumentAgentTool to generate workflow outputs (plain text) by leveraging a large model with detailed parameters to generate long texts.

You can verify this PR by:

  1. Creating a task prompt that requires outputting long text (e.g., "Search Sam Altman's information on Wikipedia and summarize it, you should only search 1 result") and running it. The expected behavior is that the workflow returns a WorkflowResult type value with isSuccessful as true, summary as a brief overview of the workflow status, and payload as a summary of Sam Altman;
  2. Creating a task prompt that requires no output and only actions (e.g., "Open the official website of OpenAI for me") and running it. The expected behavior is that the workflow returns a WorkflowResult type value with isSuccessful as true, summary as a brief overview of the workflow status, and payload as empty.

@HairlessVillager
Copy link
Contributor Author

经过下游调用者审查通过,这个 PR 合并。


After being reviewed and approved by downstream callers, this PR will be merged.

@HairlessVillager HairlessVillager merged commit c0cc30b into FellouAI:develop Feb 27, 2025
@HairlessVillager HairlessVillager deleted the feature/workflow-result branch March 4, 2025 14:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant