Feature/newnew workflow loop node

[Woo0ood]

Summary

This PR introduces the Loop Node, a feature designed to enable workflows or processes to execute repeatedly until specific conditions or formats are met. The node allows users to define internal variables and outputs that determine whether to exit the loop. A maximum iteration limit can also be set to prevent infinite loops.

The addition of the Loop Node addresses the community’s and team’s demand for executing repeated HTTP interface calls or invoking LLMs multiple times in a workflow. The implementation aligns with the feedback and requirements outlined in Issue #12354.

Resolves #12354
Close #12289

Screenshots

Some Details

This section was automatically generated by GitHub Copilot actions.

This pull request introduces a new feature for handling single loop runs in workflows and advanced chat applications. The changes include adding new API endpoints, updating the app generator and runner classes, and modifying the task pipeline to support loop events.

New API endpoints:

• Added Advanc /edChatDraftRunLoopNodeApi and WorkflowDraftRunLoopNodeApi classes to handle POST requests for running draft workflow loop nodes. (api/controllers/console/app/workflow.py)

App generator updates:

• Introduced single_loop_generate method in AdvancedChatAppGenerator and WorkflowAppGenerator to generate app responses for single loop runs. (api/core/app/apps/advanced_chat/app_generator.py) [1] (api/core/app/apps/workflow/app_generator.py) [2]

App runner updates:

• Updated AdvancedChatAppRunner and WorkflowAppRunner to handle single loop runs by adding logic to fetch and process graph and variable pool for loops. (api/core/app/apps/advanced_chat/app_runner.py) [1] (api/core/app/apps/workflow/app_runner.py) [2]

Task pipeline modifications:

• Added new events (QueueLoopStartEvent, QueueLoopNextEvent, QueueLoopCompletedEvent, QueueNodeInLoopFailedEvent) and updated the _process_stream_response method to handle these events. (api/core/app/apps/advanced_chat/generate_task_pipeline.py) [1] [2] [3] (api/core/app/apps/workflow/generate_task_pipeline.py) [4] [5] [6]

These changes collectively enhance the system's ability to manage and execute single loop runs within workflows and advanced chat applications.

Author

This PR is submitted by the AICT Team from NetDragon Websoft Inc., aiming to contribute to the open-source community. The AICT Team is dedicated to promoting the continuous optimization and functional expansion of Dify and other open-source products. By introducing practical features such as the Loop Node, the team hopes to help community users build more efficient and intelligent workflow solutions, working together to create a stronger Dify ecosystem.

Checklist

Important

Please review the checklist below before submitting your pull request.

• This change requires a documentation update, included: Dify Document
• I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!)
• I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
• I've updated the documentation accordingly.
• I ran dev/reformat(backend) and cd web && npx lint-staged(frontend) to appease the lint gods

[crazywoola]

2025-03-04.22.37.46.mov

Suggestion:

• See attached video, The LOOP TERMINATION CONDITION should be clickable and pointer-style=cursor, it can remove current condition.
• MAXIMN LOOP COUNT is set to 100 at this moment, it's good to make it as an environment variable in .env.example

[Woo0ood]

• The LOOP TERMINATION CONDITION should be clickable and pointer-style=cursor

I have addressed the first suggestion. The second suggestion will be handled later (maybe🌚..).

---

append: all done

[crazywoola]

Good job!

[crazywoola]

• Loop Node Description
  • The loop termination condition must be the return variable within the loop body.
  • LOOP_NODE_MAX_COUNT=100 can be modified via an environment variable. However, note that Run failed: Max steps 500 reached. requires modifying the WORKFLOW_MAX_EXECUTION_STEPS environment variable.