feat: enhance task management and timeout configurations in multi-age…#1442
Conversation
…nt chat - Added new environment variables for controlling task execution limits, including `SURFSENSE_SUBAGENT_INVOKE_TIMEOUT_SECONDS`, `SURFSENSE_TASK_BATCH_CONCURRENCY`, and `SURFSENSE_TASK_BATCH_MAX_SIZE`. - Updated documentation to reflect new batch processing capabilities for `task` calls, allowing for concurrent execution of multiple subagent tasks. - Improved error handling and receipt generation for deliverables, ensuring consistent feedback on task status. - Refactored middleware to incorporate search space ID for better task management.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
1 participant
…nt chat
SURFSENSE_SUBAGENT_INVOKE_TIMEOUT_SECONDS,SURFSENSE_TASK_BATCH_CONCURRENCY, andSURFSENSE_TASK_BATCH_MAX_SIZE.taskcalls, allowing for concurrent execution of multiple subagent tasks.Description
Motivation and Context
FIX #
Screenshots
API Changes
Change Type
Testing Performed
Checklist
High-level PR Summary
This PR introduces comprehensive task management controls and verification mechanisms for multi-agent chat. The core changes include: adding environment-variable-controlled timeout and concurrency limits for subagent task execution (
SURFSENSE_SUBAGENT_INVOKE_TIMEOUT_SECONDS,SURFSENSE_TASK_BATCH_CONCURRENCY,SURFSENSE_TASK_BATCH_MAX_SIZE); implementing a structured Receipt system that allows the orchestrator to verify mutating operations (deliverables generation, connector actions, KB writes) without relying on LLM paraphrasing; introducing batch-mode task execution (task(tasks=[...])) for concurrent fanout of 3+ independent specialist calls; adding a per-workspace Redis-backed spawn-pause kill switch for operational control; refactoring Celery-backed deliverable tools (podcasts, video presentations) to poll until terminal status rather than returning immediately; and updating all subagent system prompts with shared verification teachings and output contract snippets via an<include snippet="..."/>directive system.⏱️ Estimated Review Time: 1-3 hours
💡 Review Order Suggestion
surfsense_backend/.env.examplesurfsense_backend/app/agents/multi_agent_chat/middleware/main_agent/checkpointed_subagent_middleware/constants.pysurfsense_backend/app/agents/shared/__init__.pysurfsense_backend/app/agents/shared/receipt.pysurfsense_backend/app/agents/shared/receipt_command.pysurfsense_backend/app/agents/shared/deliverable_wait.pysurfsense_backend/app/agents/multi_agent_chat/middleware/main_agent/checkpointed_subagent_middleware/spawn_paused.pysurfsense_backend/app/agents/new_chat/filesystem_state.pysurfsense_backend/app/agents/new_chat/state_reducers.pysurfsense_backend/app/agents/multi_agent_chat/subagents/shared/spec.pysurfsense_backend/app/agents/multi_agent_chat/subagents/shared/md_file_reader.pysurfsense_backend/app/agents/multi_agent_chat/subagents/shared/snippets/__init__.pysurfsense_backend/app/agents/multi_agent_chat/subagents/shared/snippets/output_contract_base.mdsurfsense_backend/app/agents/multi_agent_chat/subagents/shared/snippets/verifiable_handle.mdsurfsense_backend/app/agents/multi_agent_chat/subagents/shared/subagent_builder.pysurfsense_backend/app/agents/multi_agent_chat/middleware/main_agent/checkpointed_subagent_middleware/task_tool.pysurfsense_backend/app/agents/multi_agent_chat/middleware/main_agent/checkpointed_subagent_middleware/middleware.pysurfsense_backend/app/agents/multi_agent_chat/middleware/stack.pysurfsense_backend/app/agents/multi_agent_chat/main_agent/system_prompt/prompts/routing.mdsurfsense_backend/app/agents/multi_agent_chat/main_agent/system_prompt/prompts/tools/task/description.mdsurfsense_backend/app/agents/multi_agent_chat/subagents/builtins/deliverables/system_prompt.mdsurfsense_backend/app/agents/multi_agent_chat/subagents/builtins/deliverables/tools/report.pysurfsense_backend/app/agents/multi_agent_chat/subagents/builtins/deliverables/tools/resume.pysurfsense_backend/app/agents/multi_agent_chat/subagents/builtins/deliverables/tools/podcast.pysurfsense_backend/app/agents/multi_agent_chat/subagents/builtins/deliverables/tools/video_presentation.pysurfsense_backend/app/agents/multi_agent_chat/subagents/builtins/deliverables/tools/generate_image.pysurfsense_backend/app/agents/multi_agent_chat/subagents/connectors/gmail/tools/send_email.pysurfsense_backend/app/agents/multi_agent_chat/subagents/connectors/notion/tools/delete_page.pysurfsense_backend/app/agents/new_chat/middleware/kb_persistence.pysurfsense_backend/app/agents/new_chat/tools/podcast.pysurfsense_backend/app/agents/new_chat/tools/video_presentation.pysurfsense_backend/app/tasks/chat/streaming/handlers/tool_end.pysurfsense_backend/app/tasks/chat/streaming/handlers/tools/deliverables/generate_video_presentation/emission.pysurfsense_backend/app/agents/multi_agent_chat/subagents/builtins/knowledge_base/system_prompt_cloud.mdsurfsense_backend/app/agents/multi_agent_chat/subagents/builtins/knowledge_base/system_prompt_desktop.mdsurfsense_backend/app/agents/multi_agent_chat/subagents/builtins/memory/system_prompt.mdsurfsense_backend/app/agents/multi_agent_chat/subagents/builtins/research/system_prompt.mdsurfsense_backend/app/agents/multi_agent_chat/subagents/connectors/airtable/system_prompt.mdsurfsense_backend/app/agents/multi_agent_chat/subagents/connectors/calendar/system_prompt.mdsurfsense_backend/app/agents/multi_agent_chat/subagents/connectors/clickup/system_prompt.mdsurfsense_backend/app/agents/multi_agent_chat/subagents/connectors/confluence/system_prompt.mdsurfsense_backend/app/agents/multi_agent_chat/subagents/connectors/discord/system_prompt.mdsurfsense_backend/app/agents/multi_agent_chat/subagents/connectors/dropbox/system_prompt.mdsurfsense_backend/app/agents/multi_agent_chat/subagents/connectors/gmail/system_prompt.mdsurfsense_backend/app/agents/multi_agent_chat/subagents/connectors/google_drive/system_prompt.mdsurfsense_backend/app/agents/multi_agent_chat/subagents/connectors/jira/system_prompt.mdsurfsense_backend/app/agents/multi_agent_chat/subagents/connectors/linear/system_prompt.mdsurfsense_backend/app/agents/multi_agent_chat/subagents/connectors/luma/system_prompt.mdsurfsense_backend/app/agents/multi_agent_chat/subagents/connectors/notion/system_prompt.mdsurfsense_backend/app/agents/multi_agent_chat/subagents/connectors/onedrive/system_prompt.mdsurfsense_backend/app/agents/multi_agent_chat/subagents/connectors/slack/system_prompt.mdsurfsense_backend/app/agents/multi_agent_chat/subagents/connectors/teams/system_prompt.mdsurfsense_backend/app/agents/multi_agent_chat/middleware/shared/kb_context_projection.pysurfsense_backend/app/agents/multi_agent_chat/middleware/shared/permissions/ask/request.pysurfsense_backend/app/agents/new_chat/middleware/compaction.pysurfsense_backend/app/agents/new_chat/middleware/doom_loop.pysurfsense_backend/app/agents/new_chat/middleware/permission.pysurfsense_backend/app/services/composio_service.pysurfsense_backend/app/etl_pipeline/etl_pipeline_service.pysurfsense_backend/app/services/gmail/kb_sync_service.pysurfsense_backend/app/services/google_calendar/kb_sync_service.pysurfsense_backend/app/services/jira/kb_sync_service.pysurfsense_backend/app/services/llm_service.pysurfsense_backend/app/services/onedrive/kb_sync_service.pysurfsense_backend/app/tasks/chat/stream_new_chat.pysurfsense_backend/app/utils/document_converters.py