fix(orchestrator): route review-before-finalize tasks to synchronous delegation#4682
Conversation
…delegation For 'draft a bio, then have a second agent critique it before you finalize', the orchestrator dispatched the critique via fire-and-forget spawn_async_subagent. The turn finalized immediately and the critique ran as a detached autonomous_task ~10 min later — so the parent never honored 'before you finalize' and burned a wasted run. The orchestrator has no synchronous spawn_subagent; its awaited primitives are spawn_parallel_agents (collects results before returning), blocking delegate_* specialists, and spawn_async_subagent + wait_subagent. Neither the async tool description nor the prompt named the result-gating pattern, so the model mis-routed. Add an explicit 'result-gating tasks run synchronously' rule to the orchestrator prompt and to the spawn_async_subagent description, steering review/critique/verify-before-finalize work to an awaited sub-agent. Add a prompt regression test.
📝 WalkthroughWalkthroughUpdates the ChangesResult-gating delegation rule
Estimated code review effort: 1 (Trivial) | ~5 minutes Possibly related PRs
Suggested labels: Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Warning Review ran into problems🔥 ProblemsGit: Failed to clone repository. Please run the Comment |
There was a problem hiding this comment.
🧹 Nitpick comments (1)
src/openhuman/agent_registry/agents/orchestrator/prompt.rs (1)
381-397: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick winSecond assertion is weakly coupled to the actual rule change.
The first
assert!(Line 388) meaningfully pins the new rule text. Butspawn_parallel_agentsandwait_subagentalready appear elsewhere inprompt.mdprior to this change (e.g. the pre-existing "Async background sub-agents" section), so the second assertion (Lines 393-396) would still pass even if the new rule text omitted these primitives entirely or named different tools. It doesn't actually verify these terms occur within/near the new rule.Consider tightening this to check the terms appear within the rule's paragraph specifically (e.g. by locating the substring starting at "Result-gating tasks run synchronously" and asserting the alternatives are within that slice).
♻️ Proposed tightening
- assert!( - ARCHETYPE.contains("Result-gating tasks run synchronously"), - "orchestrator prompt must carry the result-gating delegation rule" - ); - // It must steer such tasks to a synchronous/awaited primitive rather - // than fire-and-forget `spawn_async_subagent`. - assert!( - ARCHETYPE.contains("spawn_parallel_agents") && ARCHETYPE.contains("wait_subagent"), - "the rule must name the synchronous/awaited alternatives" - ); + let rule_start = ARCHETYPE + .find("Result-gating tasks run synchronously") + .expect("orchestrator prompt must carry the result-gating delegation rule"); + let rule_slice = &ARCHETYPE[rule_start..]; + // It must steer such tasks to a synchronous/awaited primitive rather + // than fire-and-forget `spawn_async_subagent`, within the rule itself. + assert!( + rule_slice.contains("spawn_parallel_agents") && rule_slice.contains("wait_subagent"), + "the rule must name the synchronous/awaited alternatives" + );🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/openhuman/agent_registry/agents/orchestrator/prompt.rs` around lines 381 - 397, The test in prompt_routes_result_gating_tasks_to_synchronous_delegation is too loosely coupled because it checks ARCHETYPE globally for spawn_parallel_agents and wait_subagent, which may pass due to unrelated existing prompt text. Tighten the assertion by isolating the paragraph/substring that begins with the new "Result-gating tasks run synchronously" rule in ARCHETYPE, then verify the alternative primitives appear within that same slice; keep the existing function name and regression intent while making the check specific to the updated rule text.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@src/openhuman/agent_registry/agents/orchestrator/prompt.rs`:
- Around line 381-397: The test in
prompt_routes_result_gating_tasks_to_synchronous_delegation is too loosely
coupled because it checks ARCHETYPE globally for spawn_parallel_agents and
wait_subagent, which may pass due to unrelated existing prompt text. Tighten the
assertion by isolating the paragraph/substring that begins with the new
"Result-gating tasks run synchronously" rule in ARCHETYPE, then verify the
alternative primitives appear within that same slice; keep the existing function
name and regression intent while making the check specific to the updated rule
text.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: b410925e-1069-4679-82cd-9214576d8788
📒 Files selected for processing (3)
src/openhuman/agent_orchestration/tools/spawn_async_subagent.rssrc/openhuman/agent_registry/agents/orchestrator/prompt.mdsrc/openhuman/agent_registry/agents/orchestrator/prompt.rs
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5f2e3eb8cd
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| later unused. Instead run it and get the result **in this same turn**: a blocking `delegate_*` | ||
| specialist, or `spawn_parallel_agents` (it collects every worker's result before returning), | ||
| or — only if you already spawned async — `wait_subagent` with a generous `timeout_secs` and |
There was a problem hiding this comment.
Allow single-agent waits for gated reviews
For single result-gating requests like the #4681 repro (“draft a bio, then have a second agent critique it before finalizing”), this guidance leaves no valid awaited path: spawn_parallel_agents only accepts two or more independent tasks (its schema/validator rejects fewer than 2), and arbitrary prose critiques often have no matching delegate_* specialist. Since wait_subagent can collect a just-spawned async child in the same turn, making it usable only after the model has already used async can still push the orchestrator into a tool error or into skipping the requested second-agent review.
Useful? React with 👍 / 👎.
|
CI + review pass — LGTM. Reviewed Checks: |
For "draft a bio, then have a second agent critique it before you finalize", the orchestrator dispatched the critique via fire-and-forget
spawn_async_subagent. The turn finalized immediately and the critique ran as a separateautonomous_task~10 min later — so the parent never honored "before you finalize" and burned a wasted detached run.Why it mis-routed: the orchestrator has no synchronous
spawn_subagent; its awaited primitives arespawn_parallel_agents(collects results before returning), blockingdelegate_*specialists, andspawn_async_subagent+wait_subagent. Neither the async tool description nor the prompt named the result-gating ("review/critique/verify before finalize") pattern, so a gating step slipped through as "background work".Fix (guidance-only, no routing refactor):
orchestrator/prompt.md: new "Result-gating tasks run synchronously (hard rule)" — such work must run in the same turn via a blockingdelegate_*,spawn_parallel_agents, orwait_subagent; never fire-and-forget.spawn_async_subagenttooldescription(): explicitly excludes result-gating tasks and points to the awaited alternatives.Closes #4681
Summary by CodeRabbit
Bug Fixes
Tests