Skip to content

Add bounded wait_for poll tool for MCP workers - #449

Merged
Shearerbeard merged 3 commits into
mainfrom
mshearer/420-wait-for-tool
Jul 28, 2026
Merged

Add bounded wait_for poll tool for MCP workers#449
Shearerbeard merged 3 commits into
mainfrom
mshearer/420-wait-for-tool

Conversation

@Shearerbeard

Copy link
Copy Markdown
Collaborator

Implements the wait_for tool described in #420.

Workers currently sleep blind for a fixed duration or burn model turns on a check/sleep loop. As noted in the issue, this accounts for ~63% of wall-clock time in TerminalBench (7,976s across 436 calls) because the model tends to overestimate. The worker guesses a duration and guesses high.

This adds a polling tool that waits for a caller-supplied condition rather than a fixed timeout. The condition is expressed at call time, so the model has to reason about what done looks like. Stop reasons (matched, settled, timeout) reflect whether it did.

Takes a probe (any MCP tool plus its arguments), a stop condition (matches, not_matches, or quiet_for_sec), and optional poll_sec / max_wait_sec (capped at 300s). Returns the stop reason, last probe output, elapsed seconds, and sample count. A timeout returns the last observation as a normal result, not an error.

Hooked up via additional_tools in build_worker_provider_agent for all providers when shared MCP is available.

Also fixes four log lines in execute_mcp_tool that hardcoded "HTTP Streamable" regardless of actual transport.

Fixes: #420

@Shearerbeard
Shearerbeard requested a review from a team July 26, 2026 19:10
@Shearerbeard
Shearerbeard marked this pull request as draft July 27, 2026 15:50
@Shearerbeard
Shearerbeard force-pushed the mshearer/420-wait-for-tool branch 5 times, most recently from ccdac80 to bf09e65 Compare July 27, 2026 20:54
@Shearerbeard
Shearerbeard marked this pull request as ready for review July 27, 2026 20:55
@Shearerbeard
Shearerbeard force-pushed the mshearer/420-wait-for-tool branch 2 times, most recently from fb39109 to 68d9980 Compare July 27, 2026 23:39
@greptile-apps

greptile-apps Bot commented Jul 27, 2026

Copy link
Copy Markdown

Greptile Summary

This PR adds a wait_for native tool that gives MCP workers a bounded, condition-driven alternative to blind sleeps, addressing the ~63% wall-clock waste identified in TerminalBench. It also corrects four log lines in execute_mcp_tool that hardcoded "HTTP Streamable" regardless of the actual transport.

  • wait_for tool (wait_for.rs): accepts a probe (any MCP tool + args), a stop condition (matches, not_matches, or quiet_for_sec), and optional poll_sec/max_wait_sec (capped at 300 s). The poll loop races each sample against the remaining budget, evaluates the condition before the elapsed check, and returns a structured WaitForOutput with stop reason, last observation, elapsed seconds, and sample count. Errors from probe calls are surfaced as typed WaitForError variants; a timeout is a normal result, not an error.
  • Wiring (orchestrator.rs): a wait_for_tools() closure builds one WaitForTool per provider arm (required because Box<dyn ToolDyn> is not Clone) and passes it through add_all_tools's additional_tools parameter, gated on shared_mcp being present.
  • Test coverage: 14 tests covering matched/not-matched/settled/timeout conditions, ceiling clamping, hanging probes, oversized observations, and wire deserialization; AdvancingSleeper drives tokio::time::advance so tests complete in zero wall-clock time.

Confidence Score: 5/5

Safe to merge; the poll loop correctly enforces the time budget, all probe-error paths are typed and tested, and the orchestrator wiring is gated on MCP availability.

The core poll loop handles every failure mode (probe error, probe hang, oversized output, timeout with/without a prior observation) and the tests exercise all of them with a paused tokio clock. Invariants are encoded in the type system (NonZeroU64, WaitBudget::new enforces poll strictly less than bound). The only findings are doc-comment style nits.

Files Needing Attention: No files require special attention.

Important Files Changed

Filename Overview
crates/aura/src/orchestration/tools/wait_for.rs New 1,315-line polling tool implementing bounded probe-and-wait logic with correct invariant enforcement, seam-injectable sleeper/dispatcher for testing, and comprehensive test coverage including edge cases for clamping, timeouts, quiescence, and hanging probes.
crates/aura/src/orchestration/orchestrator.rs Wires WaitForTool into all six provider arms via a lazy closure that correctly defers construction so each arm gets its own Box instance (required because ToolDyn is not Clone); tool is only added when shared_mcp is present.
crates/aura/src/mcp_tool_execution.rs Four log lines that hardcoded "HTTP Streamable" are corrected to "MCP tool" to accurately reflect all transports; doc comment updated to match.
crates/aura/src/orchestration/tools/mod.rs Adds the wait_for module and re-exports its public surface; no issues.
crates/aura/src/orchestration/mod.rs Re-exports StopReason, WaitForError, WaitForOutput, WaitForTool from the top-level orchestration module; straightforward plumbing, no issues.

Sequence Diagram

sequenceDiagram
    participant Model as Worker Model
    participant WFT as WaitForTool
    participant Eval as ConditionEvaluator
    participant Disp as McpProbeDispatcher
    participant MCP as MCP Server

    Model->>WFT: call wait_for(probe, until, poll_sec, max_wait_sec)
    WFT->>WFT: "WaitForCall::parse() — validate & apply defaults/ceiling"
    loop each poll cycle (until condition met or bound elapsed)
        WFT->>Disp: sample(probe) [raced against remaining budget]
        Disp->>MCP: execute_mcp_tool(tool, args)
        MCP-->>Disp: observation
        Disp-->>WFT: Ok(observation)
        WFT->>Eval: observe(observation, elapsed)
        alt condition held
            Eval-->>WFT: "Some(Matched | Settled)"
            WFT-->>Model: "WaitForOutput { reason: matched/settled }"
        else "elapsed >= bound"
            WFT-->>Model: "WaitForOutput { reason: timeout, last_observation }"
        else probe timed out
            WFT-->>Model: "Err(ProbeTimedOut) or WaitForOutput { reason: timeout }"
        else continue
            Eval-->>WFT: None
            WFT->>WFT: sleep(min(poll, remaining))
        end
    end
Loading

Reviews (2): Last reviewed commit: "docs(mcp): update execute_mcp_tool doc-c..." | Re-trigger Greptile

Comment thread crates/aura/src/orchestration/tools/wait_for.rs Outdated
Comment thread crates/aura/src/orchestration/tools/wait_for.rs Outdated
Comment thread crates/aura/src/orchestration/tools/wait_for.rs
Comment thread crates/aura/src/orchestration/tools/wait_for.rs
@Shearerbeard
Shearerbeard force-pushed the mshearer/420-wait-for-tool branch from 68d9980 to fadc2ee Compare July 28, 2026 00:08
workers can now call wait_for to poll an mcp tool repeatedly until a
stop condition holds, instead of issuing blind keystrokes sleeps.

the tool accepts a probe (any mcp tool with its arguments), a stop
condition (regex match, regex not-match, or quiet period), and
optional poll and bound seconds. it returns the stop reason, last
output, elapsed time, and sample count. a 300s hard ceiling clamps
max_wait_sec silently. timeout is a normal result, not an error.

registration follows the existing additional_tools seam in
build_worker_provider_agent: each provider arm receives a WaitForTool
instance when shared mcp is available, via a closure that constructs
a fresh Box<dyn ToolDyn> per provider variant.

closes #420
execute_mcp_tool is transport-agnostic but its log lines said
"http streamable mcp tool" unconditionally. a stdio mcp call
would log the wrong transport, which misleads operators tracing
execution. replaced with "mcp tool" at all four log sites
(call, completed, cancelled, failed).
the function doc-comment still read "execute an http mcp tool" after
the log-line rename in 333c812. updated to "execute an mcp tool" to
stay consistent with the four log sites changed in that commit.
@Shearerbeard
Shearerbeard force-pushed the mshearer/420-wait-for-tool branch from fadc2ee to afbd4ff Compare July 28, 2026 00:10
@Shearerbeard
Shearerbeard merged commit 2403fe8 into main Jul 28, 2026
10 checks passed
@Shearerbeard
Shearerbeard deleted the mshearer/420-wait-for-tool branch July 28, 2026 18:01
@github-actions github-actions Bot locked and limited conversation to collaborators Jul 28, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE]: Bounded wait_for tool with a caller-supplied stop condition

2 participants