fix(v1): bound null harness MCP connect at 60s#1977
Merged
Conversation
An unbounded streamable-HTTP handshake can wedge on an intermittent mcp-SDK race and silently burn the rollout as a 0-turn harness_timeout. Fail loud so the error is classified and retryable. Also applies current ruff format to this file (already drifted on main). Co-authored-by: Cursor <cursoragent@cursor.com>
42a8d50 to
595d321
Compare
ApprovabilityVerdict: Approved This is a straightforward defensive timeout addition (60s bound) to prevent indefinite hangs on MCP connection. The Python version bump from 3.10 to 3.11 is necessary for asyncio.timeout. Small, self-contained bug fix with clear intent. You can customize Macroscope's approvability policy. Learn more. |
wait_for runs connect_mcp in a child task; MCP/httpx cancel scopes pushed onto AsyncExitStack then fail on parent-task cleanup. timeout() keeps enter/exit in the same task. Co-authored-by: Cursor <cursoragent@cursor.com>
Contributor
Author
|
Addressed the cancel-scope review: switched from |
Resolves the conflict in verifiers/v1/harnesses/null/program.py: this branch's asyncio.timeout(60) wrap around connect_mcp() lands adjacent to main's --initial-messages-file refactor (#1974). Kept both: the timeout wrap, and main's file-based `initial` (dropping this branch's now-stale `initial = json.loads(os.environ.get("INITIAL_MESSAGES", "[]"))`, which would have silently clobbered the value main's refactor already computed earlier in main()). Also re-ran ruff format: this branch's incidental single-line reformatting of chat()/connect_mcp() no longer matches the pinned ruff (0.15.12) target — main's original multi-line wrapping is what ruff format now produces, confirmed by running ruff format --diff against main's own unmodified file (no changes). Restored it so the diff is limited to the actual timeout fix + the requires-python bump. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
xeophon
previously approved these changes
Jul 14, 2026
The bounding rationale (bounded handshake, 0-turn harness_timeout) restates what the diff and PR description already say. Keep only why asyncio.timeout instead of wait_for: cancel scopes entered onto `stack` must exit in this task. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
250a5c8
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
connect_mcpinasyncio.wait_for(..., timeout=60)so a wedged streamable-HTTP handshake fails loudly instead of eating the whole rollout budget as a 0-turnharness_timeout.main; touching it applies that format (no behavior change beyond the timeout).Test plan
Note
Low Risk
Isolated to the null eval harness startup path; improves failure visibility without touching auth, data, or shared production APIs.
Overview
MCP connect in the null harness is capped at 60 seconds so a stuck streamable-HTTP handshake fails with
asyncio.TimeoutErrorinstead of hanging until the rollout hits a 0-turnharness_timeout.connect_mcpruns insideasyncio.timeout(60)whenmcpServersis configured; the no-MCP path is unchanged. The implementation usesasyncio.timeoutrather thanwait_forso MCP/httpx cancel scopes registered on the sharedAsyncExitStackare torn down in the same task.The inline script metadata bumps
requires-pythonto>=3.11, which is required forasyncio.timeout.Reviewed by Cursor Bugbot for commit 250a5c8. Bugbot is set up for automated code reviews on this repo. Configure here.
Note
Bound null harness MCP connect with a 60s timeout
Wraps the
connect_mcpcall in program.py withasyncio.timeout(60)so MCP initialization fails withasyncio.TimeoutErrorafter 60 seconds instead of waiting indefinitely. Also bumps the script'srequires-pythonfrom>=3.10to>=3.11, which is required forasyncio.timeout.Changes since #1977 opened
Macroscope summarized cf5fa73.