fix(anthropic): ping-aware stream watchdog — eliminate NonZeroAgentExitCodeError on large-context agentic runs

[ericleepi314]

Summary

Fixes the dominant failure class in the terminal-bench 2.1 eval: 18/89 trials died with NonZeroAgentExitCodeError (mean 0.58 vs claude-code 0.72, which had zero such crashes). Root-caused to the stream idle watchdog + a Python-SDK behavior.

Root cause (verified at SDK source)

The Anthropic Python SDK drops ping keepalive events — _streaming.py:151 if sse.event == "ping": continue — so the watchdog, which iterates the typed event stream, sees dead air whenever the model works >90s between typed events. In the agentic loop every turn re-sends the growing conversation, so every stream call pays prompt-processing time-to-first-event, which scales toward minutes as context approaches opus-4-8's 1M window (the #730 1M-context fix made this common). The crash duration_ms clustered at 94–100s (just past the flat 90s window) is the fingerprint. Worse, the watchdog's recovery re-issued the request non-streaming, which the SDK refuses outright for opus-class max_tokens ("Streaming is required for operations that may take longer than 10 minutes") → fatal exit 1.

Fix (three complementary layers; TS-parity)

1. Ping-aware liveness (primary). The dropped pings are still bytes on the wire that httpx.Response.num_bytes_downloaded counts (httpx/_models.py:952, incremented per raw chunk in iter_bytes before SSE-decoding; the SDK consumes response.iter_bytes()). On each deadline the watchdog checks byte progress and re-arms instead of firing when bytes advanced — firing only on true dead air. Matches the ping-aware TS watchdog. Zero changes to the accumulation path.
2. Retry the stream, never downgrade to non-streaming. The fatal _fallback_to_chat is deleted; watchdog fire → bounded stream retry (stream_idle_max_attempts, default 3) → StreamIdleTimeout (harness-retryable phrasing) on exhaustion.
3. Two-phase deadline (fallback). First event gets a 300s grace (prompt processing), later events 90s — the time-based behavior when the byte counter is unavailable.
   Plus: explicit chat() per-request timeout (API_TIMEOUT_MS, default 600s, TS getApiTimeoutMs) which also disarms the SDK's >10-min guard for legit non-streaming callers (compaction, session title, judges).

Test plan

• Live re-smoke of 5 previously-crashing tasks (opus-4-8 + subscription + effort high), all of which died with NonZeroAgentExitCodeError in the 0.58 run:
  • dna-assembly → 1.0 PASS, feal-linear-cryptanalysis → 1.0 PASS
  • filter-js-from-html, adaptive-rejection-sampler → complete, reward 0.0, no exception (genuine model misses, not crashes)
  • write-compressor → AgentTimeoutError (the legit slow-task mode claude-code also hits)
  • Zero trials died at a 90s-multiple → the watchdog false-positive is eliminated
• regex-log (a former crash) → 1.0 PASS in an earlier stage
• 119 passing across test_stream_watchdog / test_anthropic_thinking_keeps_stream_alive / test_providers / test_ch04_api_round4 / test_query_extended_thinking / test_context_runtime_limits (byte-progress re-arm + dead-air fire, retry-then-succeed, exhaustion-raises, chat timeout, two-phase transition)
• SDK/httpx mechanism verified at library source
• Critic: reviewed across 4 rounds → "Ship the branch"

🤖 Generated with Claude Code

[ericleepi314]

Superseded by a clean-based branch (main history was rewritten under this branch, making its merge-base stale). Reopening as an identical change from current main.