Skip to content

v1.3.2: 30s panic TTFT threshold, early SSE handshake, and 180s upstream timeout

Latest
Latest

Choose a tag to compare

View all tags
@testAccountDeltas testAccountDeltas released this 13 Oct 23:02
· 1 commit to main since this release
v1.3.2
3e12e3f

Headline Fixes in v1.3.2

This release introduces critical streaming resilience guarantees to prevent client-side socket drops (ConnectionResetError 10054) during heavy Google Gemini tool-use buffering and Borg queue latency spikes.

Key Technical Enhancements

  • 30s Panic TTFT Watchdog & Early SSE Handshake:

    • When handling streaming requests (stream: true), if the upstream provider has not emitted response headers within 30.0 seconds, the proxy immediately returns HTTP 200 OK (Content-Type: text/event-stream) and dispatches periodic Anthropic event: ping keepalive frames.
    • This resets Claude Code's hardcoded 45.0s client socket read timeout, preventing silent aborts and redundant duplicate request cascades.
    • The moment upstream headers and response chunks arrive, the proxy transparently splices the stream into the already-open client connection.

  • Upstream Socket Timeout Raised to 180s (from 60s):

    • _TTFT_TIMEOUT is expanded to 180 seconds, allowing the proxy to gracefully survive monolithic functionCall schema buffering on 300k+ token sessions without dropping the connection to the gateway.

  • Zero-Drop Stability on Deep Contexts:

    • Production verified on 800+ turn agentic sessions with recursive subagents. Long Write, Edit, and Chrome DevTools evaluations (such as ttft=34.0s and ttft=50.1s) now complete smoothly without connection resets.

What's Changed

  • setup-agent.py / relay-proxy.py: Add _PANIC_TIMEOUT = 30.0, asynchronous upstream watchdog, early SSE headers dispatch, and increase _TTFT_TIMEOUT = 180.
  • README.md: Document Panic TTFT threshold, 180s socket timeout, and append-only truncation stability.

Full Changelog: v1.3.1...v1.3.2

Assets 2
Loading