v1.3.2: 30s panic TTFT threshold, early SSE handshake, and 180s upstream timeout

Headline Fixes in v1.3.2

This release introduces critical streaming resilience guarantees to prevent client-side socket drops (ConnectionResetError 10054) during heavy Google Gemini tool-use buffering and Borg queue latency spikes.

---

Key Technical Enhancements

• 30s Panic TTFT Watchdog & Early SSE Handshake:

  • When handling streaming requests (stream: true), if the upstream provider has not emitted response headers within 30.0 seconds, the proxy immediately returns HTTP 200 OK (Content-Type: text/event-stream) and dispatches periodic Anthropic event: ping keepalive frames.
  • This resets Claude Code's hardcoded 45.0s client socket read timeout, preventing silent aborts and redundant duplicate request cascades.
  • The moment upstream headers and response chunks arrive, the proxy transparently splices the stream into the already-open client connection.

• Upstream Socket Timeout Raised to 180s (from 60s):

  • _TTFT_TIMEOUT is expanded to 180 seconds, allowing the proxy to gracefully survive monolithic functionCall schema buffering on 300k+ token sessions without dropping the connection to the gateway.

• Zero-Drop Stability on Deep Contexts:

  • Production verified on 800+ turn agentic sessions with recursive subagents. Long Write, Edit, and Chrome DevTools evaluations (such as ttft=34.0s and ttft=50.1s) now complete smoothly without connection resets.

---

What's Changed

• setup-agent.py / relay-proxy.py: Add _PANIC_TIMEOUT = 30.0, asynchronous upstream watchdog, early SSE headers dispatch, and increase _TTFT_TIMEOUT = 180.
• README.md: Document Panic TTFT threshold, 180s socket timeout, and append-only truncation stability.

Full Changelog: v1.3.1...v1.3.2

v1.3.1 — 550k context window, auxiliary fast-path, and multi-tenant loop guard

Why this release

Real-world usage under heavy multi-step planning and deep code-base exploration demonstrated two operational bottlenecks:

1. The historical 300k auto-compact threshold artificially severed active working chains mid-investigation, causing lossy context summarization and forced re-reading of source files.
2. Background auxiliary calls generated by Claude Code (request-class=auxiliary for branch titles and summaries) inherited the global effort=xhigh setting, triggering 20s+ deep hidden reasoning bursts on trivial 40-character responses.

This release raises the context compaction ceiling to 550k tokens based on reverse-engineered agy.EXE checkpoint heuristics, adds an automatic fast-path for auxiliary calls, isolates loop guards per subagent, and enables transparent reconnect retries.

550k Context Compaction Ceiling

• Expanded Work Window: Increased CLAUDE_CODE_AUTO_COMPACT_WINDOW from 300k to 550,000 tokens (with CLAUDE_CODE_MAX_CONTEXT_TOKENS=1000000).
• Validated Performance: Measured live across a 793-turn session reaching 352k+ tokens with 99.3% cache hit rates and sub-10s TTFT (8.5–9.7s). Demonstrates that transformer attention latency issues were previously driven by retrospective cache invalidation rather than context depth.
• Eliminates premature lossy compression and preserves ongoing test and exploration history across complex multi-file tasks.

Fast-Path for Auxiliary Requests

• Automatic header-aware routing (x-claude-code-request-class: auxiliary):
  • Model forced to *-low tier with explicit (2048) budget (aux:low).
  • Completely eliminates 18–25s Borg reasoning pauses on background metadata queries, reducing latency to ~1.0–1.5s.

Multi-Tenant Loop Guard

• Isolated Tracking: Streak counters for silent Read loops (LOOP_STREAK_THRESHOLD = 10) and exact tool-call repetitions (_toolcall_hists) are now keyed by agent_key ("main", "sub:<agent-type>").
• Parallel subagents (e.g., 4 concurrent general-purpose workers) maintain independent streak states without false-positive crosstalk.
• When an individual subagent enters an unproductive slice-reading pattern, it receives a targeted synthetic stop-nudge to course-correct without disturbing siblings.

Upstream Reconnect Retry on ConnectionResetError

• Transparent in-stream retry: if Google upstream drops the TCP connection (ConnectionResetError 10054) before content chunks arrive (client received only SSE keepalive pings), the proxy automatically reopens upstream and continues generation within the same client connection.

Claude Desktop Adaptive Opus Integration

• Mapped claude-opus-4-8 to gemini-3.8-flash-tiered in MODEL_MAP.
• Claude Desktop /effort slider now dynamically controls native ModeAuto (xhigh/max), *-high, *-medium, and *-low.
• Updated claude-desktop-import.json generator.

Upstream Cluster Switch Telemetry

• Logs now emit CACHE_MISS(cold_prefill) whenever in_tok >= 50000 with cold cache (cache_read < 50k), providing immediate visibility into upstream TPU worker reallocations.

Upgrading

python setup-agent.py

Safe to re-run on an existing install. Existing sessions can immediately take advantage of the 550k window via launch.cmd.

v1.3.0 — native adaptive ModeAuto on xhigh, continuous SSE heartbeat, output telemetry

Why this release

Under real-world workloads, Google Antigravity / Borg TPU infrastructure experiences queue and scheduling spikes resulting in TTFT (time-to-first-token) pauses of 30s to 66s. Without keepalive frames on the wire, client socket timeouts (in Node.js/undici) trigger silent connection drops (BrokenPipeError: [Errno 32] Broken pipe in the proxy when upstream finally replies). Furthermore, hardcoding budget limits on the gateway's adaptive tiered models (gemini-3.8-flash-tiered) disabled Google's native ModeAuto thinking logic, forcing excessive reasoning on trivial turns while hiding actual token usage.

This release eliminates BrokenPipeError entirely, unlocks Google's native dynamic reasoning, and adds transparent reasoning telemetry.

Native adaptive thinking on /effort xhigh

• gemini-3.8-flash-tiered without bracket suffixes now routes cleanly to Google Antigravity ModeAuto (thinkingBudget = -1). When no artificial token cap (like (16384)) is attached, the model itself dynamically evaluates task complexity: spending zero reasoning tokens on routine file reads and scaling to deep reasoning on architectural analysis.
• Dynamic -tiered model resolver: Claude Code's /effort setting dynamically selects the underlying backend model:
  • low → *-low(2048)
  • medium → *-medium(8192)
  • high → *-high(16384)
  • xhigh / max → clean gemini-3.8-flash-tiered (ModeAuto).
• Default effortLevel in settings.json and setup-agent.py is now set to xhigh.

Continuous SSE Keepalive Ping Heartbeat (fixes BrokenPipeError)

• Automatic event: ping ticker (5.0s interval) keeps the client connection alive across long Borg queue waits and heavy reasoning spikes.
• Continuous across the entire stream: unlike simple TTFT-only guards, the idle timer resets on every received chunk and keeps running mid-stream. If generation pauses mid-response, keepalive frames continue to flow. Thread-safe lock (wfile_lock) ensures ping events and response frames never collide.
• Tested and verified live under upstream delay simulations and real 30s+ Borg spikes (pings=4, zero connection drops).

Granular output telemetry & hidden reasoning accounting

• Proxy logging now decomposes total output into visible vs hidden tokens:
  out_tok=3439(vis~1835,hid~1604)
  • vis (visible tokens): estimated tokens across generated text (text_chars), streamed reasoning (think_chars), and tool call JSON (tool_chars).
  • hid (hidden tokens): internal Borg/TPU reasoning tokens accounted for by Google's usageMetadata (thoughtsTokenCount) that are not streamed as visible deltas.
• Logs include accurate character counts (think_chars, text_chars, tool_chars) and ping counts (pings=N).
• Fixed capture of message_delta frames to ensure terminal usage and stop_reason (end_turn/tool_use) are consistently recorded.

Upgrading

python setup-agent.py

Safe to re-run on an existing install.

v1.2.1 — fix Gemini Grep/Glob InputValidationError

Why this release

Gemini (via this gateway) reliably mishandles Claude Code's multi-field Grep/Glob tool schemas — it puts the search string under description instead of pattern, or drops required fields, producing a hard failure on essentially every search:

Error: InputValidationError: Grep failed due to the following issues:
The required parameter `pattern` is missing
An unexpected parameter `description` was provided

This isn't occasional — it's the default outcome any time the model tries to search, confirmed live. The same root cause was already diagnosed and fixed once before in a sibling project (opencode, same Gemini backend): Gemini reliably handles one general shell tool (a free-text command string) but fumbles mapping intent into several named fields. Denying the specialized tools and forcing shell-only fixed it there 3/3 with zero errors — same fix applied here.

What changed

• Grep and Glob are stripped from the outgoing tools array on every request, unconditionally, with an OS-aware system hint telling the model to search via the shell instead. Unconditional and deterministic, so it never disturbs the upstream prompt cache.
• Native PowerShell tool enabled (CLAUDE_CODE_USE_POWERSHELL_TOOL=1, a Claude Code preview feature) instead of routing through Bash → powershell.exe/grep. A/B tested live on Windows: the Bash-wrapped route needed up to 5 tool calls/21s per search due to quoting and encoding friction (Bash expanding $ variables meant for PowerShell, garbled Cyrillic output); the native tool resolved the same searches in exactly 1 tool call every time, 3/3 runs, zero errors — no Bash intermediary means no quoting collision to get wrong in the first place.
  • Windows: always enabled — powershell.exe is guaranteed present.
  • macOS/Linux: this flag is cross-platform, but needs PowerShell 7+ (pwsh) in PATH, which isn't installed by default. The installer now detects pwsh at install time (shutil.which) and enables the flag only when found — if you already have it (brew install powershell), you get the same reliability improvement; otherwise it falls back to the existing grep/find shell hint, unchanged.
• The shell-hint fallback (used when the native tool isn't available) warns about Bash/PowerShell quoting ($ variables need single-quoting the whole -Command argument) and prefixes a UTF-8 console-encoding setup for non-ASCII text.

Recommended action

If you've already installed this tool, run /clear in your next Claude Code session — old conversation history containing tool_use: Grep/Glob calls can still confuse the model even after the tool is hidden. Re-running the installer (or just restarting Claude Code after a git pull) picks up the new launcher settings.

v1.2.0 — context-bloat fixes, loop guards, cache-safe nudges

Why this release

A long, deeply-investigated live incident (one session ballooned to 570k tokens, 15-53s per turn, 46 minutes stuck in a silent Read-only loop) turned up several real bugs and missing safeguards. This release fixes all of them.

Context bloat (the big one)

• Auto-compact threshold capped at 300k tokens, regardless of model window size — it used to be 88% of the window (880k on a 1M-context model), letting sessions balloon for way too long before Claude Code ever cleaned up. Measured: 570k tokens at 99.5% cache hit = 15-53s/turn; 65k tokens at a colder 37.5% hit = 3-7s/turn. The slowdown was never about caching — it's raw context size. See the README's new "Auto-compact: why 300k, not 1M" section for the full writeup and diagram.
• Old tool_result blocks over 64K chars get truncated from the middle (never the most recent turn) before forwarding — pure function of the content, so it doesn't disturb the upstream prompt cache.

Loop guards (new)

Claude Code's Read tool reads in small (~50-100 line) chunks; under sustained repetition the model's reasoning degrades to zero (confirmed live: 212 consecutive turns, 206 of them bare Read, 100% thinking=False/text=False). Two independent detectors now catch this family of failure and nudge the model to stop:

• A silent read-streak guard — N consecutive Read-only turns with no text/thinking at all.
• An exact repeat / oscillation detector (k=1/2/3, turn-grouped, canonical-JSON-hashed tool args) — catches identical retries and A/B/C oscillations, independent of the read-streak case.
• Both nudges are injected as a synthetic trailing user message, not into system — system is the top of the prompt-cache prefix, and mutating it conditionally was itself a cache-buster.
• On the last empty-turn retry, tools/tool_choice are stripped entirely so the model is forced to answer with visible text instead of another silent tool call.

Thinking budget

• /effort low raised from 1000 → 2048 tokens (below 2048 is a documented "poisonous" zone that kills Gemini 3.x's reasoning chain entirely).
• Full budget scale realigned to the native low/medium/high/xhigh/max = 2048/8192/16384/32768/65536 mapping.

Reliability

• Fixed 4 real bugs in the SSE splice-retry path: cross-splice block-index collisions, a connection-pool poisoning case on partial 429/503 reads, a _req_id race, and a false fallback-text injection on an unterminated stream.
• Exponential backoff with jitter (±40%, 1s base, 300s cap) replaces fixed-delay retries.
• 429/503 retry cap raised 3→4.
• Time-to-first-byte timeout (60s) split from the full-stream timeout (600s) — a dead gateway worker now fails fast instead of tying up a connection for up to 10 minutes.

Docs

New README section explaining the context-window-vs-auto-compact distinction with a diagram, plus mentions of the TODO-tools fix, the UTF-8 streaming fix, and a Claude Desktop setup section.

v1.1.0 — Claude Code 2.1.296 compatibility + UTF-8 streaming fix

Fix (critical)

Task planning tools were silently disabled after updating to Claude Code 2.1.296. Since 2.1.268, Claude Code hides TaskCreate/TaskUpdate/TaskList/TodoWrite from any model name it doesn't recognize as a known Claude model — our gateway models are all gemini-*, so they fell outside the whitelist and lost task planning on update, with no error or warning.

Fix: CLAUDE_CODE_ENABLE_TODO_TOOLS=1, the documented escape hatch, added to both launchers. Verified live: the tools are back.

Also in this release

• Fixed UTF-8 corruption at SSE stream boundaries. The proxy decoded each raw read(8192) chunk independently; a multi-byte character (Cyrillic, etc.) landing on a chunk boundary got corrupted into replacement characters (visible as garbled or "eaten" text in responses). Fixed with an incremental UTF-8 decoder. Verified both with a forced mid-character split and a live 52KB Cyrillic stream — zero corruption.
• CLAUDE_CODE_GATEWAY_HINT_HEADERS=1 enabled, plus proxy logging of agent-type/request-class headers and a thinking= field in the SSE-done log line — subagent requests are now distinguishable from the parent session in the proxy log. Verified live: subagents correctly receive and apply their own explicit /effort (medium vs high), independent of the parent session.
• ultracode effort value mapped to the same capped budget as high/xhigh/max.

Upgrading

python setup-agent.py

Safe to re-run on an existing install.

Full Changelog: v1.0.1...v1.1.0

v1.0.1 — subagents fixed (CLAUDE_CODE_SIMPLE was silently disabling them)

Fix

Subagents (Agent/Task tool) now actually work. Claude Code would say "launching a subagent" and then just do the work itself — the tool was never offered to the model in the first place.

Root cause (found by reverse-engineering the compiled claude.exe, undocumented): CLAUDE_CODE_SIMPLE=1 silently drops ~11 tools from the list sent to the model, including Agent, Write, Grep, Glob, WebFetch, WebSearch. Removing it alone breaks login (Not logged in) because ANTHROPIC_API_KEY needs an interactive "Do you want to use this API key? No (recommended)" confirmation that nothing can answer headlessly — which SIMPLE was silently working around.

Fix: drop CLAUDE_CODE_SIMPLE, switch ANTHROPIC_API_KEY → ANTHROPIC_AUTH_TOKEN (the gateway-token variable, confirmed in the binary's own strings, no confirmation prompt).

Verified live: 44 tools offered to the model (was 33), and it actually invokes tool_use: Agent (was 0).

Also in this release

• Proxy log now rotates at 512KB, keeps max 3 backups instead of growing forever.
• README leads with why this proxy exists, before the implementation details.

Upgrading

python setup-agent.py

Safe to re-run on an existing install.

Full Changelog: v1.0.0...v1.0.1

v1.0.0 — session cache, effort-controlled thinking, connect reliability

Highlights

• Session-sticky account routing — a conversation stays pinned to one upstream account, so Gemini's server-side prefix cache stays warm instead of going cold every time traffic round-robins to a different account.
• /effort now actually controls thinking — Claude Code/Desktop send the reasoning effort in output_config.effort, which the Gemini gateway path was ignoring. The proxy now reads it and maps it to a thinking budget: low=1000, medium=4000, high=24576 (capped, not unbounded — avoids runaway "still thinking" stalls).
• Short connect-timeout (4s) + more retries for dead-SYN connection blips — a blip now costs ~4s instead of the ~21s OS default, with 5 retries at 0.5s backoff.
• Claude Desktop import file — the installer now writes claude-desktop-import.json (gateway URL, key, 1m context, claude-* model names) ready to paste into Desktop's third-party gateway settings.
• Model picker switched from fake per-tier model names to real model versions (now that /effort handles depth).

Verified

Live-tested on a real session: 605 turns, avg 4.1s per turn, 0 unrecovered errors, worst case ~30s (down from 1.5h+ stalls before this release).

Upgrading

python setup-agent.py

Safe to re-run on an existing install — it only rewrites the managed proxy/config files.