Skip to content

v1.2.0 — context-bloat fixes, loop guards, cache-safe nudges

Choose a tag to compare

@testAccountDeltas testAccountDeltas released this 11 Oct 22:04
· 26 commits to main since this release

Why this release

A long, deeply-investigated live incident (one session ballooned to 570k tokens, 15-53s per turn, 46 minutes stuck in a silent Read-only loop) turned up several real bugs and missing safeguards. This release fixes all of them.

Context bloat (the big one)

  • Auto-compact threshold capped at 300k tokens, regardless of model window size — it used to be 88% of the window (880k on a 1M-context model), letting sessions balloon for way too long before Claude Code ever cleaned up. Measured: 570k tokens at 99.5% cache hit = 15-53s/turn; 65k tokens at a colder 37.5% hit = 3-7s/turn. The slowdown was never about caching — it's raw context size. See the README's new "Auto-compact: why 300k, not 1M" section for the full writeup and diagram.
  • Old tool_result blocks over 64K chars get truncated from the middle (never the most recent turn) before forwarding — pure function of the content, so it doesn't disturb the upstream prompt cache.

Loop guards (new)

Claude Code's Read tool reads in small (~50-100 line) chunks; under sustained repetition the model's reasoning degrades to zero (confirmed live: 212 consecutive turns, 206 of them bare Read, 100% thinking=False/text=False). Two independent detectors now catch this family of failure and nudge the model to stop:

  • A silent read-streak guard — N consecutive Read-only turns with no text/thinking at all.
  • An exact repeat / oscillation detector (k=1/2/3, turn-grouped, canonical-JSON-hashed tool args) — catches identical retries and A/B/C oscillations, independent of the read-streak case.
  • Both nudges are injected as a synthetic trailing user message, not into system — system is the top of the prompt-cache prefix, and mutating it conditionally was itself a cache-buster.
  • On the last empty-turn retry, tools/tool_choice are stripped entirely so the model is forced to answer with visible text instead of another silent tool call.

Thinking budget

  • /effort low raised from 1000 → 2048 tokens (below 2048 is a documented "poisonous" zone that kills Gemini 3.x's reasoning chain entirely).
  • Full budget scale realigned to the native low/medium/high/xhigh/max = 2048/8192/16384/32768/65536 mapping.

Reliability

  • Fixed 4 real bugs in the SSE splice-retry path: cross-splice block-index collisions, a connection-pool poisoning case on partial 429/503 reads, a _req_id race, and a false fallback-text injection on an unterminated stream.
  • Exponential backoff with jitter (±40%, 1s base, 300s cap) replaces fixed-delay retries.
  • 429/503 retry cap raised 3→4.
  • Time-to-first-byte timeout (60s) split from the full-stream timeout (600s) — a dead gateway worker now fails fast instead of tying up a connection for up to 10 minutes.

Docs

New README section explaining the context-window-vs-auto-compact distinction with a diagram, plus mentions of the TODO-tools fix, the UTF-8 streaming fix, and a Claude Desktop setup section.