Skip to content

v0.21.9

Choose a tag to compare

View all tags
@TheArchitectit TheArchitectit released this 19 Aug 19:54
· 17 commits to master since this release
v0.21.9
9710d95

What changed

Fix: the output-headroom gate — compaction now fires BEFORE the request overflows the model window.

The bug (attempt #6, GLM-4.7 at 32k)

The compaction gate judged only INPUT utilization (fires at ~80% of the context window). But vLLM-style providers reject a request when input_tokens + declared_max_tokens > context_window. For a model with a 32000-token window and 20000 maxTokens, requests overflow at ~37% INPUT — so the gate never tripped (compact never) and the session 400-looped:

Error: 400: ... request (33263 tokens) exceeds the available context size (32768 tokens)
Context overflow recovery failed after one compact-and-retry attempt
Compaction failed: Nothing to compact (session too small)

The fix

A percent-based output-headroom pre-fire check, shared between the gate and the live-trim tail cap (single source of truth, context-handler/headroom.ts):

  • Reserve = the model's plausible declared maxTokens (≤95% of the window), else clamp(MEGACOMPACT_OUTPUT_RESERVE_PCT, 0.1–0.95) × window (default 30%).
  • Trip when input + reserve + safetyMargin >= contextWindow.
  • The 0.95 plausibility bound matters: vLLM reserves the FULL declared maxTokens, so the user's real 32000/20000 (62.5%) config must be trusted, not treated as junk.
  • Replay paths (D.2/D.3) re-cap cached trim views against the CURRENT window (a mid-epoch model switch can shrink it).
  • ThrashGuard exempts headroom fires — an overflowed session is unrecoverable, so a headroom-triggered compact is never refused.
  • Tail-cap hardenings (budget floor, PREVENT-PI-002 pair-safe front-drop) are unconditional guardrail fixes.

Percent-based by construction: identical math at 32k / 64k / 128k / 200k / 256k / 512k / 1M / 5M. No hardcoded token constants in the firing path.

New flags (both dashboard-toggleable, Compaction group)

  • MEGACOMPACT_OVERFLOW_HEADROOM — the pre-fire gate (default ON).
  • MEGACOMPACT_OUTPUT_RESERVE_PCT — fallback reserve fraction (default 0.3, clamped 0.1–0.95).

New diag counter: headroomTrips.

Verification

  • 24 new tests (19 headroom + 5 settings round-trip); full suite green (4290 pass / 0 fail).
  • Full QA review via three parallel agent lenses: overflow-scenario matrix (13 window-size cases), PREVENT-PI invariants + thrash/replay interactions, settings/scanner exposure — all PASS after resolving the one flagged MAJOR (flag-scope documentation).

Install: pi update --extensions

Assets 2
Loading