v0.12.7 — one reply reserve: context budgets now measure the window your server actually serves
Fix release for the context-budget spine, driven by field report #145 (thanks @mjdufresne).
What was wrong
initwroteserved_window − 4,096into new configs while the runtime subtracted another 4,096 internally — every init-written config ran two reserves short of the window it named.- Compaction's 0.80/0.95 triggers measured the raw window while the emergency floor measured
window − 4,096. On every window under ~82K (that is every llama.cpp/Ollama/LM Studio default) the last-resort hard truncation could become eligible before the 0.95 compaction stage — the designed rescue never got its turn. - Fixing the double reserve exposed a latent gap the over-reservation had been masking: requested output tokens were never fitted to the window, so small-window servers could reject requests mid-session (vLLM validates
prompt + max_tokens ≤ max_model_lenand 400s).
What changed
max_context_tokensnow means the full served window. The reply reserve is subtracted internally in one shared function (response_reserve), and every stage — eviction, 0.80, 0.95, the emergency floor — measures the same effective limit. The ordering inversion is structurally impossible now, at every window size.- Output tokens are clamped to that same reserve wherever the window is known (session start,
/modelswap, benchnum_ctxpins): input + output fit the window by construction. - A served window too small to hold any reply (≤ 1,024 tokens) is refused loudly at startup instead of starting and failing on the first turn;
initdeclines to adopt one and says why. Practical minimum: 1,025 tokens. - Repeated emergency-floor fires within one turn escalate to a distinct log line with the fire count, the window, the overshoot, and what to change — instead of N identical ERROR lines.
- Context percentages (the REPL
ctxreading, heartbeats, compaction events) now read against the usable budget — a few points higher for the same history.
Honest scope
This fixes the sizing/ordering half of #145. The deeper re-fire mechanism — per-turn compaction output is not persisted, so a long tool-heavy turn can grow back toward the floor — is an architecture item and stays open on #145.
Known wart, documented rather than hidden: the reserve curve steps at ~12K, so windows in 12,288–14,847 get slightly less usable budget than 12,287 does. Safe on both sides (more conservative, never an error); smoothing it changes real 16K-window behavior, so it is a deliberate follow-up, not part of this fix.
Migration
Nothing breaks. A config an older init wrote (served − 4,096) still works — it is just conservative now. To reclaim those tokens, set context.max_context_tokens to your full served window, or re-run localharness init.