If you are running v0.1.0, upgrade. The headline fix affects every multi-session user.
The big one: shared prompt cache corrupted every session after the first (#46, #47)
KVCache.step is a fixed 256-token allocation increment, not the number of tokens in the cache. The trim path read it anyway, so trim_amount = 256 - cache_hit_len went negative for any shared prefix longer than 256 tokens — which is every real Claude Code session, since the system prompt alone is ~24K. The trim never ran, and each new request was appended on top of the previous conversation's KV state.
The first session after a server start worked. Every session after it was answering from foreign cache state.
Measured on one long-lived server, standard KVCache, MLX_KV_BITS=0, independent sessions sharing a ~19K-token prefix:
| correct answers | |
|---|---|
| before | 1 / 8 |
| after | 8 / 8, then 20 / 20 on a re-run |
Two different faces of the same failure, depending on model and quantization: some setups returned (No output) with HTTP 200 and exit 0; others replayed the previous session's answer verbatim in 0.1s. Both look like "the local model is flaky" and neither reports an error. An empty completion after a cache hit now logs a warning and drops the cache so the next request self-heals.
KV reuse itself is unchanged — the trim just lands on the right boundary now instead of never running.
This also invalidates benchmarks. Any tool-calling or quality numbers collected against a long-lived server on v0.1.0 are contaminated, and not in a predictable direction — we measured the bug inflating one suite's score by priming the model with a previous tool call. If you benchmarked this project and concluded local models were unreliable, it is worth re-running.
Reported by @KaoCSC, whose fresh-server control is what made it findable.
Other fixes
- Launchers hardcoded
~/.local/bin/claude(#40, #48) — anyone who installed Claude Code via npm, Homebrew, or a different prefix got a bareNo such file or directory. All launchers now resolve viaPATHand the common install locations, and a genuinely missing CLI prints the install command instead of an ENOENT. - Gemma 4's collapsed
<tool_call:Name{...}opener (#45) — 7/14 to 14/14. - Bare top-level JSON tool calls (#43) — Qwen 2.5 Coder 0/12 to 14/14.
- Prefill chunk 8192 to 512 (#44) — tripled peak memory for no gain.
- Stop on the tokenizer's real EOS, tolerate empty env ints (#41).
- Live token streaming for tool-less requests (#39).
Models
Qwen3-Coder-Next 80B added as an on-demand coder (30B stays default), Qwen3.6 27B abliterated wired into setup.sh for the 32GB tier, Qwen3.6-35B-A3B added to the doctor's 64GB+ recommendations.
Thanks to @KaoCSC and everyone who filed reports this cycle.
