0.6.1

Upgrading: prefill throughput may read lower than it did on 0.6.0 if your server keeps its KV cache in RAM or SSD between runs, as tiered engines like Radiance do. Earlier runs could be served the previous run's prompts from that cache. That is no longer possible. A drop is the old number having been wrong, not a regression in the server.

Fixed

• Prompts no longer repeat from one run to the next. Every sweep seeded its nonce generator with a constant, so each betterbench run sent the same nonces as the last. For the prefill sweep that also meant the same shuffled filler. A cache confined to the server process never noticed, but one that persists to RAM or SSD served the previous run's prompts back and inflated prefill throughput. Nonces are now seeded from OS entropy, always, with no variable to set. The only cost is that a run's prompts are no longer byte-reproducible. Token counts and timing are unaffected.
• The concurrency sweep now runs the level it names. It used to be capped by asyncio's default thread pool, so a high level (say 16) could silently serialise on a small client machine. That looked exactly like a server-side knee. The sweep now sizes its pool to the widest level.
• Prefix-cache hits are now recorded and surfaced. usage.prompt_tokens_details.cached_tokens is stored per request, and the report warns when a run that claims a cold cache was partly served from the server's. A server that doesn't report the field leaves it null, which stays distinct from a verified 0.
• --out pointing at an existing directory now fails loudly. It used to write nothing there and leave the previous run's results.json looking like this run's numbers.

Added

• Interactive comparison session. betterbench compare with no arguments opens a loopback-only gallery of every saved run. Pick two to get a pair page with a comparison band: paired decode CIs, plus median deltas for latency, prefill and concurrency. Cross-file comparisons are unpaired in time, so a banner on every pair page points to betterbench ab for a real verdict.