Skip to content

BetterBench release 0.6.0

Choose a tag to compare

@GGZ14 GGZ14 released this 11 Sep 20:06
· 21 commits to main since this release
BetterBench 0.6.0

The prefill sweep no longer sends the same text twice: the filler behind the
nonce is reshuffled every pass, so a cache keyed on block content has nothing
to reuse. Measured on Qwen3's tokenizer, an 80k-character prompt went from 918
blocks of 16 tokens with 25 distinct values among them to 917 of 917 distinct,
with nothing shared between passes.

The concurrency sweep now stops at 8 by default.

Upgrading: prefill throughput may read lower than it did on 0.5.0. That is the
measurement being corrected, not the server regressing.