Repository navigation
localperf v0.3.0 makes llama.cpp the primary runtime, stops each benchmark point when its number is known, and measures decode at long context from a restored cache instead of a repeated cold prefill. Suite files change to version 3. A suite with version 1 or 2, or a repeats field, no longer loads.
Changes
Adaptive convergence. A point repeats until the 95% confidence interval of its throughput is within ±5%, with 3 to 10 samples and a 600 s limit for each point. The remaining planned samples are skipped with the reason. The report shows the interval and the stop reason in the cell details, and marks a point that stopped without converging with *. localperf artifact check recalculates every stop decision from the stored data. The repeats field is replaced by a convergence block. See docs/2026-10-02-adaptive-convergence.md.
Restored context for decode cases. A decode case with a long prompt now primes its context once, saves the llama.cpp KV cache, and restores it into every slot before each sample, so the sample measures decode only. Every request must report its prompt as cached, or the sample fails. Snapshots are kept under the user cache directory (runner.snapshot_dir, or LOCALPERF_SNAPSHOT_DIR), so later runs skip the prime. Prefill is measured by cold cases. practical-64k gains a prefill-64k case at 1 user, and its full-context rows no longer report prefill at 6 users. On Ternary Bonsai 2 9B at 64k context, decode now measures 31 tok/s. The old method reported 5.7 tok/s, because most of each sample was prefill. See docs/2026-10-02-restored-context-decode.md.
Restored cases need llama.cpp. On vLLM, the built-in suites run only their cold cases until vLLM gets a context preparer.
llama.cpp first. llama.cpp is now the primary runtime in the README, the examples, and the agent skill. A managed llama-server gets its slots, context size, and --slot-save-path from the suite, and the detail view shows the llama.cpp server settings.
Safer runs. bench run refuses to overwrite an existing run without --replace. Resume no longer re-runs short-prompt samples, and keeps the artifact valid while a point runs again.
Mutation testing is now optional. It runs as a weekly scan and by hand, not on every push or pull request.
Install
Download the archive for your system below and check it against SHA256SUMS, or run:
go install github.com/osolmaz/localperf/cmd/localperf@v0.3.0