Skip to content

localperf v0.2.0

Choose a tag to compare

View all tags
@osolmaz osolmaz released this 02 Oct 04:49
v0.2.0
b58b2d6

localperf v0.2.0 replaces the spec files and the sweep planner of v0.1.0 with named suites and deployments. Older specs and localperf sweep plan are gone along with the localperf-sweep-* helper commands. Reports now accept only validated SQLite artifacts written by localperf bench run.

Changes

localperf bench run --suite <name> --deployment deployment.json is the new way to run a benchmark. The README lists the three built-in suites. A deployment is either a vLLM server that localperf manages or an OpenAI-compatible server that is already running, such as llama.cpp.

Every benchmark point now checks that the requested backend really ran. A result whose backend fell back to something else is marked invalid instead of reported. The built-in HTTP client adds a unique prefix to every request, so a repeated prompt cannot hit the server's prompt cache. Turn it off with "prompt_nonce": false.

Reports gained full-run timing and dedicated prefill numbers, and the HTML report and the viewer now follow the system's dark or light setting. The README has a cover and covers both deployment types.

Install

Download the archive for your system below and check it against SHA256SUMS, or run:

go install github.com/osolmaz/localperf/cmd/localperf@v0.2.0

The archive names changed to localperf-v0.2.0-<os>-<arch>.tar.gz.

Assets 7
Loading