Skip to content

Releases

BenchReceipt 0.5.0

Choose a tag to compare

@apex-mochen apex-mochen released this 13 Nov 02:42

BenchReceipt checks saved vLLM-shaped JSON before its values are shared as evidence. It creates offline HTML, JSON and Markdown receipts using the Python standard library. The ZIP includes the Apache-2.0 license and a bundle verifier.

  • Inspect one result or summarize a directory's statuses and missing fields.
  • Preserve failed and unknown counts and recompute available TTFT aggregates from successful-request samples.
  • Review per-file errors in a batch, with filename patterns and optional recursion.
  • Limit each input read to 64 MiB by default and preserve existing output files.
  • Try the bundled synthetic example without a GPU, model server or dependency installation.

Quick start

Put the five assets in one directory and open a terminal there. In PowerShell, calculate all five hashes with this command.

Get-FileHash -Algorithm SHA256 -LiteralPath 'benchreceipt-0.5.0.zip','demo.html','demo.json','demo.md','SHA256SUMS'

Compare each result with the Release assets table below. On Ubuntu, use the following commands. Compare the first result with the table's SHA256SUMS entry, then confirm the other four files report OK.

sha256sum SHA256SUMS
sha256sum --strict -c SHA256SUMS

Continue when every hash matches. If a file is missing or a hash differs, obtain a fresh copy of the assets and repeat the checks before extracting. Extract into a new unpacked directory. In PowerShell, run the following commands.

Expand-Archive -LiteralPath 'benchreceipt-0.5.0.zip' -DestinationPath 'unpacked'
Set-Location 'unpacked'

On Ubuntu, run the following commands.

unzip 'benchreceipt-0.5.0.zip' -d unpacked
cd unpacked

Check python --version for Python 3.11 or newer. If that command is unavailable, check python3 --version and use python3 in both commands below. Run the verifier first.

python -I -B verify.py

Continue when the verifier reports "passed": true and exits with code 0. If verification fails, obtain a fresh copy of the assets and repeat the hash check. Extract that copy into another empty directory and rerun the verifier before generating the example.

python -I -B benchreceipt.pyz --demo --output demo.html

Expect "status": "CHECKED" and exit code 0. Open the generated demo.html from the unpacked directory in a browser. It should label the data DECLARED_SYNTHETIC and match the companion HTML report. A repeated command with the same output filename exits with code 3 and preserves the existing report. To run it again, change --output demo.html to --output demo-2.html. The bundled README shows how to regenerate the JSON and Markdown examples.

Python requirements and validation

Minimum Python requirement. Python 3.11 or newer.

Tested recipient environments. Windows 11 with Python 3.12.14 and 3.13.14, and Ubuntu 24.04 on WSL 2 with Python 3.12.3. A public-download check also passed on GitHub-hosted Ubuntu 24.04 with Python 3.11.18.

In each listed environment, the outer checksums and ZIP verifier passed, and the generated HTML, JSON and Markdown matched the companion examples byte for byte. An independent Windows 11 source export passed 78 product tests and all 10 portable-check steps. The source CI passed on Windows Server 2025 with Python 3.11.9 and Ubuntu 24.04 with Python 3.11.18.

Every receipt reports NOT_EVALUATED for regression. Review numeric metrics, hashes, custom batch patterns and any included filenames before sharing reports from your own data.

Release assets

File Description SHA256
benchreceipt-0.5.0.zip Offline tool, README, verifier, manifest and Apache-2.0 license e4a85fe16a9a1d89440a583748a463ead448cc7870156aecc7cbc2584e2dadb0
demo.html Synthetic example report in HTML f506226fbd9c894663706ca48e641ca6ec1b67c91fc8594c1a16be7901249ad2
demo.json Synthetic example report in JSON 4d2bf238a3885f3bd68ccbde37074d11c4b2ccb6e806d9c3b32f09e6238b3f6a
demo.md Synthetic example report in Markdown 76067c5db90269d348ee7e9a29bfe639d9b86ac499268db8b333aa5099774161
SHA256SUMS Checksums for the ZIP and three example reports d54c21c6500b4a9f97ea540b19ccc4412b70d57283f1ef0c0a0f49b81b4afa41