Repository navigation
vision-bench 0.3.2
Portable benchmark reports and hardware provenance.
What's new
vision-bench export-reportturns compatible tagger runs into a self-contained HTML comparison, a Markdown report, aggregate JSON and checksums, ready for a Hugging Face Static Space.- Compares quality on a common vocabulary while keeping each model's full-vocabulary results and timing denominators explicit.
- Includes dataset fingerprints, model revisions and hashes, preprocessing settings, software versions and available hardware details.
- New tagger runs record best-effort CPU, RAM, OS and visible NVIDIA GPU/driver inventory. Older runs remain supported; missing details are marked as unrecorded.
- Public exports omit source images, raw predictions, sample identifiers, local paths and logs. See
docs/publishing-results.mdfor the workflow and metadata limitations.
Example
vision-bench export-report ./runs/pixai ./runs/caformer --title "PixAI vs CAFormer" --output ./reports/comparisonDownloads
Install the wheel with Python 3.11 or newer, or use the source archive. Select the runtime extra (cpu, cuda or dml) when running models. Offline report export does not require ONNX Runtime or Django.
Validation
97 core tests passed locally, followed by 7 focused report tests after the final report edits. Ruff checks passed. The wheel and source distribution were built, and report export was verified from an installed wheel without ONNX Runtime or Django. No new GPU model benchmark was run for this release.