Skip to content

Vision Bench v0.4.0

Latest

Choose a tag to compare

@TimeOfAnarchy TimeOfAnarchy released this 19 Nov 13:14

Vision Bench 0.4.0 expands the offline analysis workflow for image taggers and
fine-tuned checkpoints. Use the full scores from a saved run for these analyses.

  • Compare paired bootstrap intervals, per-image quality, label-frequency strata,
    and precision or recall at matched operating points.
  • Fit category thresholds or supported per-tag thresholds on a separate
    calibration run with fit-thresholds. The fitted profile records its source
    and evaluation checks the split for identical image hashes.
  • Compare renamed tags with optional aliases and implications. The report shows
    exact-name coverage alongside mapped coverage.
  • Find per-tag AP and F1 regressions between checkpoints. Use label-audit to
    review sampled unlabeled predictions or hard-images to inspect disagreement
    and low-F1 examples.

Download the wheel for Python 3.11+ and install it in a fresh environment. The
cpu extra installs the ONNX Runtime CPU provider.

python -m pip install "./vision_bench-0.4.0-py3-none-any.whl[cpu]"
vision-bench --help

The source archive includes the generated CPU example. Both package files are
listed in SHA256SUMS.

Tagging guide ·
Autotagging case study ·
Frozen reproduction kit