Skip to content

UIJudgeBench v0.4.0

Pre-release
Pre-release

Choose a tag to compare

View all tags
@soodoku soodoku released this 18 Aug 11:10
· 4 commits to main since this release
v0.4.0
92c0302
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

UIJudgeBench v0.4.0

Screenshot-contract and provider-native Batch release.

  • Adds hashed, bounded target crops with an invertible page-CSS coordinate transform.
  • Audits the exact provider-bound vision slice and fails before provider construction on stale or ungrounded inputs.
  • Repairs corpus geometry and regenerates 3,830 scored items across 665 frozen pages.
  • Requires LayoutLens 2.2.0 and aligns canary/full native-Batch controls, usage, manifests, and pricing.
  • Retains the historical Gemini diagnostic as ineligible provenance; no paid call has run under screenshot contract v2.
  • Free floors and all generated reports were regenerated; two consecutive floor runs were byte-identical.

Verification: 357 local tests; hosted CI green on Python 3.11-3.14; Ruff, Pyright, CFF, Twine, isolated-wheel install, deterministic corpus/floor gates, and independent Claude review all passed.

The benchmark data release is mirrored separately on Hugging Face. PyPI contains the harness package; it does not bundle the corpus.

Assets 4
Loading