Skip to content

Releases: CAOShurong/ohmjudge

Release list

OhmJudge v0.3.0 — answer-free blind collection

OhmJudge v0.3.0 — answer-free blind collection Latest
Latest

Choose a tag to compare

View all tags
@github-actions github-actions released this 11 Aug 10:39
v0.3.0
71f8422
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Answer-free collection for no-API experiments

OhmJudge can now collect responses from ordinary web products without giving the collector a private answer key.

ohmjudge collect \
  --tasks tasks.private.jsonl \
  --output blind-collector.html

The resulting standalone HTML contains public prompts and the suite commitment, but no expected values, tolerances, live scores, grading code, CDN, tracker, or network calls. It saves progress locally, supports JSONL checkpoint export/import, and keeps capture settings separate for each model label.

Use ohmjudge lab when you want immediate personal feedback. That separate artifact still embeds the private answer key and should not be shared before a secrecy-sensitive collection is complete.

Integrity improvements

  • collector, lab, replay, and API-runner responses are bound to the exact suite commitment
  • audit reports matched, missing, and mismatched commitments
  • a present wrong-suite commitment is always an integrity error
  • collector imports reject wrong-suite, duplicate, or locally conflicting records instead of silently combining them

Project and release hardening

  • Python 3.11-3.14 CI across Linux, Windows, and macOS
  • CodeQL, Dependabot security updates, secret scanning, and push protection
  • GitHub Actions pinned to reviewed commits
  • tag/version parity, one-time distribution builds, Linux/Windows clean installs, SHA-256 manifest, and SLSA provenance attestations

Install

python -m pip install "https://github.com/CAOShurong/ohmjudge/releases/download/v0.3.0/ohmjudge-0.3.0-py3-none-any.whl"
ohmjudge doctor

The package is not advertised on PyPI. This release provides the wheel, source archive, and SHA256SUMS directly on GitHub.

Verify a downloaded artifact with GitHub CLI:

gh attestation verify ohmjudge-0.3.0-py3-none-any.whl \
  --repo CAOShurong/ohmjudge \
  --signer-workflow CAOShurong/ohmjudge/.github/workflows/release.yml

Scope

OhmJudge remains an early research tool for fresh scalar electrical-engineering calculations, not an authoritative model ranking or evidence of professional engineering competence. The bundled model names, responses, latencies, tokens, and prices are explicitly synthetic.

Full changelog: v0.2.0...v0.3.0

Assets 5
Loading

OhmJudge v0.2.0

OhmJudge v0.2.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 04 Aug 16:52
v0.2.0
9caab27

Full Changelog: v0.1.0...v0.2.0

Loading

OhmJudge v0.1.0 — First public release

OhmJudge v0.1.0 — First public release

Choose a tag to compare

View all tags
@CAOShurong CAOShurong released this 04 Aug 14:46
v0.1.0
fb91344

OhmJudge is a zero-dependency, no-API-first laboratory for fresh executable electrical-engineering evaluation.

Highlights

  • 50 seeded task templates across 10 EE domains
  • standalone copy/paste lab for any web model
  • deterministic unit-aware grading and error taxonomy
  • cross-artifact audit and paired exact McNemar comparison
  • optional OpenAI Responses, OpenAI-compatible, Anthropic, and replay adapters
  • standalone interactive reports with Wilson intervals and a log-scaled cost-accuracy Pareto frontier
  • public synthetic demonstration and GitHub Pages site

The bundled model names, responses, latencies, tokens, and prices are synthetic fixtures, not claims about commercial models.

Loading