Releases: dtduc-git/jevassert
Releases · dtduc-git/jevassert
Release list
v0.2.0 — adapter backends
Record any backend, not just Jev.
record --backend openai|anthropic|bedrock— LLM baselines through the official system-one-adapter (pip install 'jevassert[adapter]'). Bedrock usesAnthropicBedrockwith the standard AWS credential chain.check --input-price/--output-price— cost covers input and output tokens; defaults keep Jev pricing unchanged.- Reports and
--jsonshow the models actually present in the recording, not the pack's pinned version. - GitHub Action gains optional
input-price/output-priceinputs. - 73 tests, ruff clean; existing Jev evidence is byte-identical.
Used to record the first Jev Bench matrix (Jev 1.13.0 vs Claude Sonnet 5 vs a local 7B).
v0.1.0
jevassert v0.1.0
Record/replay regression tests for Jev question packs — assert accuracy, calibration and cost in CI.
Commands
record— concurrent calls with retries,--resume,--rpm,--dry-runcost estimate,--shuffle-options,--repeat N(stability)check— accuracy with bootstrap CIs, ECE/Brier, coverage + author thresholds, threshold suggestion,--failures,--partition dev|test, JUnit + markdown evidencecompare— paired accuracy deltas + exact McNemar
Pack spec v0 loader (jevassert.packs), implemented from the canonical jev-packs SPEC.md.
Reference evidence: jev-packs registry — 9 packs, 2,990 cases / 6,430 items, verified against jev-1.13.0.
63 tests, offline CI, Apache-2.0.