Repository navigation
Releases: sathariels/jevcheck
Releases · sathariels/jevcheck
Release list
v0.2.2 — dependency pins, eval tests, and audit status
Highlights
typesafe-sdklock is 0.7.4. The install floor staystypesafe-sdk>=0.7.0.- GitHub Actions pins:
actions/checkout7.0.1,actions/setup-python7.0.0, andhashgraph-online/ai-plugin-scanner-action1.2.777, still by commit SHA. - Eval coverage for tolerance edges, multiple expectations, and error paths. CI prints a coverage report.
evaluate()accepts a concrete candidate pin that has surrounding whitespace.- The v0.1 recheck P1/P2 findings are marked resolved in be58041 (v0.2.0).
- Pin
sathariels/jevcheck/.github/actions/jevcheck@v0.2.2.@v0.2.1is the first tag that contains the Action.
Install
pip install jevcheck==0.2.2
Action
- uses: sathariels/jevcheck/.github/actions/jevcheck@v0.2.2
with:
contract: contracts/support.json
command: eval
candidate-model: jev-1.14
answers: fixtures/candidate-replay.jsonv0.2.1 — GitHub Action, tutorial, and packaging metadata
Highlights
- Composite GitHub Action
.github/actions/jevcheck. Pinsathariels/jevcheck/.github/actions/jevcheck@v0.2.1.v0.2.0does not contain the Action. - First-run tutorial (
docs/tutorial.md) and an offlineexamples/pack. Replay eval and compare need no API key. - Optional jevtriage pull-request workflow, skipped when
TYPESAFE_API_KEYis unset. - SPDX
license = "MIT", Python 3.13 and 3.14 in classifiers and CI, changelog, and Homepage / Source / Issues / Changelog URLs. - CLI commands, exit codes, and eval rules match 0.2.0.
Install
pip install jevcheck==0.2.1
Action
- uses: sathariels/jevcheck/.github/actions/jevcheck@v0.2.1
with:
contract: contracts/support.json
command: eval
candidate-model: jev-1.14
answers: fixtures/candidate-replay.jsonv0.2.0 — record / compare + fixture eval
Highlights
jevcheck record— capture baseline answers to a replay JSONjevcheck compare— candidate vs recorded baseline, or live--from-model/--to- v0.1
evalfixture-versus-candidate path unchanged - Answer-kind validation on record/compare (choice/noul/score must match)
- Package version 0.2.0; contract schema still
"0.1" - Alias summaries print
alias → resolvedwhen they differ
Verify
- CI: pytest on 3.11/3.12 + packaging (wheel/sdist)
- Local agent report: 119 passed, 1 skipped (live gated)
Install
pip install jevcheck==0.2.0
v0.1.0 — scoped fixture evaluator (alpha)
Scoped v0.1.0 alpha fixture evaluator for TypeSafe Jev behavioral contracts.
What this is
- Fixture-versus-candidate evaluation: pin expectations in a JSON/JSONL contract, eval a candidate (live or replay)
- Choice / Noul / score checks, confidence floors & drops, explicit pinning, CI-friendly exits (0/1/2/3)
- Optional Gate helper; support-triage example; packaging + mocked SDK transport tests
What this is not
- Not two-model live comparison (
compare --from/--to) — planned for v0.2 - Not a claim of verified live-service behavior for every account/model ID
- Historical audit docs under
docs/are snapshots; this tag is the ship verdict for commitbe58041
Verification (final recheck)
- 94 passed, 1 skipped (live gated by TYPESAFE_API_KEY)
- Wheel + sdist; installed-wheel CLI probes; real SDK mocked-HTTP scenarios green