Skip to content

Releases: sathariels/jevcheck

v0.2.2 — dependency pins, eval tests, and audit status

Choose a tag to compare

@cursor cursor released this 13 Oct 22:14
46637b5

Highlights

  • typesafe-sdk lock is 0.7.4. The install floor stays typesafe-sdk>=0.7.0.
  • GitHub Actions pins: actions/checkout 7.0.1, actions/setup-python 7.0.0, and hashgraph-online/ai-plugin-scanner-action 1.2.777, still by commit SHA.
  • Eval coverage for tolerance edges, multiple expectations, and error paths. CI prints a coverage report.
  • evaluate() accepts a concrete candidate pin that has surrounding whitespace.
  • The v0.1 recheck P1/P2 findings are marked resolved in be58041 (v0.2.0).
  • Pin sathariels/jevcheck/.github/actions/jevcheck@v0.2.2. @v0.2.1 is the first tag that contains the Action.

Install

pip install jevcheck==0.2.2

Action

- uses: sathariels/jevcheck/.github/actions/jevcheck@v0.2.2
  with:
    contract: contracts/support.json
    command: eval
    candidate-model: jev-1.14
    answers: fixtures/candidate-replay.json

v0.2.1 — GitHub Action, tutorial, and packaging metadata

Choose a tag to compare

@cursor cursor released this 30 Sep 20:32
694da05

Highlights

  • Composite GitHub Action .github/actions/jevcheck. Pin sathariels/jevcheck/.github/actions/jevcheck@v0.2.1. v0.2.0 does not contain the Action.
  • First-run tutorial (docs/tutorial.md) and an offline examples/ pack. Replay eval and compare need no API key.
  • Optional jevtriage pull-request workflow, skipped when TYPESAFE_API_KEY is unset.
  • SPDX license = "MIT", Python 3.13 and 3.14 in classifiers and CI, changelog, and Homepage / Source / Issues / Changelog URLs.
  • CLI commands, exit codes, and eval rules match 0.2.0.

Install

pip install jevcheck==0.2.1

Action

- uses: sathariels/jevcheck/.github/actions/jevcheck@v0.2.1
  with:
    contract: contracts/support.json
    command: eval
    candidate-model: jev-1.14
    answers: fixtures/candidate-replay.json

v0.2.0 — record / compare + fixture eval

Choose a tag to compare

@cursor cursor released this 20 Sep 00:16
38fddba

Highlights

  • jevcheck record — capture baseline answers to a replay JSON
  • jevcheck compare — candidate vs recorded baseline, or live --from-model / --to
  • v0.1 eval fixture-versus-candidate path unchanged
  • Answer-kind validation on record/compare (choice/noul/score must match)
  • Package version 0.2.0; contract schema still "0.1"
  • Alias summaries print alias → resolved when they differ

Verify

  • CI: pytest on 3.11/3.12 + packaging (wheel/sdist)
  • Local agent report: 119 passed, 1 skipped (live gated)

Install

pip install jevcheck==0.2.0

v0.1.0 — scoped fixture evaluator (alpha)

Choose a tag to compare

@cursor cursor released this 20 Sep 00:00
be58041

Scoped v0.1.0 alpha fixture evaluator for TypeSafe Jev behavioral contracts.

What this is

  • Fixture-versus-candidate evaluation: pin expectations in a JSON/JSONL contract, eval a candidate (live or replay)
  • Choice / Noul / score checks, confidence floors & drops, explicit pinning, CI-friendly exits (0/1/2/3)
  • Optional Gate helper; support-triage example; packaging + mocked SDK transport tests

What this is not

  • Not two-model live comparison (compare --from/--to) — planned for v0.2
  • Not a claim of verified live-service behavior for every account/model ID
  • Historical audit docs under docs/ are snapshots; this tag is the ship verdict for commit be58041

Verification (final recheck)

  • 94 passed, 1 skipped (live gated by TYPESAFE_API_KEY)
  • Wheel + sdist; installed-wheel CLI probes; real SDK mocked-HTTP scenarios green