Repository navigation
v0.4.0
omp-jevify v0.4.0
Install:
omp plugin install 'git+https://github.com/ericjuta/omp-jevify.git#v0.4.0' --forceWhy
A review of recent real sessions on OMP 18.12.5 found that none of them used the jv.* helpers.
The host jevify-notice sends agents straight to the built-in judge_batch(), so the receipts,
flags and review did not apply. In 3 of 4 of those sessions a direct script did the real work.
A paired eval on v0.3.x gave the same result: direct scripts scored 70/70 and Jev scored 64/70 on
structural audit tasks. This release makes notice-driven runs auditable and says plainly when a
direct method is the better choice.
Added
Structured findings grading (evals/findings.py, jevify-findings/1). This replaces the
v0.3.1 prose extractor.
- Each finding has exact claim IDs, the reviewed
source_revision, factual/action/introduced
facets, and evidence quotes that must equal the cited source lines. - Prose, misnamed IDs, a stale revision or a quote that doesn't match earns no credit. The
scoring denominators are fixed.
Durable, source-bound review inbox (jv.review_queue, review_questions, adjudicate,
review_items, review_status).
- Verdicts are stored in files, bound to rows and record the reviewer's identity.
- Adjudication is all-or-nothing.
- Changes to the source, rubric or evidence mark items stale.
grants_authorityis always false.delete_spansrefuses to run while anything is drifted or unresolved.
Stricter pilot reports (evals/run.py --pilot, jevify-pilot/1, direct-reading
strategy, result schema 6, compare_graded).
- A method recommendation, including "use direct scripting", is withheld unless all of these
hold: the arms are comparable, every case has gold from an independent adjudicator with a
distinct key digest, and each arm meets its own thresholds. - Duplicate fixture/key aliases count once.
Host-notice adoption helpers.
jv.adopt(items, states, Q, batch=b)andjv.scalarturn built-injudge_batch()output into
receipt-bound rows.jv.pilot_checkreturnsuse_directwhen more than about 40% of items are flagged, in the
uncertainty band or caught by the catch-all.jv.remainingreuses unchanged verdicts,jv.gatechecks for regressions,jv.powersizes a
sample, andjv.Ledgerkeeps tuned units out of held-out evidence.jv.facts,jv.units_from_tableandjv.refuse_elidedhandle input hygiene, and
jv.flag(report=)was added.jv.choiceenforces the 255-label route limit, andjv.plan
now reportslabels,over_labelsandpack_suggestion.jv.splice,jv.comparable,jv.exportandjv.receipt_mdwere added.- Rows now carry
state_sha256,judged_atandassembly. Receipts now carryserved_models,
comparable,host_countsandbilling.
Verification
- 456 unit tests pass. Each of the four commits passes the suite on its own in a clean worktree.
- CI passed on main (run 39577134316) and on the tag (run 39577136651).
- Offline checks made no model requests: structured grading, inbox persistence across
processes, stale-evidence rejection, replay of retained real judgment rows, and pilot
withholding. - A live smoke on OMP 18.12.5 cost about $0.0002 in total. It ran adopt, run, remaining,
splice, gate, receipt_md and export.
Boundaries
- These are workflow checks only. This release makes no claim about quality or savings.
- Provenance stays unknown. Stock 18.12.5 items expose only
key,ok,error,answers
andmodel, with no fallback or cache fields. Cached answers report the alias
~typesafe/jev-latest, so receipts that mix cached and fresh rows arecomparable: false. - Billing is an unverified estimate.
- The repository has one schema-2 answer key, which cannot support a method recommendation.
Pilot comparisons on the development cases come out withheld or inconclusive. - Installing the plugin does not change the host notice.