paired-eval 0.2.0
paired-eval 0.2.0
Added
paired_evalmodule: a single import surface matching the repository name (import paired_eval as pe), re-exporting
the public API of the implementation modules, pluspe.demo()/python3 paired_eval.py— an offline demo that
prints a complete report from two stub systems in under a second.claim_eval.fmt_p: report p-values below 1e-4 in scientific notation instead of0.0000.- Report text in English:
set_language("en")(orlang="en"oninterpret/report,--lang enon the demo).
The Chinese strings are byte-identical to before; a test asserts the English output contains no CJK
characters and that both languages expose the same placeholders.
Changed
- README rewritten from the user's perspective (why / install / ten-second demo / your own tasks / how to read the
report / API / scope). The demo output pasted in both READMEs is asserted equal to the actual output by a test, and
every code block is executed against the real API. - README gains a flow diagram and a grounded "how it relates to other tools" section (lm-evaluation-harness,
Inspect, promptfoo, openai/evals — descriptions taken from their own READMEs).
Verified on this tag: full fast suite on Python 3.9 / 3.10 / 3.12 / 3.13 (locally and in CI), hook integration, power calibration, hand-picked mutants; changed-line mutation ratchet 4/4 killed.