What changed
EvalRepro now includes a first-party Harvey LAB task-contract adapter for detecting semantic drift across exact tasks, task-prefix selections, or complete local benchmark checkouts.
Highlights
- Deterministic hashing of effective instructions, rubrics, deliverables, unknown fields, document paths, and document contents
- Credential-stripped Git provenance and shared-document inventory caching
- Hash-only output with optional identifier previews
- Path and symlink protections for task, instruction, document-root, and source-file inputs
- CLI support for reproducible local comparisons
Validation
- Python 3.11, 3.12, and 3.13
- Ruff format and lint, strict mypy, and distribution builds
- 72 offline tests with 95.25% branch coverage on Python 3.12
Full details: CHANGELOG.md