Skip to content

v0.8.0-alpha.3 - Eval42 regression gates

Pre-release
Pre-release

Choose a tag to compare

@yuyukosama2004 yuyukosama2004 released this 16 Jul 12:54
5fed019

Eval42 PhoneMall implementation milestone

This prerelease records the target-project implementation of Eval42 Phases 1 and 2:

  • protected, merchant-only, disabled-by-default retrieval evaluation endpoint
  • shared read-only retrieval service with raw and eligible ranked candidates
  • deterministic 20-case offline evaluator and report generation
  • Recall@5, MRR, forbidden-product, hard-constraint, and unexpected-empty metrics
  • Baseline v1 fingerprints, case-hash comparison, regression gates, and execution budgets
  • CI report artifacts and an opt-in protected live nightly workflow

Main commit 5fed019 passed the complete 14-job CI workflow.

Important boundary

The 20 source-derived labels remain pending-human-review in issue #58. The fixture baseline proves deterministic regression plumbing; it is not a certification of live recommendation quality. Live nightly evaluation remains disabled until the protected deployment, secrets, and reviewed live baseline exist.