feat(eval): offline golden-set eval harness (Step 5.2)

[officialCodeWork]

Summary

Completes the Step 0.8 eval skeleton (stub retrieval, no nDCG, RAGAS-only faithfulness) into a deterministic, offline, CI-stable golden-set eval.

• Real retrieval over a synthetic corpus. A new eval/golden_set_v0/ harness drives the real HybridRetriever over a 5-domain synthetic corpus on the noop SPIs. NoopKeywordStore already ranks by token overlap; a deterministic HashingEmbedder gives the dense path real signal (NoopEmbedder returns zero vectors), and the two fuse via RRF — no network, no LLM, identical numbers on every OS.
• 500-query / 5-domain golden set committed under tests/eval/golden/, generated reproducibly from the corpus and drift-gated (a test fails if the committed files diverge from the generator). 3 passages/concept so Citation Precision and nDCG aren't degenerate.
• Five metrics — Recall@k, MRR, nDCG@k (new), Faithfulness (new dependency-free lexical_faithfulness; RAGAS optional), Citation Precision — reported overall + per-domain + per-difficulty, as report.json/report.md and a self-contained HTML report (render_html_report).
• Additive type changes. EvalMetrics/EvalReport gain defaulted nDCG + by_domain/by_difficulty (new GroupEvalMetrics); run_eval's defaults are unchanged so the Step 0.8 contract + tests hold.
• CLI. python -m eval.golden_set_v0.harness [--domain/--queries/--k/--check]; ragctl eval run --html + nDCG/per-domain output; eval show per-domain.
• --check enforces the thresholds.yaml hard-gate floor now; the regression-delta comparison + committed baselines/main.json + PR-comment workflow are Step 5.3.

Default full run: recall@10 0.91 · mrr 0.88 · nDCG@10 0.86 · faithfulness 0.93 · citation precision 0.27 — clearing the 0.70/0.60/0.80 floors, with real per-domain (0.82–0.98) and easy>medium>hard spread.

Documentation

• docs/adr/ADR-0027-offline-eval-harness.md — decisions (synthetic corpus on noop SPIs, dependency-free lexical faithfulness vs RAGAS, committed+drift-gated golden set, harness in eval/ not rag_config).
• docs/reference/eval-harness.md — public API: metrics, golden-set format, harness + ragctl eval usage, EvalReport shape, extension points.
• docs/architecture/golden-set-eval.md — design/internals: why noop backends give a real signal, corpus/golden-set co-design, metric edge cases, relationship to Step 5.3, reviewer checklist.
• docs/README.md, TRACKER.md (5.2 ✅, Phase 5 → 2/7, 59/84, next = 5.3), CLAUDE.md updated.

Test plan

• ruff check . + ruff format --check . clean
• mypy --strict packages/ apps/gateway/ — no issues (278 files)
• RAG001 logging check clean
• Golden-set drift gate: python -m eval.golden_set_v0.generate --check up to date
• PolicyEngine coverage linter green (corpus builder allowlisted)
• Full workspace suite: 2127 passed / 4 skipped (92 in the eval suite, ~38 new)
• python -m eval.golden_set_v0.harness --check passes the threshold floor

🤖 Generated with Claude Code