Build the shared harness/ package the per-tool benchmarks consume as thin entry points: a subprocess runner (rac strictly as an external CLI on PATH, DG-ADR-0001 — zero engine imports), a deterministic scorer (P@k / R@k at k ∈ {1,3,5}, MRR, full-returned-list hard negatives, conformance pass-rate), a {metrics, metadata, per_query} scorecard writer with fixed-precision rounding, and a --check gate (exit 0 clean / 1 gate failure / 2 usage error) mirroring the rac eval flag surface: --check | --update-baseline, --json, --root, --queries, --baseline, --config.
Roadmap: rac/roadmaps/benchmarks/tool-benchmarks.md in rac-core (codename tool-benchmarks). Contract: rac-core ADR-097 (extends ADR-066); scorecard shape per rac/designs/tool-benchmark-harness.md.
Build the shared
harness/package the per-tool benchmarks consume as thin entry points: a subprocess runner (rac strictly as an external CLI on PATH, DG-ADR-0001 — zero engine imports), a deterministic scorer (P@k / R@k at k ∈ {1,3,5}, MRR, full-returned-list hard negatives, conformance pass-rate), a{metrics, metadata, per_query}scorecard writer with fixed-precision rounding, and a--checkgate (exit 0 clean / 1 gate failure / 2 usage error) mirroring therac evalflag surface:--check | --update-baseline,--json,--root,--queries,--baseline,--config.Roadmap:
rac/roadmaps/benchmarks/tool-benchmarks.mdin rac-core (codenametool-benchmarks). Contract: rac-core ADR-097 (extends ADR-066); scorecard shape perrac/designs/tool-benchmark-harness.md.