feat(coin-gym): head-to-head baseline-vs-team benchmark with a measurable verdict (#2825)

[rysweet]

Why

The COIN Gym harness could run the baseline and team strategies separately
(coin-gym run <model> --strategy …), but there was no single command that
runs both head-to-head on the same target set and renders a verdict on whether
the multi-agent team measurably beats the single-model baseline. That verdict
is the goal's stated done-gate ("multiagent measurably above baseline") and the
payload for the Signal milestone report. This PR delivers it.

What

New command:

coin-gym benchmark <model> [--profile <name>] [--targets <path>] [--margin <pct>] [--json]

• Runs baseline and team on the same target set; scores both.
• Diffs COIN's two headline metrics — reach and precision — against a
  material --margin (default 1.0 pts) and classifies:
  TEAM-BEATS-BASELINE / BASELINE-BEATS-TEAM / TIE / MIXED-TRADEOFF.
• Persists both runs under the profile, so the baseline feeds straight into
  improve --holdout fresh — the iterative climb.
• --json emits the machine-readable head-to-head as the Signal
  milestone-report payload; a signal: one-liner aids posting.

On the bundled sample the team ties reach at 60% but the abstain gate lifts
precision 60% → 100%, so the verdict is TEAM-BEATS-BASELINE — the
skwaq-style precision win, now a first-class, reportable result:

baseline:  reach 60.0%  precision 60.0%   R:3/W:2/A:0/T:0/N:0/E:0   [claude-opus-4-6-baseline-…]
team:      reach 60.0%  precision 100.0%  R:3/W:0/A:2/T:0/N:0/E:0   [claude-opus-4-6-team-…]
delta:     reach +0.0 pts (flat)   precision +40.0 pts (improved)   [margin 1.0 pts]
verdict:   TEAM-BEATS-BASELINE
signal:    COIN Gym (offline scaffold): claude-opus-4.6 — team vs baseline on you/coin@v1-sample → TEAM-BEATS-BASELINE (Δreach +0.0, Δprecision +40.0 pts)

Offline scaffold (Phase 4): the grade is a mock oracle — a real baseline-vs-team
result needs a coin evaluate grade on the Phase-3 VM (#2823). LOCAL-ONLY.

Verdict rule (deltas = team − baseline, margin m)

improved if delta > m, regressed if delta < −m, else flat.

• improves ≥1 metric & regresses none → TEAM-BEATS-BASELINE
• regresses ≥1 & improves none → BASELINE-BEATS-TEAM
• improves one & regresses the other → MIXED-TRADEOFF
• neither → TIE

Merge-ready evidence

1. qa-team scenarios (gadugi-test validate + run).
New scenario tests/qa-scenarios/coin-gym-benchmark-verdict.yaml drives the CLI
end-to-end and asserts the TEAM-BEATS-BASELINE verdict + JSON payload.

• gadugi-test validate -f … → ✓ Scenario "simard-coin-gym-benchmark-verdict" is valid
• gadugi-test run -d <tmp> → ✓ Passed: 1 ✗ Failed: 0 (exit 0, verdict confirmed live)

2. Docs updated (user-facing surface = new CLI command).

• docs/howto/run-the-coin-gym-harness.md — new benchmark section (verdict
  table, sample output, --json/--margin, iterative-climb tie-in) + commands list.
• docs/research/coin-benchmark-and-skwaq-study.md — CLI sketch (3.5) updated.

3. Quality-audit (SEEK→VALIDATE→FIX, ended clean).
Reviewed edge cases: precision denominator = 0 (uses the harness's existing
ratio 0-convention → all-abstain vs all-wrong ties, consistent with the scorer);
strict-vs-margin boundary (exactly m = flat, tested); negative/NaN/inf --margin
rejected at the CLI and clamped in the pure API; bare --json stripped before the
value-pairing parser; both runs persisted with matching targets/offline-scaffold.
No unrelated code touched. Final cycle clean.

4. CI-equivalent checks green locally.

• cargo test --lib coin_gym → 128 passed; 0 failed (16 new: 11 benchmark
  unit + 5 CLI).
• cargo clippy --all-targets --all-features --locked -- -D warnings → clean.
• cargo fmt --check → clean.
• cargo build --bin coin-gym → ok; manual human + --json runs verified.

6. Diff focused. 3 new files (benchmark.rs, tests_benchmark.rs, the qa
scenario) + surgical edits to mod.rs (module decl, dispatch, usage,
cmd_benchmark), tests_cli.rs (usage assertion + 5 tests), and two docs. No
unrelated edits.

Scope note

This is the offline head-to-head measurement contract (Phase 4). The real
baseline-vs-team leaderboard delta still depends on the Phase-3 azlin VM
(#2823); benchmark labels every offline run explicitly and never submits
externally.

Advances #2825.

[rysweet]

CI status — one inherited, unrelated red check (coverage)

Every quality gate for this feature is green:

Check	Result
pre-commit (build + clippy --all-targets -D warnings + full test suite + fmt)	✅ pass
install-real	✅ pass
e2e-dashboard	✅ pass
build, docs	✅ pass
cargo-audit, cargo-deny, cargo-vet, npm-audit, GitGuardian	✅ pass
coverage	❌ inherited infra failure — not this PR

coverage is a repo-wide regression from #3171, not this change. #3171
repointed lbug to a git fork, but scripts/provision-lbug-prebuilt.sh's
lbug_version() only matches the lbug = "=X.Y.Z" crate form, so on a cold
cache it dies with could not determine lbug version from Cargo.toml — before
any crate compiles. It fails identically on a clean re-run and on any PR
rebased onto current main. Full root-cause analysis + suggested fix filed as
#3201.

I deliberately did not fold the lbug/coverage infra fix into this PR: a
correct fix must also stop verify.yml/coverage.yml from force-exporting
LBUG_LIBRARY_DIR/LBUG_INCLUDE_DIR (the fork's build.rs trusts them and
skips its source self-build when they're set), i.e. a coordinated multi-file
change in the exact area #3171 owns — out of scope here and would violate the
"focused diff" rule. Once #3201 lands on main, a rebase turns coverage
green.

Merge-ready evidence (unchanged, re-verified after rebase onto main)

• Tests: cargo test --lib coin_gym → 128 passed / 0 failed (16 new).
• Lint/format: clippy --all-targets --all-features --locked -D warnings
  clean; cargo fmt --check clean.
• qa-team scenario: tests/qa-scenarios/coin-gym-benchmark-verdict.yaml —
  gadugi-test validate ✅ and gadugi-test run ✅ (verdict asserted live).
• Conflict resolution: rebased onto origin/main; docs conflicts with docs(coin-gym): reference the reproducible baseline-vs-team measurement #3152
  resolved by keeping both the reference-doc link and the new benchmark
  section.

[github-actions]

📊 Coverage Summary

Generated by cargo llvm-cov --workspace --summary-only (nightly, excluding test files)

Module	Lines	Covered	Coverage
Total	167306	141041	84.3%

_{Coverage data from CI run. Test files matching tests?/ are excluded from line counts.}