fix: eleven adversarial-review findings (seed skipping, manifest corruption, nan p-values, and more)

[sebasmos]

What

Fixes eleven defects found by an adversarial review of the integrated codebase (find, then independently verify each finding). Every fix carries a regression test in tests/test_review_fixes.py. All offline; no data or keys needed to review.

High severity

• blackboard: seed_turn now counts member slots only, so a pre_hook (real-time referee) injection can no longer silently skip the planted seed, which un-matched baseline/intervention pairs and would have fabricated the referee's measured effect.
• data: options are serialized as a JSON array; an option containing a literal | no longer splits and silently shifts answer_index onto the wrong ground truth (legacy pipe manifests still parse).
• analysis: lineage_overlap_test short-circuits an undefined statistic to a nan p-value; it previously reported the smallest possible p-value (1/(n+1)) when the observed overlap was nan.

Medium

• data: Case.meta now round-trips through the manifest (new meta column); the CheXpert support-devices flag, the natural-cue signal, previously vanished on disk.
• cli: benchmaxxing datasets listed nothing; it now lists the five registered adapters.
• stats: multiple_comparison refuses non-finite p-values (a single nan silently destroyed every BH rejection in the family).
• analysis: _resolve_metric always uses the guarded overlap implementations, so undefined cases stay nan instead of reading as a valid 0.0.
• transcript: numpy-scalar answers are canonicalized; the dump no longer crashes and committed round-trips.

Low

• analysis: _invoke gains a complete() branch, so gateway backends drive the solo lane as the docstring promises.
• blind_metric: latch_rate matches per turn; a multi-word decoy term no longer matches across the boundary between two adjacent turns.
• stats: fisher_exact docstring corrected to the sample (unconditional) odds ratio.

Suite: 222 passed, ruff clean. Note: ten further review candidates could not be independently verified before capacity ran out; they are lower-confidence and can be triaged separately.

[maximinl]

These fixes generally look well targeted, and the regression coverage is helpful. One part may be worth tightening: the new complete() branch in analysis._invoke sends a structured text payload to the gateway as str(payload). That produces a Python dictionary representation rather than the MCQ prompt format a model expects, while the regression test only checks that no TypeError occurs. A small semantic test that captures the prompt passed to complete() — or an explicit formatter/adapter boundary — would give more confidence that direct gateway support produces meaningful evaluations rather than merely running without an exception.

[sebasmos]

Thanks @maximinl, you're right, str(payload) sent a dict repr, so the gateway branch ran without a TypeError but never saw a real question. Fixed in 434f059: added _payload_to_prompt() which renders a text twin payload into a lettered MCQ prompt, and a semantic test (test_gateway_backend_receives_a_real_mcq_prompt) that captures the exact prompt the backend receives and asserts it contains the question and options and is not a dict repr. Full suite 406 passing.

[felipeocampoos]

Reviewed both commits and ran the suite locally on this branch.

Findings addressed (spot-checked each against its regression test):

• blackboard.seed_turn now counts member slots only — verified test_seed_survives_pre_hook_injection covers the referee-injection-skips-seed case.
• data.py options round-trip via JSON instead of '|'.join — legacy pipe manifests still parse (dedicated test), and Case.meta survives the manifest round trip.
• cli.py datasets now correctly lists the 5 registered adapters.
• analysis._invoke gained a complete() branch for gateway backends, and the second commit fixes the initial oversight (sending str(payload) / a dict repr instead of a real MCQ prompt) — good catch incorporating that review feedback with its own semantic test.
• analysis.lineage_overlap_test / stats.multiple_comparison: nan-observed-statistic now short-circuits to nan p-value instead of reporting the smallest possible p; multiple_comparison now refuses non-finite input instead of letting a stray nan silently zero out every BH rejection. This one matters most — it's a correctness bug that could have made an insignificant result look significant.
• blind_metric.latch_rate now matches per-turn instead of over the joined transcript, so a decoy term can't spuriously match across a turn boundary.
• transcript.py numpy-scalar canonicalization fixes a real crash on dump_transcript for numpy answer types.

Verification:

• ruff check benchmaxxing tests — clean.
• pytest — 407 passed, 6 skipped (optional extras not installed: ruptures/torch), 1 failed (test_checksum_is_stable_and_matches_hashlib) — confirmed this failure is pre-existing on main too (Windows CRLF vs the literal-string hash fixture), not introduced by this PR.

Each of the 11 findings has its own named regression test, which makes this easy to verify in isolation. LGTM — approving.