results: multi-round cascade dynamics, adoption is single-shot not building (#130)

[sebasmos]

What (closes #130)

Multi-round cascade dynamics: does shortcut adoption build over deliberation rounds, or is it single-shot? Under the plausible (case-anchored) cascade, the committee runs for K rounds and the holdout's answer is recorded at each round, shared vs isolated.

Result (MedQA, 40 cases, K=5)

round	1	2	3	4	5
shared adoption	0.175	0.15	0.175	0.15	0.15
isolated adoption	0.0	0.0	0.0	0.0	0.0

Null: adoption does not compound over rounds (round 1 vs round 5 McNemar p = 1.0; not monotone). The holdout that adopts the plausible shortcut does so immediately; repeated exposure recruits no one else. This sharpens the story: the cascade is driven by plausibility (a single-shot decision at first contact), not by accumulating social pressure.

Acceptance (#130)

• Per-round adoption curves, shared vs isolated. ✅
• Simple test for monotone increase (round-1-vs-round-K McNemar + a non-decreasing check). ✅
• Unit test on a hand-built transcript where adoption grows (tests/test_multi_round.py, 4 tests). ✅

Reproduction

Keyless from the committed cache (new_api_calls_this_run = 0). The per-round analysis (per_round_holdout_answers, adoption_curve, is_monotone_nondecreasing) is pure, no key/network. ruff clean. See experiments/cascade/README.md.

[sebasmos]

Parser-bug correction (2026-07-21). The answer parser (_parse_choice / _parse) matched the first \b([A-E])\b in the response, which on long reasoned answers is the stray article "A", so it scored ~85%% (flash) / ~97%% (flash-lite) of answers as option A. The parser is now fixed on this branch (prefers \boxed{X} / "final answer is X" / the last named option). Corrected numbers:

• Multi-round (corrected): the anchored-cascade adoption is real, ~0.28-0.33 shared vs 0.0 isolated, still flat over 5 rounds (single-shot, McNemar p=0.63). Committed artifacts to be regenerated.

[Agastya191]

ood work on multi_round.py and test_multi_round.py. Splitting the pure per-round analysis (per_round_holdout_answers, adoption_curve, is_monotone_nondecreasing) out from the runner and testing it against a hand-built transcript where adoption does grow is the right way to make a null result credible, and the _parse fix was a good catch.

The first problem is that experiments/cascade/README.md and the pull request body still carry the pre-fix numbers. The table reads 0.175, 0.15, 0.175, 0.15, 0.15 with McNemar p = 1.0, while the committed results/multi_round_summary.json regenerated in c4aac6e reads 0.275, 0.3, 0.275, 0.325, 0.325 with gained 3, lost 1, p = 0.625. The writeup contradicts the artifact of record, and "the shared curve is flat (~0.15-0.18 every round)" does not describe a curve that drifts up about five points from round 1 to round 5. I would regenerate the table and the Read paragraph directly from multi_round_summary.json and restate the finding as no significant compounding at p = 0.625 rather than as flat.

The second is that in multi_round.py the isolated arm cannot come out nonzero. The isolated holdout's prompt is _mcq(case) with an empty board prefix, byte-identical to the base_p used to compute bare, so at temperature 0 off the same cache it returns bare in every round, and wrong is selected with the condition o != bare. That makes the isolated 0.0 row true by construction rather than measured, so it carries no weight as the solo counterfactual. You have options here too. I would either give the isolated holdout a prompt that lets its trajectory actually move, or keep the row and say plainly in the README that the isolated arm is zero by design.

[sebasmos]

Closing in favor of #225: both points fixed (stale README/body regenerated, and the isolated arm's real bug: its prompt was byte-identical to the bare prompt every round since board text only ever read peer votes, which are never visible in isolated mode, so adoption there was 0 by construction, not measurement; now reminds the holdout of its own previous-round answer instead, run for real and re-verified keyless), rebased fresh off current main so it does not delete #142/#143/#146/#148/#154/#157/#161/#219.