Cascade: hard-case / uncertainty selector (where cascades can actually form)

[sebasmos]

Context

Cascades need an uncertain committee. Seeding a confident-correct committee (random MedQA items are mostly easy) is the worst case for finding an effect. We need to select the cases where the solo models are near chance or wrong, and run the cascade there.

What to build

A selector that, given per-model solo results (we already have solo_records.jsonl from the full run), returns the case ids where the committee is uncertain, e.g.:

• solo accuracy on the clean twin is at/below chance, or the models disagree, or confidence is low;
• expose a select_uncertain_cases(records, k) helper and a CLI flag on the cascade runner to restrict to them.

Acceptance

• Given the existing solo_records.jsonl, the helper returns a non-empty, deterministic set of "hard" case ids.
• The cascade runner can be pointed at that subset.
• Unit test on a small synthetic record set (some easy, some hard) asserts only the hard ones are selected.

Pure analysis over existing outputs; no data or key needed to test.