regen(examples): refresh all 12 regenerable walkthroughs + fix 4 latent defects

[lucapinello]

Why

11 of 13 committed examples were last regenerated 2026-04-21..04-29 and had never been refreshed, so they predated two correctness fixes:

• #92 T1 — variants were scored on a 1 bp region, which fixed-input oracles map into an N-padded output window, collapsing the effect. This moved the raw predictions.
• #119 — the percentile denominator was the raw sample count instead of the CDF grid width.

The committed examples had a median |percentile| of exactly 1.0000.

The cleanest proof is a numerical fingerprint: FTO's committed DNASE:HepG2 quantile is 0.38082765845992667, which is exactly 727/1909 — 1,909 being that track's effect_counts. Under #119 the same rank gives 727/10000 = 0.0727, a 5.24× inflation on that row.

What changed

example	median |pct|	≥95th with |effect| < 0.1
variant_analysis/SORT1_rs12740374	1.0000 → 0.8025	80 → 2
validation/TERT_chr5_1295046	1.0000 → 0.9646	104 → 42
validation/…_with_CEBP	1.0000 → 0.7657	82 → 2
variant_analysis/FTO_rs1421085	1.0000 → 0.6342	24 → 2
variant_analysis/BCL11A_rs1427407	1.0000 → 0.8669	18 → 4
sequence_engineering/region_swap	1.0000 → 0.9988	16 → 6
…/integration_simulation	1.0000 → 0.9984	16 → 10
variant_analysis/SORT1_enformer	0.9995 → 0.9587	10 → 4

batch_scoring is starkest: all 5 SORT1-locus variants previously read max_quantile = 1.0000, four of them for |effect| < 0.07. They now spread 0.9059–1.0000, and the four LD proxies land at 0.18–0.91 — a real gradient instead of a wall of ≥99th.

226 → 64 contradictory rows. The residual 64 are the genuine #83 problem and concentrate in RNA_SEQ and CAGE — exactly the layers measured to be unfixable by better position sampling. Those need the magnitude floor, not a rebuild.

The flagship SORT1 conclusion survives. Effects grew 3–7× under #92 (DNASE:HepG2 +0.449 → +1.332, CEBPB +0.274 → +3.046), so its ≥99th is now earned rather than an artefact.

Four defects fixed along the way

1. Three examples could not be regenerated at all. BCL11A_rs1427407, FTO_rs1421085 and SORT1_rs12740374_with_CEBP had their generator entries commented out — collateral in fc38632 ("fix: support mixed-resolution tracks … in IGV"), which added no prose, unlike the ENFORMER_EXAMPLES block twelve lines below whose deliberate removal ends "Do not re-introduce them." audits/2026-05-08_post_pr79_merge_audit.md:133 had flagged the decision as open; it sat for three months. They're live docs (README.md:149, examples/walkthroughs/README.md:108, plus a reproduction notebook each), so re-enabled after verifying all 13 track ids still resolve.

2. ⚠️ BCL11A was scoring a sequence that does not exist. It declared ref="G" at chr2:60490908; hg38 has T there (AAACA[T]TTCCC). base.py:464-470 only warns and substitutes, so the "ref" arm was a synthetic non-reference sequence and the "alt" arm was the actual reference base — the effect was inverted and measured against a sequence not in the genome. The warning fired on every regeneration and was invisible in the log.

Corrected to the hg38-oriented T>G here, in generate_walkthrough_notebooks.py:179 (which had independently declared the same wrong allele), in the notebook, and in three prose references. Every BCL11A effect sign flips. Regenerated with zero mismatch warnings. 1 of 4 declared examples was wrong; the warn-and-substitute default is #128.

3. Two committed TSVs were never regenerated. example_output.tsv was written only in the AlphaGenome path, so the Enformer and ChromBPNet paths rewrote example_output.json and left the .tsv in the same directory alone. They disagreed about which tracks were even scored — ENCFF571HTM @ quantile 1.0 vs ENCFF430NNH @ 0.9605. Extracted _write_tsv() and called it from all three paths; also taught rerender_examples.py to refresh an existing TSV from the same rehydrated report.

4. Two READMEs documented the bug as intended behaviour. SORT1_rs12740374/README.md explained the uniform ≥99th as a display choice against "~10,000 random SNPs" — the background is 1,697–1,909 random genomic positions with a random alt allele: not 10,000, not SNPs, and not gnomAD (no code samples gnomAD, despite docs/NORMALIZATION_GUIDE.md:389). batch_scoring/README.md carried the same claim, conflated effect percentile with regional activity, and its input snippet listed rs629301 while its output table listed rs1626484 at the same position with different alleles — and rs629301/rs12037222/rs2228603 aren't scored by the generator at all. Both rewritten with measured statistics; all three variant lists realigned to the generator.

Also: advanced_multi_oracle_analysis.ipynb cell 62 referenced images/MA0139.1.svg, resolving to a nonexistent examples/notebooks/images/.

Known gaps, deliberately left

• rs12740374_SORT1_legnet_report.html and the consolidated multi-oracle report regenerate at 131 MB / 139 MB (from 1.29 MB / 9.46 MB) — above GitHub's hard 100 MiB limit, so they cannot be committed. Both left at their previous bytes and therefore now stale against their siblings. Known as P0 in AUDIT_CHECKLIST.md:172 with a manual guard I failed to run. Filed as #129 with a proposal to enforce it in to_html() rather than a checklist line.
• #126 — LegNet's percentiles have never worked (builder writes bare HepG2, oracle asks LentiMPRA:HepG2; all 3 rows unreachable), which is why the multi-oracle LegNet cell renders an em dash that reads like deliberate suppression.
• #127 — two identical runs disagree on 454 numeric fields with 36 sign flips; for CAGE the run-to-run noise (median 0.0054) exceeds the effect being reported (median 0.0058).

One regression worth a look before merging

discovery/SORT1_cell_type_screen swaps its 3rd cell type, left lobe of liver → amniotic epithelial cell. Not a percentile artefact — that screen ranks by alt_value × |effect| (raw values), so it's #92's windowing fix. The new pick has ref_value 53.0 vs 184.9 and |effect| 2.90 vs 1.84: a bigger fold-change on a 3.5× quieter baseline. For a canonical liver variant that's a worse answer, and it argues for the low-activity qualifier proposed in #83 — _is_low_baseline needs |effect| > 1.5 and alt_value < 5, so it doesn't catch this one.

Verification

pytest -m "not integration": 507 passed, 4 skipped, 0 failed

Full write-up in audits/2026-08-01_example_refresh_and_mouse_removal.md, which also covers #121 (the mouse-model removal) and the HuggingFace dataset revision.

Independent of #121 — no example touches a mouse track or a ChromBPNet CHIP row, so the two can merge in either order.

🤖 Generated with Claude Code