Repository navigation
Data and analysis release for the third revision of the manuscript.
- Fix: in v0.1.11 the replicate directories of the counterbalanced end-to-end runs (experiments/audit_cb_) and of the interleaved temperature runs (experiments/temp_il_) were published incompletely (a single misnamed directory per run instead of rep01..rep12 / rep01..rep06). This release contains all 120 replicate directories; use this release, not v0.1.11, for those runs.
- New: harness/scripts/audit_sim.py and experiments/audit_sim.json, a simulation of the audit statistics with known truth (dense, opposing, sparse and population scenarios; vignette-level and replicate-paired sign-flip tests; excess detection; interval coverage). It replaces the injection into observed draws, which is no longer run.
- audit_validation.py now reports the number of non-zero vignette-level shifts, the smallest attainable sign-flip p-value and a post hoc replicate-paired sign-flip test.
- verify_paper_numbers.py checks 681 claims against these artifacts.