Repository navigation
Highlights
Evidence: two blind controlled studies published
evals/runs/controlled/codex-20260825/andcodex-20260826/— each 28 held-out cases × 2 conditions × 3 fresh-context seeds = 168 real Codex executions, 84 randomized A/B pairs blind-rated by two independent non-OpenAI model raters, ratings frozen before unblinding, paired bootstrap (10,000 resamples) against preregistered gates.- Second run passes the primary quality gate for the first time: composite
+0.315, 95% CI[0.110, 0.579](threshold+0.3); framework ahead on 6/7 dimensions, concision at parity, win rate 33.3% → 45.2%, zero framework responses leaking a local path (45 of 46 critical-error records on the baseline side). - Efficiency gates still fail structurally:
7.9xmedian output-token overhead (p9026.7x) against a1.15x/1.3xbudget; semantic density per 1,000 tokens 1.9 vs 19.9. The controller keepseffectiveness_claim_eligible: falseand the claim statusinsufficient_evidence. No readability or efficiency gain is claimed. Both READMEs' Evidence status sections now report this instead of only the one-case pilot.
Contract fixes motivated by the first study
- The final response must contain the report itself — never a saved-file path, pointer, or scratch path — in every host adapter, the micro-contract,
SKILL.md, and the core contract. - Proportional ceremony: short single-session answers skip checkpoints, draft files, and script audits; the file-backed bookend is reserved for long, multi-session, multi-agent, or durable-artifact work.
Absorbed reporting standards (all mappings in docs/REPORTING-STANDARDS.md)
- BLUF (AR 25-50), SBAR, Minto pyramid, US plain-language / ISO 24495-1, NN/g scanning research, WCAG 2.2 heading structure, Google SRE postmortems, IBCS SUCCESS / ISO 24896:2026.
- A two-round scientific-reporting literature survey: ASA p-value discipline, effect sizes with intervals, causal-language ladder, CONSORT/PRISMA accounting, uncertainty-visualization and error-bar semantics, benchmarking-crimes countermeasures.
- De-AI tone capability distilled from the MIT-licensed
shuorenhuaskill: a fidelity contract (tone edits never change facts), CN/EN signal-to-action table, misfire protection for technical vocabulary.
New surface area
- 3 display modules:
ablation,benchmarking,natural-tone(5 → 8). - 4 template assets:
sbar-handoff,executive-onepager,rebuttal-response,release-card(8 → 12), each with an audit-clean finished example and receipts underexamples/templates-2026082{6,8}/. - 7 deterministic audit warnings:
generic-heading,heading-level-skip,long-sentence(CJK-aware),deep-list-nesting,cjk-halfwidth-punctuation,ai-tone-boilerplate, plus the chart-first slide contract; all silent on known-good fixtures and showcase reports. - Tightened slide surface contract from showcase feedback: 3+ data points must be charted with the takeaway encoded inside the visual.
Repository
- READMEs restructured after ARIS open-source conventions (numbered contents, dated What's New with PR links, collapsible release history, acknowledgements, star history).
- Evidence-first GitHub Pages landing page with the showcase deck PDF.
reportctl --version,CITATION.cff, and the skill card are back in sync at 0.7.0.
Verification
- 309/309 unit tests (Python 3.9),
check_test_envpreflight clean, benchmark harness smoke, and dist reproducibility all green at 2ade309. - Protocol and template content are unchanged since PR #5; this release adds only the version bump, the Evidence status rewrite, and the changelog close.
Full detail: CHANGELOG.md