Skip to content

Gate-2 amendment proposal 1: mean-over-draws scoring - #96

Merged
MaxGhenis merged 2 commits into
masterfrom
gate2-amendment-proposal
Jul 8, 2026
Merged

Gate-2 amendment proposal 1: mean-over-draws scoring#96
MaxGhenis merged 2 commits into
masterfrom
gate2-amendment-proposal

Conversation

@MaxGhenis

Copy link
Copy Markdown
Contributor

Gate-2 amendment proposal 1: mean-over-draws scoring

A public amendment PROPOSAL for gate 2, mirroring gate 1's amendment-2 ceremony. It is an inert object: gates.yaml gate_2 stays locked: true, status: locked, and the entire thresholds subtree is byte-identical to origin/master (the change is a pure 671-line insertion of the gate_2.amendment_proposed sibling; a test byte-compares the subtree). No model reads it; the harness scores only the locked block. Nothing is ratified — this is the object a referee round adjudicates.

The estimator change

Per-cell scoring moves from one frozen simulation draw to the mean over K=20 draws per gate seed:

  • Locked: |ln(r_candidate,s / rate_a,s)|, r_candidate,s from one replicate (sim_seed 4200+s).
  • Proposed: |ln(r̄_candidate,s / rate_a,s)|, where is the mean over K=20 draws of the cell rate (the mean of the candidate statistic across draws, not the mean of the |ln| scores). Draw seeds default_rng(5200+k), k=0..19 — the committed forensics convention (Gate-2 chronic-cell forensics: pathway, parity, and draw-noise decomposition #94).
  • Tolerances UNCHANGED. The 46-cell / 4-of-5-seed conjunction UNCHANGED. Only the candidate statistic each cell is scored on changes.

Why it's an estimator fix, not a loosening. The locked tolerances derive from the 100-seed half-vs-half split floor (round(floor mean + 4·floor sd, 3)), which is real-vs-real and contains no simulation-draw noise. The single-draw estimator injects a per-cell draw-noise term the tolerance never budgeted for. Averaging over 20 pre-registered draws shrinks that injected noise by √20 and moves the certified statistic toward the draw-noise-free rate the tolerance was measured against — aligning the estimator with the tolerance's own derivation basis without touching the error budget.

Operating characteristics (recomputed from the committed forensics #94)

Per failing cell: measured tilt vs tolerance, the single-draw clip probability the locked estimator carries, and the mean-of-20 estimator's. Single-draw column reproduces the artifact's committed prob_train_draw_clips_tolerance to 1e-16; mean-of-20 is the same rate-scale normal model with sd/√20. Train-side (side B vs rate_b) — the only committed multi-draw evidence.

cell tol |tilt| tilt/tol single-draw clipP mean-of-20 clipP verdict
mean_lifetime_marriages|male 0.047 0.0463 99% 0.49 0.59 BOUNDARY
share_widowed.75+|female 0.185 0.1573 85% 0.34 0.20 BOUNDARY
mean_lifetime_marriages|female 0.051 0.0361 71% 0.10 0.0005 NOISE-DOMINATED
share_divorced.45-54|female 0.212 0.1033 49% 0.03 0.00 NOISE-DOMINATED
widowhood.75+|female 0.373 0.1020 27% 0.00 0.00 NOISE-DOMINATED
completed_fertility.c1970s 0.171 0.0597 35% 0.00 0.00 NOISE-DOMINATED

The estimator is not a pass-machine. The four NOISE-DOMINATED cells collapse toward 0 (their single-draw clips were draw noise). The two BOUNDARY cells are not rescued: share_widowed.75+|female stays material, and mean_lifetime_marriages|male rises 0.49 → 0.59 because its systematic tilt genuinely exceeds tolerance on 4 of 5 seeds — averaging drives the estimator toward that real level, which fails harder. A model with gross level errors fails by orders of magnitude at any K: candidate 2's committed cells clip at 2.4–4.5× tolerance (e.g. share_widowed.65-74|female score 1.359 vs tol 0.300 on all 5 seeds), 15–136× the draw-noise sd.

No self-rescue

Prospective-only (inherits gate_2.governance.amendment_rules, which inherits gate 1). Candidates 1–9's committed verdicts stand (all FAIL). The illustrative_retroactive_application block is applied: false: it examines candidates 8 and 9 (the triggering runs) and honestly marks the outer verdict NOT_COMPUTABLE_OUTER — the committed gate-2 artifacts hold only one outer draw per seed, and the forensics' 20 draws are train-side, so recomputing the amended outer verdict would need new outer simulations this proposal does not run. The train-side reading shows neither would pass anyway (each keeps a real boundary-level residual). Path to pass: a fresh candidate-10 registration, one-shot on seeds 0–4 under the amended estimator.

Considered and rejected

  • Widen tolerances — changes the certified error budget; the estimator fix does not.
  • Unfreeze the RNG / best-of-N draws — adversarial selection; the mean over a pre-registered fixed K is unbiased.
  • K=100 — 5× compute for ≤1.05× precision (the split floor dominates total sd; the draw term is already sub-dominant at K=20). Arithmetic recorded and machine-checked.

Compute cost

One committed one-shot run (candidate 8) took 65.9 s for 5 seeds × 1 draw; K=20 multiplies the per-seed simulation ~20× → ≈ 22 min/candidate. The forensics measured 20 train-side draws × 5 seeds at 155.1 s per-seed compute — minutes, not hours. Recorded and acceptable.

Tests

A parallel gate-2 amendment block in tests/test_gates_derivations.py (mirroring the gate-1 amendment tests) binds the proposal to the committed artifacts: per-cell OC recomputes from runs/gate2_forensics_v1.json; K=20 and the 5200+k seed convention pinned; the k=100 arithmetic and compute cost recompute; applied: false with candidates 8/9 FAIL and the outer verdict marked not-computable; and a byte compare of gate_2.thresholds vs origin/master proving no locked value moved. All are dormant-safe (skip if amendment_proposed is absent), exactly as the gate-1 proposal-object tests went dormant after ratification.

Ceremony next steps

Adversarial referee round on this proposal → fixes if any → verification → maintainer ratification by merge → a follow-up that flips the estimator live in the locked protocol (as PR #79/#81 did for the gate-2 lock and #67/#69 for gate 1's amendment 2). Do not merge this PR as a ratification.

Evidence chain: #94 (forensics) + #93 (candidate 8) / #95 (candidate 9) runs + the candidate 1–9 ladder on #42.

🤖 Generated with Claude Code

…no self-rescue)

Adds gate_2.amendment_proposed to gates.yaml as an inert public proposal
object (mirrors gate 1's amendment-2 pattern). Per-cell scoring moves from
one frozen simulation draw to the mean over K=20 draws per gate seed (draw
seeds 5200+k), at the SAME locked tolerances and 4-of-5 conjunction. No
locked value changes: gate_2.thresholds is byte-identical to origin/master
(the diff is a pure insertion), locked stays true, and no model reads the
block.

Carries the full ceremony structure: changes with machine-bound derivations,
operating-characteristics evidence recomputed from runs/gate2_forensics_v1
.json, honest disclosure (the estimator is not a pass-machine -- boundary
level cells are not rescued; mean_lifetime_marriages|male rises 0.49->0.59),
considered_and_rejected (widen tolerances / best-of-N / K=100), compute-cost
note (~20 min/candidate), path_to_pass (fresh candidate 10), and an
illustrative_retroactive_application block with applied:false marking the
outer verdict not-computable (candidates 8/9 stay FAIL).

Parallel gate-2 amendment test block in tests/test_gates_derivations.py binds
the OC recompute, K=20 + seed convention, applied:false, the k=100 arithmetic
and compute cost, and a byte compare of the locked thresholds vs
origin/master. All dormant-safe (skip when amendment_proposed is absent).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@vercel

vercel Bot commented Jul 8, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
social-security-model Ready Ready Preview, Comment Jul 8, 2026 2:48pm

Request Review

@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Verdict: AMEND BEFORE RATIFYING.

Adversarial round on the gate-2 amendment-1 proposal object (head 9bb9305), against the standard set by gate 1's amendment-2 rounds (PR #67 comments 4904161939 / 4905067301). I recomputed every load-bearing number from the committed artifacts in a detached read-only worktree — the forensics, the floors, candidates 1–9, and the proposal's own tables — and perturbation-tested the new bindings on a scratch copy. The estimator change itself is sound, candidate-independent, and verified to the last digit save one; the three required fixes are record-honesty and prospective-spec items, not redesign. Ratifiable after them.

1. Goalpost timing (check 1): accommodation-shaped on its face, but the estimator argument stands without candidates 8–9 — and the precedent's pressure-valve tell is absent. I verified the derivation-basis claim from runs/gate2_floors_v2.json: the floor is |ln(rate_A/rate_B)| between two person-disjoint REAL halves (no simulation anywhere in it), and all six cited tolerances recompute exactly as round(floor mean + 4·sd, 3) (e.g. mean_lifetime_marriages|male: 0.012429 + 4·0.008631 → 0.047). Decisively: the locked block's own ratified OC was computed with zero draw noisefaithful_candidate_oc.method models a faithful candidate's score as half-normal with the floor's realized_sigma, and I reproduce p_seed_pass 0.9404 and p_gate 0.9685 exactly as the product of those floor-only cell probabilities. The lock ceremony's round-1 finding 1 (PR #79 comment 4910467957) prescribed "one replicate, simulation seed = s" while asserting the half-vs-half floor "is then exactly the right null" — which is true for the candidate's expected rate, not for one replicate. So the single-draw estimator contradicted the locked design's own ratified operating characteristics from birth; this proposal resolves that contradiction in the direction the ratified OC already pointed, without touching the budget. Quantified (train-side forensics, independence approx): correcting only the six measured cells for single-draw noise, the faithful seed-pass drops 0.9404 → 0.885 (gate 0.9685 → 0.896; the truth is lower still — the other 40 cells' draw sds are unmeasured), while mean-of-20 restores 0.939 / 0.967 ≈ the ratified numbers. And the gate-1 precedent's tell — the triggering candidate flipping to PASS — is absent here, inverted even: a candidate with c8's full residual structure clears the gate with probability 0.017 under the locked estimator and 0.011 under the amended one (the male cell's per-seed tilt exceeds tolerance on seeds 0/1/2/4 and share_widowed.75+|female's on seed 0, so averaging fails it harder), while a candidate that fixed the two boundary cells but kept the four noise-dominated tilts goes 0.787 locked → 0.975 amended — i.e. the amendment stops failing the candidate class the ratified 0.9685 says should pass, and nothing else. Timing is disclosed plainly (process_statement), the shape was pre-committed in the c9 grading comment (4914756135: same tolerances, prospective-only, c1–9 stand) before drafting, and no_self_rescue is verbatim. No fix required on this finding.

2. OC integrity (check 2): every table number recomputes. From runs/gate2_forensics_v1.json I recomputed all twelve clip probabilities under the stated rate-scale normal model: the single-draw column reproduces the artifact's committed per-seed prob_train_draw_clips_tolerance to ≤1.1e-16, and the mean-of-20 column is the same model at sd/√20, matching all six YAML entries to 4 dp (0.4854→0.5927, 0.3396→0.2023, 0.1018→0.0005, 0.0264→0.0, 0.0048→0.0, 0.0036→0.0). Tilts, tilt/tolerance ratios, and the BOUNDARY/NOISE-DOMINATED partition match the forensics summary fields; the estimator is the mean of the RATE then |ln| (bound by test_gate2_amendment_k_and_seed_convention, which also pins K=20 and checks the artifact's draw_seeds == 5200..5219); the male boundary cell rises 0.49→0.59 exactly as disclosed. The c2 gross-level illustration holds: scores/tolerance 4.53 / 3.23 / 2.55, rate multiples e^score 3.89 / 2.38 / 2.44, and share_widowed.65-74|female clips on all five seeds (1.287–1.359). One digit does not recompute — required fix A: the c2 example's outer_score: 1.3593; the committed runs/gate2_hazard_v2.json seed-3 score is 1.358959… → 1.359. The string "1.3593" appears nowhere else in the repo; it slipped because test_gate2_amendment_c2_gross_level_still_fails binds at abs=1e-3, looser than the exact round(score, 4) equality the c8/c9 rows get. In an object that becomes the permanent amendment record, every stated number must recompute (the gate-1 rounds' standard).

3. Selection hazards (check 3): closed, with one record-honesty gap — required fix B. The draw-seed convention is frozen and candidate-independent: the 20-draw design was registered at candidate-8 grading time (#42 comment 4913512779, item 3) and the default_rng(5200 + k), k=0..19 enumeration committed in the forensics artifact's protocol — both before candidate 9's registration and before this proposal — and the stream is distinct from the single-draw 4200+s; change 2 pins "mean over ALL 20, no subset" foreclosing best-of-N; K matches the committed evidence rather than being tuned (and no future candidate's 20-draw outer rates are knowable in advance — correctly contrasted with gate-1's c10 situation). The K=100 rejection arithmetic recomputes exactly: max sd(K=20)/sd(K=100) over the six cells = 1.0506 at share_widowed.75+|female (0.03818 → 0.03634), next-worst 1.0331. The illustrative block is applied: false, both candidates keep committed_verdict: FAIL, and NOT_COMPUTABLE_OUTER is the honest call — the committed run artifacts hold exactly one outer draw per seed and the forensics' 20 draws are train-side; I verified all 7 (c8) + 8 (c9) disclosed failing cells match their artifacts' all_failing_gated_cells exactly at round(score, 4), with full coverage (no cherry-picking). The gap: the proposal's currently_locked.rule states the locked rule as "ONE simulation replicate (simulation seed 4200 + s)". The locked text does not say that — thresholds.protocol.candidate reads "(one replicate, simulation seed s)" (the floors-v2 protocol echoes it), while the 4200+s stream is the registration convention every committed run v1–v9 uniformly used (each artifact's protocol.sim_rng_rule: numpy.random.default_rng(4200 + seed)). Materially harmless — practice was uniform and registered per governance — but the amendment record must not present committed practice as locked text (precedent finding-1/-4 class: the description of the rule must recompute from the contract). Fix: one added line in currently_locked naming the divergence (locked text says seed s; every committed registration used 4200+s), and the flip checklist in fix B below.

4. Contract coherence + candidate-10 spec (check 6): the flip's edit list, and three prospective gaps — required fix C. No locked language blocks the proposal as an inert object, but the flip PR must change, at minimum: thresholds.protocol.candidate ("one replicate, simulation seed s" → K=20 draws at 5200+k, mean rate — also correcting the seed-s wording per fix B), thresholds.statistic and thresholds.protocol.pass_rule (r_candidate → r̄_candidate), and a note that faithful_candidate_oc's draw-noise-free basis now matches the estimator (its numbers stand; under the old estimator they were unachievable — finding 1); the leftover "DRAFT seed-level conjunction" label inside the locked pass_rule should be cleaned in the same authorized edit. For a candidate-10 run the proposal underspecifies three things a referee round will need: (i) per-draw storage — pin that the fresh run's artifact commits the per-draw per-cell rates (20 × 46 × 5), not just the means, so r̄ recomputes from the artifact; (ii) an undefined-draw rule — what happens if a gated cell's rate is undefined on some draw (empty simulated denominator); it must be pre-specified (e.g. run-invalidating), because any silent draw-skip reintroduces exactly the selection change 2 forecloses; (iii) a per-draw dispersion disclosure — the mean-of-20 makes per-draw over-dispersion invisible (the single-draw estimator penalized it, accidentally but really), so mirror gate-1 amendment 2's catastrophe-cap instinct at report-only weight: commit per-cell per-draw sd alongside the mean (a hard cap is optional; gate-1's proved decorative on the evidence).

5. Scope, tests, and suite (checks 4+5): clean. git diff origin/master...HEAD is a pure insertion — gates.yaml +671/−0 in a single hunk (the amendment_proposed sibling; set(gate_2) − set(master.gate_2) == {amendment_proposed}), tests +402/−0 (one import + one appended block) — so the locked thresholds subtree is byte-identical to origin/master at the diff level, which is stronger than the test's parsed-subtree equality (minor: the docstring/PR-body say "byte-compares"; either say structural compare or add the diff fact). All 11 new tests are dormant-safe (skip when amendment_proposed is absent — the gate-1 proposal tests in the same file are correctly skipping now, 10 of them) and genuinely load-bearing: on a scratch copy I perturbed mean_of_20_clip_prob 0.5927→0.5827, applied: falsetrue, K 20→25, and locked tolerance 0.047→0.048 — each broke its specific test (the last also broke the pre-existing floor binding), and restoring returned green. Compute cost recomputes: c8 committed 65.9 s; ×20/60 = 21.97 → 22 min; the forensics' per-seed compute is 29.3–32.5 s across the five seeds summing 155.1 s (nit: the forensics_measured_20draw_5seed_per_seed_compute_seconds field holds the 5-seed total). Full suite from the worktree: 983 passed, 41 skipped, 0 failed (exit 0, 18m42s).

Required fixes.
A. Correct the c2 illustration's outer_score 1.3593 → 1.359 (committed value 1.358959…), and tighten test_gate2_amendment_c2_gross_level_still_fails to bind outer_score == round(score, 4) exactly as the c8/c9 rows do. (2)
B. Add one line to currently_locked disclosing that the locked protocol text reads "one replicate, simulation seed s" while every committed run's registered convention was default_rng(4200 + seed) (uniform v1–v9), and extend the flip note to enumerate the locked-text edits the flip performs: protocol.candidate, statistic, pass_rule, the faithful_candidate_oc basis note, and the stray "DRAFT" label. The amendment record must describe the locked rule as written, not as practiced. (3, 4)
C. Pin the candidate-10 run contract in path_to_pass (or a fresh_run_artifact_schema key): per-draw per-cell rates committed in the run artifact; a pre-specified undefined-draw rule (no draw may be dropped or substituted post-registration); and a report-only per-cell per-draw sd disclosure. (4)

The core change — mean over 20 pre-registered draws of the cell rate, at byte-identical tolerances, prospective-only — needs no redesign: it is the estimator the locked block's own ratified OC and the lock-round's "exactly the right null" claim already presupposed, it fails the triggering candidates harder, and every operating characteristic recomputes from committed artifacts. Fix the record; then ratify.

A: c2 illustration outer_scores set to round(committed,4) from
   runs/gate2_hazard_v2.json (1.3593->1.359, 0.869->0.8689,
   0.894->0.8938); c2 test tightened from abs=1e-3 to exact
   round(score,4), matching the c8/c9 convention.
B: add currently_locked.registration_divergence (locked text says
   "simulation seed s"; all committed runs v1-v9 used
   default_rng(4200+seed) per protocol.sim_rng_rule); add
   flip_on_ratification.flip_edits enumerating the flip's locked-text
   edits (protocol.candidate, statistic, pass_rule r->rbar,
   faithful_candidate_oc basis note, stray DRAFT label); fix "byte
   -compares"->parsed-compare wording and relabel the 155.1s field as
   the 5-seed total.
C: add fresh_run_artifact_schema (referenced from path_to_pass): per
   -draw per-cell rates (20x46x5) so rbar recomputes; a run
   -invalidating undefined-draw rule (no silent drop); a report-only
   per-draw dispersion disclosure (per-cell per-draw sd + max per-draw
   |ln| per cell).

Amendment tests updated to bind A/B/C, dormant-safe (skip when
amendment_proposed is absent). Locked gate_2.thresholds unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Referee fixes A/B/C applied

Applied the three required fixes from the adversarial review (comment 4915412987). The estimator proposal itself is unchanged; the locked gate_2.thresholds subtree stays byte-identical to origin/master; every new test is dormant-safe (skips when amendment_proposed is absent — verified). Proposal stays draft, pending the referee round.

A — the c2 illustration now recomputes to the last digit

c2_gross_level_illustration outer_scores set to round(committed score, 4) from runs/gate2_hazard_v2.json, and the test tightened from abs=1e-3 to exact round(score, 4) — the same convention the c8/c9 rows already use:

cell (seed) was now committed
share_widowed.65-74|female (3) 1.3593 1.359 1.358959…
share_divorced.55-64|female (3) 0.869 0.8689 0.868860…
widowhood.45+|male (0) 0.894 0.8938 0.893838…

B — record honesty

  • currently_locked.registration_divergence added: the locked text (thresholds.protocol.candidate) reads "one replicate, simulation seed s" — it does not name the 4200 offset — while every committed run v1–v9 uniformly registered numpy.random.default_rng(4200 + seed) (each artifact's protocol.sim_rng_rule, all nine identical, verified in-test). Disclosed as a registration-vs-locked-text divergence; materially harmless.
  • flip_on_ratification.flip_edits enumerates every locked-text edit the flip PR will make: protocol.candidate wording, statistic (r→r̄), pass_rule (r→r̄), the faithful_candidate_oc basis note, and the stray "DRAFT" label. Each entry's locked_text_now is verified present at its locked_path in the current locked block by test.
  • Wording nits: the test docstring/comment "byte-compares" → "parsed (structural) compare" (the test compares the parsed thresholds subtree); the 155.1 field renamed …_per_seed_……_total_… and labelled the 5-seed total (= forensics total_per_seed_compute_seconds = sum of the per-seed 29.3–32.5 s), bound in-test.

C — candidate-10 run contract (fresh_run_artifact_schema, referenced from path_to_pass)

  • (i) per-draw per-cell rates committed in the run artifact (20 × 46 × 5) so r̄ recomputes cell-by-cell from the artifact;
  • (ii) a pre-specified undefined-draw rule — a draw with an undefined/empty cell (zero simulated denominator) invalidates the run (no silent drop, skip, substitute, or re-roll), so no post-hoc draw selection re-enters;
  • (iii) a report-only per-draw dispersion disclosure — per-cell per-draw sd and max per-draw |ln| per cell, reported not gated.

Tests / lint

tests/test_gates_derivations.py: c2 test tightened, compute-cost test extended to bind the relabelled total, plus three new bindings (test_gate2_amendment_registration_divergence_disclosed, …_flip_edits_enumerated, …_fresh_run_artifact_schema). All gate-2 amendment tests skip cleanly when amendment_proposed is removed (dormant-safe). black -l 79 and ruff clean.

Verified: test_gates_derivations.py — 45 passed / 10 skipped (gate_2 amendment bindings all green). Working tree differs from origin/master only in gates.yaml and tests/test_gates_derivations.py; the locked gate_2.thresholds subtree is untouched.

@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Verdict: RATIFY AS-IS.

Verification round on fix commit 5b70840 (round 1: comment 4915412987, AMEND BEFORE RATIFYING with findings A/B/C; fixes summarized in comment 4916048161). I re-verified in a detached read-only worktree at 5b70840, recomputing every checked number from the committed artifacts rather than trusting the fix summary or the tests. All three findings are genuinely resolved, the fix commit touched nothing beyond what it claims, and nothing new broke. This proposal object is ready for maintainer ratification by merge; the estimator flip remains a separate authorized follow-up PR per flip_on_ratification.

A — resolved: all three c2 digits recompute exactly, and the binding is now exact. I recomputed each c2_gross_level_illustration outer_score from runs/gate2_hazard_v2.json (both the per_seed[*].gated_cells and verdict.all_failing_gated_cells paths — they agree):

cell (seed) committed score round(·,4) stated
share_widowed.65-74|female (3) 1.3589595608… 1.359 1.359 ✓
share_divorced.55-64|female (3) 0.8688597516… 0.8689 0.8689 ✓
widowhood.45+|male (0) 0.8938381198… 0.8938 0.8938 ✓

Note the fix corrected not just the flagged 1.3593 but also the other two rows (0.869 → 0.8689, 0.894 → 0.8938), which the tightened convention required — under the old abs=1e-3 binding both would have silently passed. test_gate2_amendment_c2_gross_level_still_fails now asserts outer_score == round(src["score"], 4) — exact, same convention as the c8/c9 rows. Perturbation: reverting 0.8689 to the round-1 value 0.869 (which the old loose binding accepted) now fails the test; restore → green. Secondary fields re-verified: tolerances match the artifact, score/tolerance ratios 4.5/3.2/2.5, rate multiples 3.9/2.4/2.4, and the seed-invariant recomputes (clips on all 5 seeds, range 1.287–1.359).

B — resolved: the record now describes the locked rule as written, and the flip is enumerated.

  • currently_locked.registration_divergence is present and accurate on both sides: the locked gate_2.thresholds.protocol.candidate really reads "…SIMULATES side A's persons' family histories (one replicate, simulation seed s)…" and the string 4200 appears nowhere in the locked thresholds subtree; and all nine committed artifacts (runs/gate2_hazard_v1.jsonv9.json, not just a spot-check) have protocol.sim_rng_rule == "numpy.random.default_rng(4200 + seed)", identical.
  • flip_on_ratification.flip_edits enumerates exactly the five edits round 1 required — protocol.candidate wording, statistic (r→r̄), protocol.pass_rule (r→r̄), the faithful_candidate_oc basis note, and the stray "DRAFT" label. I resolved each entry's locked_path in the live locked block and confirmed each locked_text_now is present at it (5/5); the quoted OC numbers in the basis-note edit (p_seed_pass 0.9404, p_gate_pass_4_of_5 0.9685) match the locked block's values. Perturbation: changing one locked_text_now to text absent from the locked block (r_candidate,sr_candidateX,s) fails test_gate2_amendment_flip_edits_enumerated; restore → green.
  • Wording nits applied: the test docstring/comment now says parsed (structural) compare — with the accurate added fact that the pure-insertion diff makes the subtree byte-identical at diff level too — and the 155.1 field is renamed forensics_measured_20draw_5seed_total_compute_seconds, labelled as the 5-seed TOTAL. That relabel is correct: the forensics artifact's per_seed_compute_seconds = {29.3, 31.6, 31.2, 32.5, 30.5}, summing to exactly 155.1 = total_per_seed_compute_seconds. The test asserts the old per-seed-labelled key is gone and binds the new one to both artifact fields. Perturbation: renaming the field back fails test_gate2_amendment_compute_cost_recorded; restore → green.

C — resolved: the candidate-10 run contract is pinned. fresh_run_artifact_schema exists with all three required elements: (i) per_draw_per_cell_rates required, shape: [20, 46, 5] (K_draws × gated_cells × gate_seeds), rule requiring every draw's per-cell rate so r̄ recomputes from the artifact; (ii) undefined_draw_rule pre-specified and required — an undefined cell rate on any draw invalidates the run, "No draw may be dropped, skipped, substituted, or re-rolled post-registration; rbar is always the mean over ALL K registered draws" — closing the silent-drop selection channel; (iii) per_draw_dispersion_disclosure with report_only: true, gated: false, committing both per_cell_per_draw_sd and max_per_draw_abs_ln_per_cell. path_to_pass references the schema by name and summarizes all three requirements. test_gate2_amendment_fresh_run_artifact_schema binds shape, invalidation wording, non-gating, and the path_to_pass reference. Perturbation: shape [20,46,5] → [20,45,5] fails the test; restore → green.

Cross-cutting — nothing new broke.

  • Scope: git diff origin/master is still a pure insertion — gates.yaml +803/−0 in a single hunk, tests/test_gates_derivations.py +531/−0 (one import + one appended block), no other files. The insertion occupies exactly the amendment_proposed block (lines 1067–1869); I byte-compared the live gate_2 locked region (lines 634–1066) against origin/master with cmpbyte-identical — and parsed-compared name/description/thresholds and the entire rest of the file — all equal. set(gate_2) − set(master.gate_2) == {amendment_proposed}; thresholds.locked: true, status: locked unchanged.
  • Fix-commit surface: the 9bb9305 → 5b70840 diff removes only the three c2 digits, the per-seed-labelled compute field and its prose, the byte-compare wording, the old abs=1e-3 assert, and one path_to_pass sentence end — i.e., exactly the advertised fixes. No OC number, tilt, verdict, tolerance, or change-1/change-2 text was touched.
  • Round-1 content still recomputes: independent recompute (own scipy implementation, not the test helper) of three OC cells from runs/gate2_forensics_v1.jsonmean_lifetime_marriages|male single-draw 0.485383 (matches committed prob_train_draw_clips_tolerance to 1.1e-16) → mean-of-20 0.5927 (rises, BOUNDARY); share_widowed.75+|female 0.3396 → 0.2023; mean_lifetime_marriages|female 0.1018 → 0.0005 — all matching the YAML at 4 dp with tilts and verdicts agreeing with the forensics summary. The c8/c9 illustrative rows: 7/7 and 8/8 exact round(score,4) matches with full coverage of each artifact's all_failing_gated_cells; applied: false; outer verdicts still NOT_COMPUTABLE_OUTER.
  • Dormant-safety: on a scratch copy with the amendment_proposed block deleted, tests/test_gates_derivations.py runs 31 passed / 24 skipped, zero failures — all 14 gate-2 amendment tests (11 round-1 + 3 new) skip cleanly, alongside the 10 dormant gate-1 proposal tests.
  • Perturbations: four new-binding perturbations run on the scratch copy (c2 exact digit, flip_edit locked_text_now, schema shape, compute-field name) — each broke its specific test and restored green; scratch copy verified byte-identical to the branch afterward.
  • Full suite (repo .venv, from the worktree): 986 passed, 41 skipped, 0 failed in 20m49s (exit 0). That is round 1's 983 passed + exactly the 3 new fix-commit tests; skips unchanged at 41; the only warnings are benign sklearn lbfgs ConvergenceWarnings. The anticipated QRF sklearn-drift did not manifest: the gate-1-era QRF reproduction test guards on populace-fit (which pins scikit-learn<1.9 and runs in a dedicated venv), absent from this .venv (now sklearn 1.9.0), so it skips rather than drift-failing — even with PSID staged at ~/PolicyEngine/psid-data. I confirmed the same on origin/master (ran tests/test_gate1_qrf_baseline.py in a separate worktree: 9 passed, 1 skipped, exit 0, skip reason "populace-fit not installed"), and git diff origin/master --stat shows only gates.yaml and tests/test_gates_derivations.py differ — so had any failure occurred it would have been in a byte-identical-to-master file and pre-existing. None did. All 45 tests in the amendment's own file pass (14 amendment tests green).
  • Lint: black -l 79 --check and ruff check clean on the test file, as claimed.

Round 1 found the estimator change itself sound and candidate-independent; this round confirms the record now matches it digit-for-digit. Fixes A/B/C verified resolved. RATIFY AS-IS — merge ratifies the proposal object; the live flip follows in its own authorized PR per flip_on_ratification.flip_edits.

@MaxGhenis
MaxGhenis marked this pull request as ready for review July 8, 2026 15:28
@MaxGhenis
MaxGhenis merged commit fec27eb into master Jul 8, 2026
7 checks passed
@MaxGhenis
MaxGhenis deleted the gate2-amendment-proposal branch July 8, 2026 15:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant