Skip to content

Clarify PR 18 statistical validation#19

Merged
drernie merged 4 commits into
mainfrom
codex/validate-pr18-followups
Jul 18, 2026
Merged

Clarify PR 18 statistical validation#19
drernie merged 4 commits into
mainfrom
codex/validate-pr18-followups

Conversation

@drernie

@drernie drernie commented Jul 18, 2026

Copy link
Copy Markdown
Contributor

Summary

This follow-up to #18 independently validates its algebraic calculations, Monte Carlo analysis, and Measurement 5.6 wording, then corrects several statistical and descriptive issues.

The core results from #18 remain intact:

  • the algebraic audit reproduces the 84-element crack, the Phi spectrum, the 120-dimensional Lie closure, and the 256-dimensional associative envelope;
  • Proposition 4.2’s left/right-handedness gauge follows from exact conjugation coupling;
  • Theorem 3.10(6)’s retained/sampled role-exchange symmetry is now directly certified for both transition functionals;
  • the event-resampling protocol reproduces the reported 0.00101 value for seed 9, with seeds 1–4 returning 0.00101–0.00103.

The two symmetry claims

This PR now distinguishes two claims that the historical discussion had conflated:

  • Theorem 3.10(6): exchanging the retained and sampled variables is invisible to the invariant transition functionals. The audit directly certifies τ(z,x)=τ(x,z) and A(z,x)=A(x,z) on 200 crack/pure pairs.
  • Proposition 4.2: after fixing which variable is retained, moving that role from the left side of multiplication to the right is conjugation-gauge. The conjugate-coupled trajectories agree pointwise.

The historical 0.13240 versus 0.13761 verdict purported to make retained/sampled role exchange detectable, so it concerned Theorem 3.10(6), not Proposition 4.2.

Historical D3 reconciliation

The original script used non-conjugate initial states, a continuum sampler, a sequential RNG stream, and an endpoint statistic. It coherently measured a noisy finite-time contrast, but it did not isolate a preferred role assignment.

The controlled reproduction separates the questions:

  • Exact historical design, 500 seeds: mean difference -0.000144, standard deviation 0.005270; seed 30 is -0.96σ, with 157/500 runs at least as extreme.
  • Paired initialization test, 200 seeds: mean e0-e1 effect +0.000045, 95% CI [-0.000437, +0.000526].
  • Left/right handedness with conjugate-related initial conditions and events: exactly zero by Proposition 4.2’s pointwise coupling.

Thus the historical gap is ordinary Monte Carlo variation for its actual design and provides no evidence against role-exchange invisibility.

Death-rate corrections

  • Survival-conditioned mortality now uses actual at-risk trajectory-steps rather than scheduled N*T exposure after trajectories have died.
  • Measurement 5.6 names the observables precisely:
    • rejected annihilating proposals per completed step under event resampling: about 1.0e-3;
    • annihilation hazard per at-risk step under survival conditioning: about 2.2e-4.
  • The event-resampling reproduction remains unchanged: seed 9 produces 0.00101; four additional seeds produce 0.00101–0.00103.

Other cleanup

  • Spine-share uncertainty is clustered by trajectory rather than treating repeated time observations as independent.
  • The stale audit assertion that only one orientation is stable is replaced with separate statements for external role assignment, role-exchange symmetry, and left/right gauge equivalence.
  • Historical scripts no longer claim that later discrete-design reruns reproduced the original continuum protocol exactly.
  • The stale death-rate follow-up documentation now records the eventual protocol resolution.

Checks

  • uv run --with pytest python -m pytest -q — 48 passed
  • uv run python verify/occurrence_i_audit.py — PASS
    • τ(z,x)=τ(x,z): maximum error 3.59e-16
    • A(z,x)=A(x,z): maximum error 2.78e-17
  • uv run python verify/occurrence_i_rigor_historical_d3_test.py — reproduced seed 30 exactly; 500 historical-protocol and 200 paired-initialization replications
  • uv run python verify/occurrence_i_rigor_gauge_check_survival.py — reproduced corrected clustered uncertainties and at-risk hazards
  • uv run python verify/occurrence_i_rigor_death_rate_followup.py — reproduced tolerance and horizon checks with corrected exposure
  • uv run python verify/occurrence_i_rigor_death_rate_resolution.py — reproduced 0.00101 for seed 9 and 0.00101–0.00103 across additional seeds

@drernie
drernie marked this pull request as ready for review July 18, 2026 13:11
@drernie
drernie merged commit b62ea83 into main Jul 18, 2026
2 checks passed
@drernie
drernie deleted the codex/validate-pr18-followups branch July 18, 2026 19:38
@drernie

drernie commented Jul 18, 2026

Copy link
Copy Markdown
Contributor Author

Post-merge review — closure-level rubric

From Bench d'Claude

Ran this PR through the occurrence/spec evaluation rubric: decompose the change into atomic claims and score each at three independent levels —

  • Mathematical — true as a theorem/derivation, independent of any run?
  • Reproducibility — does the original code + seed rerun to the reported number?
  • Causal — is the mechanism behind the effect identified (demonstrated cause vs. merely-contributing factors)?

Verdict per level: confirmed / refuted / open / n/a. Reviewer: Bench (Claude), from a fresh clone at PR head 968e66f, numpy installed — all certificates and scripts executed, not inspected.

CLAIM 1: Corrected [D3] assertion — role exchange (retained vs. sampled) is symmetric
         (Thm 3.10(6)); left/right handedness is separately conjugation-gauge (Prop 4.2),
         replacing the prior "only one orientation is stable."
  Mathematical:    confirmed  — ran the audit; backing identities certify at machine zero:
                                C13b 2.8e-16, C13c-tau 3.5e-16, C13c-A 0.0 (tol 1e-12).
                                The old assertion was the refuted one.
  Reproducibility: confirmed  — full audit exits 0 on this branch, 0 failures.
  Causal:          n/a        — theorem-backed correctness fix; no empirical mechanism.

CLAIM 2: New executable certificates C13c-tau, C13c-A verify tau and A are
         exchange-symmetric on crack/pure pairs.
  Mathematical:    confirmed  — exact operator computation over 200 crack/pure samples;
                                errors 3.5e-16 / 0.0.
  Reproducibility: confirmed  — re-ran, pass.
  Causal:          n/a        — a certificate; no empirical effect.

CLAIM 3: Measurement 5.6 now names its two observables correctly — 1.0e-3 = rejected
         annihilating proposals per completed trajectory-step; 2.2e-4 = annihilation
         hazard per at-risk trajectory-step; difference = survivorship selection.
  Mathematical:    n/a        — an estimand-naming correction, not a theorem.
  Reproducibility: confirmed  — re-ran BOTH: attempt rate seed 9 -> 0.00101 (exact match;
                                seeds 1-4 -> 0.00101-0.00103); hazard ~1.8-2.2e-4 across
                                N=200-20000, tolerance-invariant at 2.2178e-4.
  Causal:          confirmed  — the two numbers measure different populations by
                                construction; the gap is the selection. Definitional, not
                                a fitted mechanism.

CLAIM 4: The historical 0.13240/0.13761 gap is reproduced and shown to be an
         uncontrolled comparison, not a real effect.
  Mathematical:    n/a
  Reproducibility: confirmed  — ran historical_d3_test.py: seed 30 -> standard=0.13240,
                                swapped=0.13761, exact to the digit.
  Causal:          confirmed  — DEMONSTRATED: the paired e0-e1 control isolates the
                                initial-condition axis, mean +0.000045, 95% CI
                                [-0.000437, +0.000526] (contains 0) — no effect when
                                controlled. CONTRIBUTING but not individually isolated:
                                continuum sampling, shared RNG stream.

BOTTOM LINE: Sound. Every certificate and every reported number re-ran clean — machine
zero or exact-to-the-digit. The [D3] fix removes a genuinely wrong assertion and replaces
it with one its own certificates back; Measurement 5.6's denominators are now honest.
Mathematical and reproducibility levels confirmed by direct execution, not inspection.
Agrees independently with Codex's "no changes required" and Rigor's merge recommendation.

Method: cloned the repo, ran verify/occurrence_i_audit.py (exit 0, 0 failures), verify/occurrence_i_rigor_historical_d3_test.py, verify/occurrence_i_rigor_death_rate_resolution.py, and verify/occurrence_i_rigor_death_rate_followup.py. The one thing outside my reach: I could not trigger/observe CI — worth a glance that the merge is green, though every check the PR ships passed locally.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant