C2ST forensics: what the classifier reads at 0.547 by MaxGhenis · Pull Request #54 · PolicyEngine/microcosm-dynamics

MaxGhenis · 2026-07-06T18:07:53Z

Reported-not-gated. No holdout contact. This is a forensic read of two already-run gate-1 candidates, not a gate run. Every reported number compares a candidate's seed-0 panel against the seed-0 train persons' real windows (or one candidate against the other, or two disjoint train halves) — never the holdout. gates.yaml and the committed gate runs/ artifacts are untouched. The holdout is projected in exactly one place — a sanity anchor that proves the mirrored classifier equals the harness C2ST bit-for-bit — and that check makes no gate decision and touches no locked threshold.

Both best candidates land pairs-view C2ST ≈ 0.547 against the holdout (segment splicing PR #51: 0.5456; anchored rank-transition kernel PR #53: 0.5470), just over the locked ≤ 0.53. The brief asked whether that is one shared residual signal and what coordinate the classifier reads. It is not a shared signal, and the two candidates fail through structurally different defects that happen to land at the same AUC.

The mirrored classifier is the gate's own: HistGradientBoostingClassifier(random_state=0), weighted 5-fold stratified-CV ROC AUC, equal class mass, real class 0 / candidate class 1 (the order score_view uses — the AUC is not symmetric in it). The sanity anchor reproduces the harness classifier_two_sample_auc to 1e-12 on both candidates and both views, so every attribution below is a statement about the gate classifier, not a look-alike. Features are the projected window columns: pairs (earnings_t0, earnings_t1, age), runs (earnings_t0, earnings_t1, earnings_t2, age), where t0 is the later period and t1/t2 the earlier (the harness lifts ahead rows); weights are not features.

Ranked findings (pairs view, candidate vs train, seed 0)

1. Shared-vs-distinct verdict: DISTINCT. The classifier trained to tell the two candidates apart scores 0.5815 (pairs) and 0.6349 (runs) — higher than either candidate's own separation from real data (splice-vs-train 0.5543, kernel-vs-train 0.5497). The real-vs-real noise floor (two disjoint train halves) is 0.5049. The candidates are more distinguishable from each other than from real; the 0.547 coincidence is two different defects, not one signal. (In the runs view the A-vs-B gap widens to 0.6349, exactly because the two defects diverge more where the joint is wider.)

2. Splice = a marginal-earnings defect (over-inflated upper tail). Its separation is carried by the earnings levels, marginally: earnings_t0 alone = 0.5423, earnings_t1 alone = 0.5343, and the earnings-only pair = 0.5542 ≈ the full 0.5543. Age is inert (0.4958). The distribution shows why: q99 of the earlier-period earnings is 286,823 vs train 260,000 and the top-confidence decile (the windows the classifier is surest are synthetic) has q99 = 368,939, q90 = 130,000 (train 100,000). The per-segment scale factor b / donor_boundary_earnings, clipped to [0.2, 5], multiplies already-high donor segments further out — a right-tail distortion the tree splits on directly.

3. Kernel = a joint-transition defect (persistence deficit), marginals intact. Every single-period earnings marginal is ≈ chance (earnings_t0 = 0.5026, earnings_t1 = 0.5030), but the (t0, t1) joint = 0.5465 ≈ the full 0.5497 — the signal lives entirely in the dependence between consecutive periods. The log persistence correlation is 0.7072 vs train 0.7304 (too weak), and the top-confidence decile is windows with two high positive earnings that fail to co-move: both-positive share 0.948 (train 0.795), persistence 0.521, q50 57,833 (train 34,496). The 20-bin rank discretization + within-bin uniform placement + add-one (Laplace) smoothing reproduce the marginals (via Qhat) but flatten the fine joint. In the runs view this compounds: the two-step-apart pair earnings_t0 + earnings_t2 is the most separable pair (0.5794, above adjacent pairs), and the runs-view C2ST (0.5959 vs train; 0.5984 published) exceeds the pairs-view because a 3-window sees two chained kernel steps and the persistence error accumulates.

4. The round-number break is real but is NOT what the classifier reads. Real PSID earnings are whole dollars clustered on round numbers (87.5% multiples of 100, 0% non-integer, distinct-ratio 0.127). Both candidates break this — splice 23.2% non-integer, kernel 28.9%, distinct-ratio roughly doubled — splice via the non-round scale factor, kernel via Qhat quantile interpolation between round grid points. But a threshold-splitting tree is nearly blind to the lattice: snapping candidate earnings to the nearest $100 barely moves the AUC (splice 0.5543 → 0.5446; kernel 0.5497 → 0.5503). Snapping to $1000 makes it worse (0.577), and to $5000 makes it trivial (0.91) by over-coarsening real structure. So the footprint is cosmetic for this metric; chasing it is not a fix path.

All numbers are stored in runs/c2st_forensics_v1.json (per-feature marginal/pair AUCs, distributional summaries, the rounding-repair ablation, decision-region profiles, the A-vs-B and noise-floor values) and cross-checked by tests/test_c2st_forensics.py (no PSID / no populace-fit needed).

Design implications for the next candidate

Kernel is the more promising base (marginals already correct; only the joint is off) but must raise joint resolution. The three levers, in order: (a) more rank bins or a continuous conditional draw instead of within-bin uniform — the uniform placement is a direct persistence leak; (b) drop or shrink Laplace smoothing on well-populated anchor bins — add-one flattens the high-mass diagonal that carries persistence; (c) treat the chain: error compounds per backward step, so the runs-view (0.54 threshold) is the binding constraint, and any fix must be validated on window-3, not just window-2.
Splice needs its scaling constrained to preserve the marginal, specifically the right tail. The [0.2, 5] clip is the culprit; a tail-aware cap (or matching donors on boundary rank rather than boundary level, so a high boundary does not recruit a donor that is then scaled further up) would pull q99 back toward 260k.
Do not spend effort on rounding. Snapping to a lattice does not move this classifier and can hurt; the separation is distributional (splice tail) and dependence-structural (kernel persistence), not cosmetic.
A hybrid is plausible: the kernel's marginal fidelity with a persistence-preserving draw, or splice donors selected in rank space with the kernel's marginal anchoring. The two defects are orthogonal (finding 1), so fixing one need not reintroduce the other.

What's in this PR

scripts/c2st_forensics_lib.py — the harness-exact mirrored classifier and the probes (feature attribution, distributional forensics, rounding-repair ablation, decision-region probe, shared-signal test).
scripts/run_c2st_forensics.py — deterministic driver (seed 0): regenerates both candidates from the merged runners, runs every analysis on both views, writes the artifact.
runs/c2st_forensics_v1.json — a new artifact (does not touch any committed gate artifact).
tests/test_c2st_forensics.py — internal-consistency checks on the stored numbers.

Reproduce (needs the staged PSID family files; the kernel's regime gate needs populace-fit, so the gate venv):

.venv-gate/bin/python scripts/run_c2st_forensics.py

🤖 Generated with Claude Code

Reported-not-gated, no-holdout-contact forensic read of the two best gate-1 candidates (segment splicing #51, rank-transition kernel #53), both landing pairs-view C2ST ~= 0.547 vs the locked <= 0.53. Every reported value compares a candidate's seed-0 panel against the seed-0 TRAIN persons' real windows; the holdout is projected only to prove the mirrored classifier equals the harness C2ST bit-for-bit. gates.yaml and committed runs/ artifacts are untouched. Verdict: the residual is NOT a shared signal. The two candidates are more distinguishable from each other (A-vs-B 0.5815 pairs / 0.6349 runs) than from real (splice 0.5543, kernel 0.5497; noise floor 0.5049), and fail through orthogonal defects: - Splice: a marginal-earnings defect. Earnings level marginals carry the full AUC (earnings_t0 alone 0.5423); the [0.2, 5] scale clip inflates the right tail (q99 286,823 vs 260,000; top-decile q99 368,939). - Kernel: a joint-transition defect with intact marginals. Every single-period marginal ~ chance (0.503); the (t,t+1) joint carries the AUC (0.5465). Log persistence 0.7072 vs 0.7304; the 20-bin + within-bin-uniform + Laplace draw flattens the joint, and the error compounds over the backward chain (runs-view t0+t2 pair the most separable at 0.5794; runs C2ST 0.5959 > pairs). - Round-number break is real (both inject non-integer earnings; real PSID is 0%) but NOT what the tree reads: snapping to $100 barely moves the AUC. Cosmetic, not a fix path. Adds scripts/c2st_forensics_lib.py (harness-exact mirrored classifier + probes), scripts/run_c2st_forensics.py (deterministic seed-0 driver), runs/c2st_forensics_v1.json (new artifact), and tests/test_c2st_forensics.py (internal-consistency checks, no PSID/fit). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

vercel · 2026-07-06T18:07:59Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
social-security-model	Ready	Preview, Comment	Jul 6, 2026 6:08pm

…locks) (#55) The ninth pre-registered gate-1 run. Replaces candidate 6's discretized rank kernel with continuous empirical conditional draws and deepens the conditioning memory: a k-nearest-neighbor conditional rank bootstrap over train transition records, matched on the next two generated-or-real ranks (v1, v2) and the person's continuous anchor rank a. Frozen spec registered before the run in issue #42 (issuecomment-4896132094); implemented literally, no tuning. Mechanics (all constants fixed a priori at registration): - Rank machinery (cells, Qhat_pos/rhat, continuous anchor rank u_A) reused byte-for-byte from candidate 5b; protocol machinery imported from the baseline runner; participation gate is candidate 2's backward regime gate. - Donor pools per seed from train positives: pairs (u_prev, u_next) -- candidate 6's kernel pairs, verified bit-identical (64,603 seed 0) -- triples (u_prev, u_next, u_next2) where the next-later observed period is also positive, and re-entry pairs (candidate 6's, 4,994 seed 0). Records pinned in a stable (person_id, period) order fixing the k-NN tie-break. - Conditional draw: triples when v2 exists else pairs; distance |u_next - v1| + 0.5|u_next2 - v2| (triples) + 0.25|u_A - a|; k=25 nearest; one record drawn with probability proportional to its weight; generated u_prev is that record's u_prev exactly (no binning/smoothing/jitter). Earnings = Qhat_pos of the target cell at u_prev. Zero crossings use the re-entry pool matched on |u_A - a| alone. Verdict: gate_1_pass = False (geometry 2/5, battery 2/5; needs >=4/5 both). The deepened memory is a real improvement over candidate 6 (which failed 0/5 both) but does not clear the conjunction. - Geometry: the only failing metric is the runs-view (window-3) c2st_auc, failing seeds 0/2/4 (0.548/0.546/0.549 vs 0.54) and passing 1/3 (0.539/0.532). The pairs-view c2st passes on all five seeds (0.516-0.529 <= 0.53); every other geometry threshold passes everywhere. - Battery: the only failing metric is the 10-year autocorrelation, failing seeds 2/3/4 (0.445/0.442/0.465; deviations 0.094/0.097/0.074 vs tolerance 0.07) and passing 0/1 (0.470/0.473). This is the registered risk: two-step-plus-anchor memory does not carry the full permanent share on 3/5 seeds. Every other battery tolerance passes on all five seeds. Determinism: battery_reference reproduces bit-exact before scoring; the live seed-0 reproduction test passes in the gate venv; the run reproduces from the seeds alone. Reported-not-gated diagnostics per the registration (neighbor distances, triple/pair share, donor reuse, drawn corner mass by anchor quintile, clamped share) are carried in the artifact. Deliverables on this branch: scripts/run_gate1_candidate7.py, runs/gate1_rank_knn_v1.json (schema gate1_rank_knn.v1, spec_registration = the candidate-7 comment), tests/test_gate1_qrf_candidate7.py. Full pytest green (249 passed, 7 skipped); ruff clean; black -l 79 clean. Refs issue #42, forensics #54, and the eight prior run PRs (#40, #43, #44, #48, #50, #51, #52, #53). Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

…pace (#56) * Gate-1 candidate 7: k-NN conditional rank bootstrap (fails 2/5 both blocks) The ninth pre-registered gate-1 run. Replaces candidate 6's discretized rank kernel with continuous empirical conditional draws and deepens the conditioning memory: a k-nearest-neighbor conditional rank bootstrap over train transition records, matched on the next two generated-or-real ranks (v1, v2) and the person's continuous anchor rank a. Frozen spec registered before the run in issue #42 (issuecomment-4896132094); implemented literally, no tuning. Mechanics (all constants fixed a priori at registration): - Rank machinery (cells, Qhat_pos/rhat, continuous anchor rank u_A) reused byte-for-byte from candidate 5b; protocol machinery imported from the baseline runner; participation gate is candidate 2's backward regime gate. - Donor pools per seed from train positives: pairs (u_prev, u_next) -- candidate 6's kernel pairs, verified bit-identical (64,603 seed 0) -- triples (u_prev, u_next, u_next2) where the next-later observed period is also positive, and re-entry pairs (candidate 6's, 4,994 seed 0). Records pinned in a stable (person_id, period) order fixing the k-NN tie-break. - Conditional draw: triples when v2 exists else pairs; distance |u_next - v1| + 0.5|u_next2 - v2| (triples) + 0.25|u_A - a|; k=25 nearest; one record drawn with probability proportional to its weight; generated u_prev is that record's u_prev exactly (no binning/smoothing/jitter). Earnings = Qhat_pos of the target cell at u_prev. Zero crossings use the re-entry pool matched on |u_A - a| alone. Verdict: gate_1_pass = False (geometry 2/5, battery 2/5; needs >=4/5 both). The deepened memory is a real improvement over candidate 6 (which failed 0/5 both) but does not clear the conjunction. - Geometry: the only failing metric is the runs-view (window-3) c2st_auc, failing seeds 0/2/4 (0.548/0.546/0.549 vs 0.54) and passing 1/3 (0.539/0.532). The pairs-view c2st passes on all five seeds (0.516-0.529 <= 0.53); every other geometry threshold passes everywhere. - Battery: the only failing metric is the 10-year autocorrelation, failing seeds 2/3/4 (0.445/0.442/0.465; deviations 0.094/0.097/0.074 vs tolerance 0.07) and passing 0/1 (0.470/0.473). This is the registered risk: two-step-plus-anchor memory does not carry the full permanent share on 3/5 seeds. Every other battery tolerance passes on all five seeds. Determinism: battery_reference reproduces bit-exact before scoring; the live seed-0 reproduction test passes in the gate venv; the run reproduces from the seeds alone. Reported-not-gated diagnostics per the registration (neighbor distances, triple/pair share, donor reuse, drawn corner mass by anchor quintile, clamped share) are carried in the artifact. Deliverables on this branch: scripts/run_gate1_candidate7.py, runs/gate1_rank_knn_v1.json (schema gate1_rank_knn.v1, spec_registration = the candidate-7 comment), tests/test_gate1_qrf_candidate7.py. Full pytest green (249 passed, 7 skipped); ruff clean; black -l 79 clean. Refs issue #42, forensics #54, and the eight prior run PRs (#40, #43, #44, #48, #50, #51, #52, #53). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Downstream relevance of candidate 7: what the classifier residual costs in benefit space REPORTED, NOT GATED. Reads no gate, changes no gate. Translates the candidate-7 population-view classifier residual into the downstream benefit-magnitude space a Social Security analysis consumes, as evidence for whether a FUTURE public gate amendment (which would need its own referee round) is warranted. gates.yaml and every committed gate artifact are untouched. The functional is a STATUTE-SHAPED PROXY, not the full 42 USC 415(b) AIME: the filtered panel's careers are partial (biennial PSID, prime age 25-59, 1998-2022), so a faithful highest-35 AIME cannot be formed. The proxy caps each positive-earnings year at its wage base, indexes to 2022 by the NAWI ratio, averages the top min(10, n_pos) indexed years over (count * 12 * 2), and runs the result through the exact 2022-eligibility 415(a)/415(g) PIA formula. It is monotone in lifetime earnings; the constant scale cancels in every real-vs-candidate comparison. Every artifact and doc says so. Per gate seed 0-4, candidate 7 is regenerated deterministically over the holdout via the merged candidate-7 machinery (filter-first load, locked person-disjoint 0.2 split, train marginals/donor pools/participation gate, backward k-NN chain) -- reproducing the committed candidate-7 artifact's seed-0 diagnostics to float precision, so the histories pushed through the proxy are exactly the ones the gate scored. Both real holdout and candidate (same persons, same rows, only earnings differ; anchor held real) go through the proxy, weighted by each person's anchor-period weight. Gaps are anchored to a real-vs-real noise floor at the same scale: the ctx20 construction (fraction=0.4 seed=1000+s, then fraction=0.5 seed=s) on the seed's TRAIN split. Findings (pooled over 5 seeds; candidate-vs-real / real-vs-real noise): - Overall shape holds: KS 0.025 (within the 0.028 noise floor); mean gap +1.85% and median +1.02% (above the tight sub-0.5% real-vs-real floor but well inside the paper's +/-5% AIME-percentile criterion). - Deciles d2-d9 within +/-5% (mostly under 2%); the bottom decile d1 is +32.6% vs a +4.1% noise floor -- a near-zero-denominator artifact (bottom PIA-proxy ~$290-500/mo, so ~$80-145 absolute gaps read as huge percents), not a wholesale divergence. - Person-level (a bootstrap, not a predictor): weighted MAE $315/mo, RMSE $493/mo; only 26.8% of persons within 5% and 40.8% within 10% of their real PIA-proxy. It matches the distribution, not who-is-where. - Concentration: the residual sits in the zero-anchor group (Q0: mean +9.3%, median +22.9%, KS 0.097, 4.9% within 5%) -- the low/no-attachment workers the k-NN bootstrap conditions weakest and whom the progressive PIA formula weights most. Positive-earner quintiles Q1-Q4 are within a couple percent. Deliverables: - scripts/build_downstream_relevance.py: deterministic; the pinned functional; oracle parameters loaded once with the pe-us revision (bf71be3b) recorded. - runs/downstream_relevance_c7_v1.json: per-seed + pooled measurements, the noise-anchor rows, the pe-us revision, the candidate-7 artifact sha256 reference, reported_not_gated=true and a one-line purpose. - tests/test_downstream_relevance.py: always-runnable internal-consistency (gaps recompute from stored per-side statistics; schema; weighted-stat helpers) + a seed-0 reproduction pin (skipif PSID, importorskip populace.fit) that passes live in the gate venv. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

Amendment OBJECT under the locked pre-registration contract. It PROPOSES changes and changes NOTHING: the locked gate_1.thresholds block is untouched (locked: true stays; every locked value byte-identical), no model reads the new subsection, and nothing takes effect until a fresh adversarial referee round runs on this proposal and the maintainer ratifies by merging. New committed anchor (real-vs-real PIA-proxy floor at deployment scale): - runs/pia_proxy_floor_9822.json + scripts/build_pia_proxy_floor.py: the ctx20 construction (40% draw seed 1000+s, halved at seed s) on the FULL locked filtered panel, both disjoint halves pushed through the pinned PIA-proxy functional imported verbatim from build_downstream_relevance (single source of truth). Records per seed and mean/sd: |mean % gap| 1.86+/-0.76, |median % gap| 1.61+/-0.99, weighted KS 0.0288+/-0.0078, and the Q0 (zero-anchor subgroup) gap (pooled |Q0| 2.66%; per-seed 5.25+/-4.53). The floor is real-vs-real, so the oracle import path does NOT pull populace.fit; the reproduction test needs only PSID and runs live (seed 0, pinned to float precision). gates.yaml gate_1.amendment_proposed (pure addition, zero deletions): - change 1: runs-view c2st_auc_max moves gated -> reported-not-gated (PR #56: candidate-7 KS inside the real-real noise floor, central benefit gaps <=1.9%; the window-3 residual is benefit-immaterial texture per PR #54). - change 2: a NEW gated benefit_space block on the candidate PIA-proxy panels: |mean %|<=5, |median %|<=5, deciles d3-d9 <=5 (d1 AND d2 reported-not-gated -- the new anchor shows their real-real floor clips 5% on 5/5 and 3/5 seeds), weighted KS <= committed anchor mean + 4*sd (machine-checkable derivation = 0.0599), and pooled |Q0 mean %|<=5. The 5% bands cite the paper's pre-stated +/-5%-of-SSA-statistics success criterion (docs Phase-1 acceptance table); only KS is floor-derived. Candidate 7 measured +9.3% on Q0, so the amendment ADDS strictness where the evidence says it bites -- recalibration toward decision-relevance, not relaxation. Discriminator check on committed evidence: real-vs-real PASSES the block, candidate 7 FAILS it on Q0. tests/test_gates_derivations.py: binds the proposed block -- the KS band equals anchor mean + stated k*sd, every proposed gated metric carries a derivation or an a-priori-source citation, the anchor's stats recompute from its stored per-seed values, and the reported-not-gated d1/d2 partition matches the anchor's per-decile fragility. ruff + black -l 79 clean; full pytest green in the repo .venv (275 passed, 9 skipped -- the skips are pre-existing populace.fit gate reproductions). Evidence chain: PR #54, PR #55, PR #56, issue #42. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Gate-1 amendment proposal: decision-relevant recalibration Amendment OBJECT under the locked pre-registration contract. It PROPOSES changes and changes NOTHING: the locked gate_1.thresholds block is untouched (locked: true stays; every locked value byte-identical), no model reads the new subsection, and nothing takes effect until a fresh adversarial referee round runs on this proposal and the maintainer ratifies by merging. New committed anchor (real-vs-real PIA-proxy floor at deployment scale): - runs/pia_proxy_floor_9822.json + scripts/build_pia_proxy_floor.py: the ctx20 construction (40% draw seed 1000+s, halved at seed s) on the FULL locked filtered panel, both disjoint halves pushed through the pinned PIA-proxy functional imported verbatim from build_downstream_relevance (single source of truth). Records per seed and mean/sd: |mean % gap| 1.86+/-0.76, |median % gap| 1.61+/-0.99, weighted KS 0.0288+/-0.0078, and the Q0 (zero-anchor subgroup) gap (pooled |Q0| 2.66%; per-seed 5.25+/-4.53). The floor is real-vs-real, so the oracle import path does NOT pull populace.fit; the reproduction test needs only PSID and runs live (seed 0, pinned to float precision). gates.yaml gate_1.amendment_proposed (pure addition, zero deletions): - change 1: runs-view c2st_auc_max moves gated -> reported-not-gated (PR #56: candidate-7 KS inside the real-real noise floor, central benefit gaps <=1.9%; the window-3 residual is benefit-immaterial texture per PR #54). - change 2: a NEW gated benefit_space block on the candidate PIA-proxy panels: |mean %|<=5, |median %|<=5, deciles d3-d9 <=5 (d1 AND d2 reported-not-gated -- the new anchor shows their real-real floor clips 5% on 5/5 and 3/5 seeds), weighted KS <= committed anchor mean + 4*sd (machine-checkable derivation = 0.0599), and pooled |Q0 mean %|<=5. The 5% bands cite the paper's pre-stated +/-5%-of-SSA-statistics success criterion (docs Phase-1 acceptance table); only KS is floor-derived. Candidate 7 measured +9.3% on Q0, so the amendment ADDS strictness where the evidence says it bites -- recalibration toward decision-relevance, not relaxation. Discriminator check on committed evidence: real-vs-real PASSES the block, candidate 7 FAILS it on Q0. tests/test_gates_derivations.py: binds the proposed block -- the KS band equals anchor mean + stated k*sd, every proposed gated metric carries a derivation or an a-priori-source citation, the anchor's stats recompute from its stored per-seed values, and the reported-not-gated d1/d2 partition matches the anchor's per-decile fragility. ruff + black -l 79 clean; full pytest green in the repo .venv (275 passed, 9 skipped -- the skips are pre-existing populace.fit gate reproductions). Evidence chain: PR #54, PR #55, PR #56, issue #42. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Apply the amendment referee round: honest sourcing and full disclosure The referee (PR #57 review) found two text-level defects, no numeric ones. First, the +/-5% bands claimed the paper as their source; the paper carries no such criterion. The a_priori_source now states the exact provenance — the DRAFT Phase-1 acceptance tables, in AIME units — notes that PIA concavity makes a 5% PIA-space band stricter than the draft figure (roughly 7-9% AIME-equivalent), and declares the bands as this amendment's own proposal-time criterion. The source key renames to amendment_5pct_criterion accordingly. Second, the runs-view c2st demotion silently changed the geometry-block verdicts of four committed runs; a demotion_disclosure block now lists every flip, shows that no overall gate-1 verdict changes (each run still fails the battery and, under the amendment, the benefit-space block — referee-verified baseline pooled Q0 +19.0), and acknowledges where the anti-persistence duty concentrates. Also aligns the anchor reproduction test's pe-us pin check with the repo precedent (PR #41): skip with both revisions named on a mismatched checkout instead of failing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

… regime (fails under the amended gate) (#62) The eleventh pre-registered gate-1 run and the first candidate scored under the amended gate (PR #57/#59). Candidate 7's machinery verbatim with two registered changes; the only calibration is the registered train-side SMM for lambda. Verdict under the amended gate: FAIL. Geometry 0/5, battery 2/5, pooled Q0 -17.89% (> 5). Two registered changes: 1. SMM-calibrated donor-coordinate blend. The k-NN third distance term becomes |lambda*u_w(donor) + (1-lambda)*u_A(donor) - u_A(target)| at the 0.25 weight; u_w is candidate 8's shrunk permanent rank, u_A the anchor ranks. lambda is calibrated per seed on the train split by SMM in the 5b tradition (grid {0,...,1.0}, autocorrelation-ladder SSE on the first 2,000 train persons, ties to the smaller lambda). lambda=0 reproduces candidate 7; lambda=1 candidate 8. 2. Zero-anchor participation regime. Zero-anchor holdout persons draw participation from a gate refit only on zero-anchor train pairs and their re-entry innovations from a zero-anchor-restricted re-entry pool. Positive-anchor persons keep the shared gate and full pools exactly as candidate 7. Candidate 9 does NOT adopt candidate 8's attachment distance. Findings: - Chosen lambda per seed: {0: 0.2, 1: 0.0, 2: 0.3, 3: 0.1, 4: 0.0}. The train SMM never chose lambda above 0.3 and chose lambda=0 (candidate 7) on two seeds. - The c9 pooled 10-year autocorrelation rung (0.514) lands inside the reference band [0.469, 0.609] and between the c7/c8 bracket (0.459/0.670) -- the blend achieved its aim on the 10-year rung in the pooled mean. But no single lambda on the grid lands all three rungs simultaneously: the registered risk materialized (the blend changes all three rungs together). Battery passes only 2/5 (seeds 2, 3); seed 0 fails the 2-year rung (dev 0.059), seeds 1/4 (lambda=0) leave the 10-year rung short. - Geometry 0/5: the binding constraint is the pairs-view c2st_auc (0.531-0.550, all just over 0.53) on all five seeds; benefit-space additionally fails on seeds 0/2/3. - The zero-anchor participation regime closed the never-worker resurrection (generated all-zero share 0.3155 vs real 0.3147, gap +0.0007, vs PR #61's +20.7pp for the shared gate) -- the participation law now matches reality. But the level over-corrected: pooled Q0 PIA-proxy moved from c7 +9.3% / c8 +12.2% to c9 -17.89% (the restricted re-entry pool plus the resurrection fix together subtract more than the +9.3% they were meant to remove). Artifact runs/gate1_rank_knn_v3.json (schema gate1_rank_knn.v3): per-seed chosen lambda + SMM ladders, the amended-gate scorecard (the benefit_space block per seed + pooled Q0), Q0 participation diagnostics, and the standard diagnostics. The battery-reference bit-exact precheck reproduced every committed value before scoring. Tests: seed-0 reproduction (live in .venv-gate) + the amended-verdict recomputation block (24 pass in .venv-gate; 311 pass / 16 skip in the repo .venv). Links issue #42; base machinery candidate 7 (#55); u_w candidate 8 (#58); benefit-space functional #56; C2ST forensics #54; amended gate #57/#59; Q0 forensics #61. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

…ted) (#66) Two reported-not-gated diagnostics on the run-12 near-miss (candidate 10, PR #64): pairs-view C2ST clipped 0.53 on gate seeds 0 and 1 (0.5315, 0.5330; mean 0.5279; 3/5 geometry, battery and benefit blocks passed). Neither diagnostic runs the gate or touches gates.yaml or any committed runs/ artifact. Verdict: NOISE. The clipping is sampling noise around a sub-threshold mean, not a reproducible failure and not split-dependent signal. Diagnostic 1 (scripts/c10_seed_extension.py) measures the same committed candidate on protocol-identical splits at seeds 5-19, scoring only the pairs view. 20-seed candidate c2st mean 0.5234 (sd 0.0059), 5.0 SE below the 0.53 line; only the two original seeds clip; P(>=2 of 5 clip) ~= 0.13, so ~87% of protocol-identical 5-seed gates would pass the pairs-C2ST bar. The candidate sits +0.012 (6.8 SE) over the real-vs-real ctx20 floor -- a genuine but sub-threshold residual. The floor is recomputed under the gate venv (.venv-gate, scikit-learn 1.8.0) so it shares the candidate's classifier version; the committed floor was built under .venv (scikit-learn 1.9.0) and the C2ST is version-sensitive per-seed (mean stable ~0.5109). The cross-venv derivation-vs-scoring mismatch is filed separately. Diagnostic 2 (scripts/c10_forensics.py) mirrors the PR #54 forensics (c2st_forensics_lib reused verbatim; candidate-vs-TRAIN, no holdout contact) at failing seeds 0,1 and passing seeds 2,4. The same seed-invariant earnings-texture residual drives the C2ST in every seed (generated earnings ~2.7x too many distinct values, ~29% non-integer, mild persistence excess; earnings marginals carry it, age at chance). No coherent signal separates failing from passing seeds: the ~0.007 failing-vs-passing gap is on the same signal, and the only large deltas are seed-0 top-decile descriptives that reverse in seed 1. No candidate-11 lead. Decision implied: feeds the mean-based classifier gating amendment as the noise measurement (NOT proposed or drafted here). Deliverables: the two scripts, runs/c10_diagnostics_v1.json (both blocks + a synthesized verdict; reported_not_gated), and tests/test_c10_diagnostics (17 consistency tests + a seed-5 reproduction pin). ruff + black -l 79 clean; full pytest green in the repo venv (the seed-5 pin runs live under the gate venv). Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

vercel Bot deployed to Preview July 6, 2026 18:08 View deployment

MaxGhenis marked this pull request as ready for review July 6, 2026 18:11

MaxGhenis merged commit c203999 into master Jul 6, 2026
7 checks passed

This was referenced Jul 6, 2026

Candidate 2 design: latent-permanent conditioned chained QRF (research memo) #42

Open

Gate-1 candidate 7: k-NN conditional rank bootstrap #55

Merged

Downstream relevance: what the classifier residual costs in benefit space #56

Merged

MaxGhenis mentioned this pull request Jul 6, 2026

Gate-1 amendment proposal: decision-relevant recalibration #57

Merged

MaxGhenis mentioned this pull request Jul 6, 2026

Gate-1 candidate 8: permanent-rank donor matching #58

Merged

MaxGhenis mentioned this pull request Jul 7, 2026

Gate-1 candidate 9: calibrated memory and a zero-anchor participation regime #62

Merged

MaxGhenis mentioned this pull request Jul 7, 2026

Candidate-10 diagnostics: seed noise vs microtexture #66

Merged

Navigation Menu

Uh oh!

C2ST forensics: what the classifier reads at 0.547 - #54

C2ST forensics: what the classifier reads at 0.547#54
MaxGhenis merged 1 commit into
masterfrom
c2st-forensics

MaxGhenis commented Jul 6, 2026

Uh oh!

vercel Bot commented Jul 6, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

MaxGhenis commented Jul 6, 2026

Ranked findings (pairs view, candidate vs train, seed 0)

Design implications for the next candidate

What's in this PR

Uh oh!

vercel Bot commented Jul 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

vercel Bot commented Jul 6, 2026 •

edited

Loading