[PHYSICS] zero-host pure-completion fallback (#29) + selection z-caps (#30) + deep-incompleteness validation by JasperSeehofer · Pull Request #32 · JasperSeehofer/MasterThesisCode

JasperSeehofer · 2026-07-10T19:05:32Z

Stack position

Part 3 of 3 (decision D2, merge order pending): stacked on PR #31 (perf/eval-vectorization); GitHub will retarget to main as the stack merges beneath it.

What

ed46390 test: pin the legacy zero-host silent-skip behavior (regression baseline, pin-first convention)
8db6c6e [PHYSICS] [PHYSICS BUG][CRITICAL] Zero-host events silently dropped from the joint likelihood — should contribute pure-completion p_i = B_num/D (58% of deep-campaign events) #29: zero-host events contribute the pure-completion likelihood p_i = B_num/D — the exact L_cat → 0 limit of the Gray mixture (Gray et al. 2020 Eqs. 29+32; G2a limiting case 2) — replacing the silent skip that dropped 58% of depth-1.5 campaign events and railed seed1000 low. WARNING promotion + per-h host-lookup yield metric; catalog_only keeps the legacy skip; hosts-present events bit-unchanged.
f29a5e7 [PHYSICS] [CAMPAIGN BLOCKER] seed1000 posterior rails to the lower grid edge (MAP h=0.60, both channels) — effective host catalogue ends at z≈0.3 under the EMRI mass prune #30 groundwork: cap the selection integrals at max_redshift (no-op in production config; binds in the synthetic fixture).
fa50ad5..e0c429e L-A validation (GSD quick task 260710-sjm, verified): z_support catalogue-support-truncated mode in the independent P–P harness (validation/pp_coverage.py), pin-first + bit-identical None path + anchor re-run identical on all shared keys.

⚠ Validation verdict (read before campaign decisions)

The 8-cell synthetic sweep (results/pp_coverage_deepvenue_20260710/SUMMARY.md) shows the pure-completion fallback ensemble is biased HIGH at deep incompleteness in the clean two-branch limit: +0.7…+5.4% in h with cov68 collapsing to ≤0.27 at completion fractions 0.22–0.85 (controls and comp_frac≈0 cells exactly calibrated; bias grows with comp_frac and σ_z).

Consequences:

EXP-40 prediction registered: the post-[PHYSICS BUG][CRITICAL] Zero-host events silently dropped from the joint likelihood — should contribute pure-completion p_i = B_num/D (58% of deep-campaign events) #29 seed1000 re-eval should come off the h=0.60 rail, but the risk flips to interior-but-biased-high.
Issue [CAMPAIGN BLOCKER] seed1000 posterior rails to the lower grid edge (MAP h=0.60, both channels) — effective host catalogue ends at z≈0.3 under the EMRI mass prune #30 / decision D1: explicit z-truncation recovers exact calibration in the harness — quantitative support for the truncation option.
Caveat: production host-found events carry a compensating B_num admixture the harness's clean limit omits; whether the full Gray mixture restores calibration at 60–95% incompleteness is the natural follow-up (harness full-mixture branch).

This PR does NOT relaunch anything; #29/#30 remain open pending deep-venue validation (EXP-40) on cluster return.

Verification

test_zero_host_completion flipped from the old-behavior pin with an independent B_num/D = (1−w_G)·L_comp cross-check; pipeline/kernel goldens unchanged.
Whole-repo fast suite green (829 passed); full slow suite running on this tip.
Live seed600 frozen-venue A/B (this branch vs pre-fallback perf tip, identical inputs) running overnight → real-data fallback footprint (13 restored events expected), results/seed600_ab_20260710/.

🤖 Generated with Claude Code

https://claude.ai/code/session_01QDfQjz3Bv6oKNUQycTSDis

…sion baseline) Asserts the current defect explicitly before the [PHYSICS] fallback commit flips it: an event whose catalogue lookup returns None leaves an empty posterior entry at every h (excluded from the combine) and writes no diagnostics row, while other events are bit-unaffected by its presence. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…B_num/D (#29) Replaces the silent 'if possible_hosts is None: continue' skip (unexamined 2024-03-28 legacy, 19f9b88) with the exact L_cat -> 0 limit of the Gray mixture: p_i = (beta_G*0 + B_num)/D = B_num/D, via empty host lists into p_Di (all downstream guards already existed). On the depth-1.5 campaign the skip silently dropped 58% of events (catalogue-support-conditioned sample) and railed the seed1000 posterior to the low grid edge. - catalog_only mode keeps the legacy skip (no completion term there). - Skip promoted from DEBUG to WARNING + per-h host-lookup yield metric. - Gate: test_zero_host_completion flipped from the old-behavior pin (ed46390), with an independent value cross-check via the reference-run mixture identity B_num/D = (1-w_G)*L_comp; hosts-present events proven undisturbed (bit-equal) + pipeline/kernel goldens unchanged. Refs: Gray et al. (2020) arXiv:1908.06050 Eqs. 29+32; Gray, Messenger & Veitch (2022) arXiv:2111.04629 Eq. 5; G2a limiting case 2. Closes #29 pending deep-venue validation on cluster return. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Threads z_max_cap through the three selection precomputes — D(h), beta_Gbar(h), and the global in-catalogue selection sum — as min(z_max(h), cap), with evaluate() passing cosmological_model.max_redshift and the combine fast path passing HOST_DRAW_Z_MAX (diagnostic-only there). The numerator-side candidate window was ALREADY capped at max_redshift (p_D's min(z_max, redshift_upper_limit)); without this the selection integrals ran to the p_det horizon instead, so any future analysis truncation would break the beta_G = D - beta_Gbar identity and bias w_G. Truncation is now a single safe knob (Model1CrossCheck.max_redshift). Mandel, Farr & Gair (2019), arXiv:1809.02063: the selection function must match the event-inclusion criterion. Production is UNCHANGED at current constants: the real-pool horizon is z_max(h) <= ~1.33 for h in [0.60, 0.86] < max_redshift = 1.5 (no-op cap; gated by test_z_max_cap_above_horizon_is_a_noop bit-identity). The pipeline parity golden IS re-pinned: the synthetic fixture's fake d_L = 4z pool has p_det support beyond z = 1.5, so the cap binds there — shift is the pure event-independent D(h)-domain fingerprint (identical rel shift for every event at fixed h: 1.4%/2.7%/4.5% at h = 0.68/0.73/0.78, both channels). New unit gates: cap no-op above horizon, beta_Gbar = (1-f)*D identity on the capped domain, global-selection galaxy dropping. Part 2 of the 2026-07-10 approved physics change (with 8db6c6e / #29). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…a note - H0_BIAS_RESOLUTION.md: new SS3.20 (zero-host silent drop -> deep-venue rail, mechanism, fix commits 8db6c6e/f29a5e7, validation prediction); corrected the false SS3.2 claim that zero-host events 'contributed only via L_comp'. - DATA_INVENTORY.md: evaluation-log rows for the seed1000 local combine (MAP railed at 0.60, campaign NO-GO), the Re-evaluate-tier pipeline change (#29/#30 -> deep-venue posteriors stale), and the seed600 Omega_m era mismatch provenance note (sim 0.25 vs eval 0.2726; A/B venue only). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…-truncated mode (L-A)

…d-mode change - Add test_z_support_none_golden_pin freezing run_coverage() output at HEAD for a tiny volume-kernel config (n_realizations=2, n_events=25, seed=20260710) - Pin-first per ed46390 -> 8db6c6e convention; proves the z_support=None path stays bit-identical after Task 2's truncated-mode change

…p_coverage harness - PPCoverageConfig.z_support: float | None = None catalogue support ceiling - --z-support CLI flag threaded into the config; completion_fraction added to the CLI print line - _run_realization splits events by z_host < z_support (existing single-host kernel branch, unchanged) vs z_host >= z_support (new pure-completion B_num(h)/D(h) branch, capped at Z_MAX_POP per issue #30, sharing D(h)'s exact unnormalized measure) -- the L_cat -> 0 limit of the Gray et al. (2020) mixture that production commit 8db6c6e (issue #29) installed - run_coverage aggregates completion_fraction per truth - golden pin (z_support=None) stays bit-identical; limiting-case (z_support=Z_MAX_POP), small-z_support (~0.05, finite/normalizable posterior), and monotonic-membership-split tests added

- 8-cell sweep (z_support in {0.2,0.3,0.5,1.0} x sigma_z in {0.015,0.035}, kernel=volume); z_support=1.0 is the untruncated CONTROL at each sigma_z - Anchor bit-identity re-run (sigma_z=0.10, no z_support) + .results-only diff instructions against the sigmaz_scan_20260703 anchor - SUMMARY.md verdict format: per-cell x truth table, control comparison, +/-2*SE (~0.085) coverage-collapse and 2*SEM bias-flag criteria, 3 carried caveats (1D-only, single-host clean limit, hard vs soft truncation) - No sweep executed; orchestrator runs it post-merge (handoff item L-A)

…ode)

…r BIASED HIGH at >20% completion fraction 8 cells (z_support {0.2,0.3,0.5,1.0} x sigma_z {0.015,0.035}, volume kernel, n_real=120): controls calibrated; z_support=0.5 inert (comp_frac~0); at comp_frac 0.22-0.85 coverage collapses (cov68 down to 0.008) with +0.7...+5.4% HIGH H0 bias growing with comp_frac and sigma_z; h_true=0.84 rails HIGH. Anchor re-run: all shared keys byte-identical (completion_fraction key added). EXP-40 prediction registered: post-#29 seed1000 risk flips to biased-high. Ref: RUNBOOK.md + SUMMARY.md verdict; issue #29/#30 validation (L-A).

…(L-A) — summary, verification (passed), state

JasperSeehofer · 2026-07-10T19:07:42Z

Full slow suite on this tip: 851 passed, 6 skipped, 0 failures (2:18, coverage 69%). The overnight seed600 A/B (drift gate + fallback footprint) is the remaining live verification; results land in results/seed600_ab_20260710/.

…mented 713fbd1 D_g fix (+0.057→+0.025); #29 footprint 13 events, mean +0.0003, MAP unchanged run_A @fc45d1f vs 562918e run_live: 1D combined identical to 5 decimals (per-event worst rel 2.6e-8, spline-d_L tolerance). 2D moves 0.787→0.7546 mean on identical inputs = the declared Category-B semi-analytic D_g denominator fix (713fbd1, PR #31) — 57% of the open +0.057 2D residual was the MC denominator defect; remaining venue 2D residual +0.025. run_B @f29a5e7 vs run_A: hosts-present bit-identical both channels; exactly 13x17 empty->filled flips; #30 caps no-op; combine: 13 restored, 2 excluded by zero-floor (net 11); 1D MAP unchanged, mean +0.00032. Yield metric clean. Heavy per-h artifacts (8.5 GB/arm 2D) untracked per convention; combined posteriors + run_metadata force-added (redteam-artifact precedent).

JasperSeehofer · 2026-07-10T21:09:58Z

Overnight seed600 real-data fallback footprint measured (results/seed600_ab_20260710/ANALYSIS.md, commit e520f3b):

Hosts-present events bit-identical to the pre-fallback arm in both channels, all 17 h — the [PHYSICS BUG][CRITICAL] Zero-host events silently dropped from the joint likelihood — should contribute pure-completion p_i = B_num/D (58% of deep-campaign events) #29 "hosts-present undisturbed" claim and the [CAMPAIGN BLOCKER] seed1000 posterior rails to the lower grid edge (MAP h=0.60, both channels) — effective host catalogue ends at z≈0.3 under the EMRI mass prune #30 z-caps no-op both confirmed on the full 3,355-event venue.
Exactly 13 × 17 empty→filled flips per channel: the 13 zero-host events (0.4%) now contribute B_num/D. Per-h yield metric's first real-data run logged cleanly at every h.
Combined footprint: 1D MAP unchanged (0.745), mean +0.00032 (6% of the grid step); 2D MAP unchanged, mean +0.00043. Combine bookkeeping: of the 13 restored, 2 are excluded by the zero-floor (net 11 contributing) — relevant to Paper A's 3,343-events correction.
Consistent with the L-A synthetic sweep: at 0.4% completion fraction the fallback is negligible; bias onsets at ≳20%.

… evening)

…nd per-branch tilt diagnostics to pp_coverage (EXP-41/N-1) - mixture_mode config field: two_branch (default, bit-identical), gray (host events get the full Gray 2020 Eqs. 29+32 mixture (beta_G*L_cat_i + B_num)/D with per-host D_g_i of Eqs. A.9/A.10 — production 713fbd1 analog), conditioned (membership-conditioned inverse N_i/beta_G, B_num/beta_Gbar — N-2b probe) - membership_on_observed flag: route membership on observed z_gal instead of true z_host (N-2d probe) - beta_G/beta_Gbar precomputed on D(h)'s own node grid so the zs>=Z_MAX_POP limiting-case identity is exact - per-branch tilt diagnostics dlogL_dh_host_mean / dlogL_dh_completion_mean in all modes (None sentinel, never NaN) - CLI: --mixture-mode, --membership-on-observed - 6 new tests (TDD RED->GREEN); golden pin + full-dict zmax-match pass unchanged (two_branch posterior line untouched) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QDfQjz3Bv6oKNUQycTSDis

…ontrast — STILL BIASED (worse than clean limit) 12/12 truncated gray cells (zs in {0.2,0.3}) fail both pre-registered criteria: full Gray Eqs. 29+32 mixture AMPLIFIES the deep-incompleteness high bias (worst +0.123 in h at zs=0.2/sigma_z=0.035 vs two-branch +0.032); tilt diagnostics show the B_num admixture flips the host branch from counterweight (-26..-182) to positive tilt (+47..+166). Conditioned (N-2b) does not calibrate either (+0.005..+0.044) => defect is not w_G(h) bookkeeping alone. Controls healthy: zs=1.0 local-ratio |bias|<=0.004; zs=0.5 inert; sigma_z=0.10 anchor .results byte-identical on all pre-existing keys. N-1 fork: production composition SUSPECT at deep incompleteness; depth+fallback NOT demonstrated safe at the estimator level. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QDfQjz3Bv6oKNUQycTSDis

…y, state VERDICT: full Gray mixture does NOT restore calibration at deep incompleteness (worse than the clean two-branch limit); conditioned inverse does not rescue. Handoff N-1 fork adjudicated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QDfQjz3Bv6oKNUQycTSDis

… in pp_coverage - mixture_mode='exact': in-catalogue events use the volume-kernel numerator TRUNCATED at z_support (Mandel-Farr-Gair 2019 arXiv:1809.02063 single conditioning via 1/D; membership G=1[z_true<z_support] is observed data); no beta_G, no D_g_i; zero-host events keep B_num/D (Gray 2020 tiling) - beta_G/beta_Gbar computed only for gray/conditioned; exact takes the sentinel else-branch (golden-pin bit-identity preserved) - --n-z-quad CLI flag threads into config.n_z_quad (small-sigma_z sampling) - tests: ValueError guard, zs=0.95 MAP match (measured bit-identical), deep-truncation completion_fraction exact equality vs two_branch, determinism, CLI flag threading; all pre-existing pins unchanged

… verdict - VERDICT: STILL BIASED vs strict pre-registered criteria (1/12 rows pass), BUT mechanism decomposed: exact truncation removes the ENTIRE sigma_z- dependent leak (bias +0.012..+0.037 -> +0.002..+0.005; cov68 restored to 0.48-0.71; rails 0.45-0.92 -> 0.00-0.19) - N-2c sigma_z ladder: two_branch bias climbs +0.003->+0.037 with sigma_z; exact FLAT; modes converge at sigma_z->0 => leak confirmed as the sigma_z-dependent component; +0.003..+0.005 sigma_z-independent completion-branch floor survives (N-3 target) - N-2d observed-membership: gray worsens up to +0.054 at sigma_z=0.035; exact hard clamp misspecified under obs-z membership (soft form needed) - production-correction candidate flagged for /physics-change: membership- truncated (soft) in-catalogue kernel integrands

Mechanism decomposition: sigma_z-dependent deep-incompleteness bias = membership-support kernel leak (removed by exact truncation, ladder confirms convergence at sigma_z->0); sigma_z-independent completion floor +0.002..+0.005 remains for N-3. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QDfQjz3Bv6oKNUQycTSDis

…step CLI flag - _inference_population_weight(z, tilt): w_pop(z) * exp(tilt*z), strict tilt == 0.0 gate returns the untilted weight object unchanged (bit-identical default path; golden pins pass unmodified) - inference_wpop_tilt threaded through all four INFERENCE call sites (host volume kernel, B_num completion numerator, D(h), beta_G); generative truth draw _sample_detected_redshifts never tilted - --inference-wpop-tilt and --h-step CLI flags threaded into PPCoverageConfig - 5 new tests: gamma=0 bit-identity + golden-pin guard, gamma!=0 inequality, determinism, --h-step round-trip + grid-size, strict map_mean monotonicity in gamma (direction measured: ascending)

…ator RUNBOOK (before runs) - N-3 lever-arm prediction: real, ~linear in gamma; magnitude is the measurement - floor prediction: genuine discriminator, both outcomes pre-stated (artifact => grid/quadrature discretization; persistent => composition residual vs campaign SEM) - 8-run tilt ladder + 3-run floor grid, gamma=0 baselines cited (not re-run)

…arm NEGLIGIBLE, floor PERSISTENT - N-3 lever arm d(map_mean)/dgamma = +0.0001...+0.0017 (real, monotone, ~linear, tiny): D1 headline Dh(gamma_10%=0.127) <= +0.0004 in h (<= +0.05% of truth; exact mode <= +0.015%) — the deep regime is NOT meaningfully population-prior-driven under exp(gamma*z) tilts (ratio structure self-cancels); prior-sensitivity escape hatch for the sigma_z-independent floor CLOSED - floor discriminator: exact gamma=0 floor (+0.0026/+0.0044 at 0.62/0.72) stable to <= 0.0002 under h_step 0.004->0.002->0.001 and n_z_quad 160->320 => PERSISTENT composition residual, not grid/quadrature - composition contrast: two_branch (leak-carrying) ~7x more prior-sensitive than exact at the 0.72 truth; both negligible - 8 tilt JSONs + 3 floor JSONs + logs; RUNBOOK pre-registered at c78c2f5 before any run; prediction (i) magnitude expectation honestly REFUTED

Prior-sensitivity lever arm negligible (<=0.05% h per 10% w_pop misspecification, ratio structure self-cancels); sigma_z-independent floor persistent under grid/quadrature refinement. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QDfQjz3Bv6oKNUQycTSDis

…ag in pp_coverage The harness generative model decides detection on the TRUE z (draws from w_pop*p_det before the noise draws), so the exact conditional keeps p_det(A(z)/h) inside both branch numerators; the MFG no-p_det-inside form (arXiv:1809.02063) applies only to data-thresholded detection. Flag off (default) is bit-identical; probes the persistent sigma_z-independent completion-branch floor from 260711-117/1ps. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QDfQjz3Bv6oKNUQycTSDis

…loor is not the latent-detection factor Deep exact cells unchanged (dbias <= +0.0006); untruncated controls flip -0.003 -> +0.004..+0.006 with degraded cov68, so the formally-exact p_det-inside conditional measures worse than the MFG form here. Sharpened floor candidate: sigma(dL_obs)-vs-sigma(dL_true) noise-model approximation (O(sigma_f^2), matches all floor properties). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QDfQjz3Bv6oKNUQycTSDis

…ledger consolidation Session 2026-07-11: handoff N-1/N-2/N-3 executed (quick tasks 07n/117/ 1ps/27m). Deep-incompleteness bias decomposed: dominant membership-support kernel leak (exact mode removes it), Gray mixture amplifies, prior sensitivity negligible, p_det-inside refuted; small O(sigma_f^2) floor remains with a sharpened noise-model candidate. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QDfQjz3Bv6oKNUQycTSDis

…low, debrief + model discipline Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QDfQjz3Bv6oKNUQycTSDis

…model probe) Add sigma_dl_model_in_likelihood (config + --sigma-model-in-likelihood) to pp_coverage.py: the GW-likelihood factor uses the z-dependent model/true-distance width sigma_f*A(z)/h (carrying its own 1/sigma(z) normalization via _norm_pdf) instead of the constant observed-distance sigma_f*dL_obs. Applies to the host kernel numerator (every mixture_mode) and the completion B_num; the p_det selection integrals (D(h), gray D_g_i) are unchanged (they integrate p_det, not the GW likelihood). Combined with --pdet-in-numerator it is the fully-consistent exact conditional for the latent-thresholded generative model. Probes the sharpened floor candidate from results/pp_coverage_pdetnum_20260711 SUMMARY.md sec.2: the constant observed-distance inference sigma is an O(sigma_f^2) mismatch vs the sigma_f*dL_true generative noise, the scale of the +0.002..+0.005 sigma_z-independent residual floor. Default off is bit-identical (regression guard: exact zs=0.3 sz=0.035 results block byte-identical to the committed exactmode JSON). ruff + ruff-format + mypy clean; 33 pp_coverage tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QDfQjz3Bv6oKNUQycTSDis

…he dominant floor component 2x2 {const-sigma, model-sigma} x {p_det-inside off, on} on exact deep cells (zs 0.2/0.3) + inert controls (zs 0.5/1.0), plus n_events scaling (250/1000/4000) and a fine-grid confirm (h_step 0.001). Pre-registered predictions in RUNBOOK.md (written before the runs). - model-sigma + p_det-inside (the fully-consistent exact conditional for this latent-thresholded model) removes ~85-90% of the +0.002..+0.005 floor: MAP bias -> <= +0.0008 on the deep cells AND nulls the -0.002..-0.004 control offset, with cov68 nominal at campaign-scale n. Neither half alone works (model-sigma alone over-corrects negative; p_det alone was the 27m refutation) — they are the two halves of one conditional. - n-scaling: the const-sigma floor is FLAT in n with cov68 COLLAPSING (h=0.72 0.63->0.38->0.12) => a real asymptotic model bias, not a finite-sample MAP-skew. A tiny second-order residual (~+0.0005, ~15x below campaign sigma_boot) survives the corrected estimator, visible only at n=4000. - fine-grid confirm: coarse(0.004) == fine(0.001) biases to +-0.0001 — not a grid-quantization artifact. Floor decomposition COMPLETE: (i) dominant membership-support kernel leak (260711-117) + (ii) inference-noise-model floor (this task) + (iii) negligible second-order residual. Practically subdominant for Paper B; a required design input to the user-gated soft-f(z)-kernel production correction (do NOT add p_det alone). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QDfQjz3Bv6oKNUQycTSDis

…edger (decomposition COMPLETE) H_sigma CONFIRMED: the sigma_z-independent floor is the inference noise-model approximation (sigma(dL_obs)-vs-sigma(dL_true) width + p_det-inside, the two halves of the latent-threshold exact conditional). Both together remove ~85-90%; const-sigma floor is a real asymptotic bias (flat in n, cov68 collapses), not finite-sample skew; tiny 2nd-order residual ~15x below campaign sigma_boot. Floor decomposition COMPLETE. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QDfQjz3Bv6oKNUQycTSDis

…N-4 shallow-venue probe Add d50_gpc/w_pdet_gpc (config + --d50-gpc/--w-pdet-gpc) to pp_coverage.py, threaded through detection_probability and every call site (population sampler, D(h), beta_G, host/completion p_det factors). Lets the harness model a shallower detection horizon: d50=0.23, w=0.037 -> z_median 0.044 (the seed600 shallow venue, vs the default 1.85 -> z_median 0.28). For N-4(a): test whether the calibrated volume-kernel/no-truncation estimator develops the seed600 +0.0132 offset when the venue is made shallow. Default (1.85/0.30) is bit-identical (regression guard: exact zs=0.3 sz=0.035 results byte-identical to committed exactmode). ruff/format/mypy clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QDfQjz3Bv6oKNUQycTSDis

…stimator-intrinsic (large sigma_z/z at low z) (a) Depth ladder (calibrated volume kernel, no truncation, sigma_z=0.035): calibrated at the commission depth (z_med 0.28, bias -0.002) but develops a strong POSITIVE bias as the venue shallows -> +0.011 at z_med 0.056, +0.030 at z_med 0.044 (seed600 depth), cov68 collapsing. P-B confirmed (P-A refuted). (b) sigma_z sweep at the shallow rung: bias VANISHES at sigma_z<=0.015 (-0.002, calibrated) and appears only at sigma_z=0.035 -> it is a sigma_z/z effect (~0.8 at z_med 0.044): the host-z kernel truncates at z>=0 and the volume/Eddington correction (derived for an untruncated kernel) stops cancelling. (b) jackknife on the on-disk seed600 run_live JSONs (no re-eval): reproduces the raw +0.0132; the residual is broad/systematic (62% of events tilt high, Gini 0.65, trimming the top-|tilt| events GROWS the residual) -> not outlier-driven, matching the per-event depth mechanism. Load-bearing caveat: full attribution to seed600 needs its low-z redshift-error model (sigma_z/z ~ O(1)?); cross-seed systematic-vs-scatter needs the campaign. A z>=0-truncation-aware volume kernel would address BOTH the deep membership leak and this shallow effect (user-gated /physics-change, not this task). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QDfQjz3Bv6oKNUQycTSDis

…edger Shallow-venue 1D +0.0132 EXPLAINED: estimator-intrinsic sigma_z/z-at-low-z truncated-volume-kernel Eddington effect (venue depth ladder + sigma_z sweep; seed600 jackknife shows it is broad/systematic). Full seed600 attribution pending its low-z sigma_z model; cross-seed needs the campaign. A z>=0-truncation-aware volume kernel would address BOTH the deep membership leak and this shallow effect. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QDfQjz3Bv6oKNUQycTSDis

…hallow explained), N-5 optional, production-kernel convergence Both ranked probes done: deep-incompleteness bias fully decomposed (kernel leak + noise-model floor, hx1) and shallow +0.0132 explained (sigma_z/z-at-low-z volume-kernel truncation, iic). Remaining local: N-5 (optional 2D) + the seed600 low-z sigma_z model check that closes N-4 attribution. Both regimes converge on one user-gated production fix (z>=0-truncation-aware volume kernel). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QDfQjz3Bv6oKNUQycTSDis

… are photo-z (σ_z/z~O(1)) Closes the load-bearing caveat from [L8]: seed600's effective redshift uncertainty at z≈0.046 is large-fractional photo-z, so the σ_z/z-at-low-z truncated-volume-kernel Eddington effect applies and the shallow +0.0132 residual is attributed to it (not an unexplained offset). Measured on the reduced GLADE+ catalogue seed600 evaluated (inline, no re-eval), z-shell 0.03-0.06 (z_med 0.046, n=767,552): 89.7% photometric, σ_z median 0.0344, σ_z/z median 0.65 — near-exact match to the harness σ_z=0.035 rung that gave +0.030; spec-z minority (10.3%) at σ_z/z≈0.033 is the calibrated counterweight the jackknife saw. Code trace (airtight): the likelihood host-z kernel width IS the catalogue σ_z (bayesian_statistics.py:2243, host_z_error_eff=sqrt(σ_z²+σ_z_pv²)) and the [PHYSICS] z>=0 clamp is active for low-z photo-z hosts z_g<4·σ_z (:2234-2239); at z_g=0.046, 4σ_z=0.14>z_g. Un-truncated-derived volume/Eddington correction stops cancelling. Docs only (no code/physics change). Cross-seed systematic-vs-scatter still needs the campaign. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QDfQjz3Bv6oKNUQycTSDis

…presentation gate (user-gated) Scoping ONLY — no production code. Assembles the physics-change presentation the hard gate requires so the user can decide whether/how to proceed with the z>=0-truncation-aware / photo-z-marginalized volume host-z kernel that both bias regimes converge on ([L7] deep membership leak + [L8] shallow σ_z/z Eddington). Contents: OLD formula (bayesian_statistics.py:2243-2317 volume_deconv kernel); the identified σ_z/z~O(1) limitation (commission-d2 calibrated at z_med 0.28 → -0.002 vs [L8] shallow z_med 0.044 → +0.030); candidate directions (A truncated-normal- consistent, B photo-z-marginalized soft membership); the [L7] distance-error coupling constraint (do NOT fix the z-kernel alone); references (Gray 2020, MFG 2019, CFH 2018, Mastrogiovanni/ICAROGW 2305.10488, Alfradique/Bom 2023-2026 full-photo-z-PDF practice, Wang-Chen 2408.10382 tolerance); dimensional analysis; six binding regression gates (must reproduce commission-d2 deep calibration AND remove the shallow bias); open user decisions D1/candidate/coupling/validation. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QDfQjz3Bv6oKNUQycTSDis

…n-in-M impact now -0.0022 Optional handoff item N-5 (2D-channel subsample dependence). Re-ran the G7row9 494-event seed600 driver at HEAD (713fbd1 D_g fix present) on the archived shallow venue. Under current code the 494-event 2D subsample is well-behaved: edge_mass 0.216->0.003 (pre-fix railing toward 0.86 GONE), mean 0.790->0.768; it sits +0.0135 above the full-venue 0.7546 = subsample-selection offset, not a code defect. 1D subsample 0.745 reproduces the venue +0.013 residual. No additional 2D subsample/grid pathology; venue-level +0.025 2D residual stays campaign-gated (D4). Caveat: the pre-fix artifact is NOT a clean D_g-only baseline (its 1D 0.730 predates #29/z-clamp changes); the clean D_g attribution stays in the L-B full-venue A/B. Bonus: post-D_g-fix Eddington-in-M impact on the 2D mean is -0.0022 (was -0.020 pre-fix) => the bayesian_statistics.py:2400-2401 comment + quoted value are now stale (flagged, not edited - physics-trigger file). Driver: threaded allow_shallow_pool=True through BOTH evaluate() and combine_posteriors() (both build a SimulationDetectionProbability guarding the campaign-depth pool) so the archived shallow seed600 venue re-runs, same precedent as the seed600 A/B. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QDfQjz3Bv6oKNUQycTSDis

…d, N-5 done, production fix scoped) N-4 shallow attribution CLOSED (seed600 low-z σ_z/z~O(1) photo-z), N-5 2D subsample well-behaved post-D_g-fix, production kernel fix scoped (user-gated /physics-change). Remaining is cluster-gated (EXP-40, campaign) + user decisions D1-D5 + the production /physics-change awaiting approval. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QDfQjz3Bv6oKNUQycTSDis

…nt all) D1 evidence-driven (defer framing to EXP-40/campaign); D2 one combined deployment (#22->#31->#32 + #27 CLU); D3 Paper A on hold until pipeline+results satisfy, then upgrade; D4 defer 2D +0.025 to campaign (no more local bias work); PROD implement all (full truncated-normal x volume-prior + soft photo-z membership + distance-error coupling kernel, new normalization_mode, volume_deconv stays golden). Execution sequence (physics hard gate) recorded. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QDfQjz3Bv6oKNUQycTSDis

…dy to execute) Nails down the code-level plan after reading the production kernels: volume_trunc is SHALLOW-only (z>=0 floor + unified numerator support; no-op on deep venue by construction; deep leak is a separate z_support-edge change). Substantive change = integrate N_g over the per-host galaxy window (not the shared GW window), restructuring the batched hot path; volume_deconv stays byte-identical. Decisive gate is the seed600 494-event A/B (genuine empirical uncertainty whether the numerator-window is the lever). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QDfQjz3Bv6oKNUQycTSDis

Part 1 formula approved via /physics-change gate; execution deferred to a fresh session per user (hot-path batched-kernel restructuring wants full context). Pasteable kickoff with code locations, golden-guard, and the decisive seed600 494-event empirical gate. STATE marked NEXT SESSION START HERE. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QDfQjz3Bv6oKNUQycTSDis

…t (2026-07-12) Advisory pre-Phase-2 convention trace from the orbiter coverage model (read-only on code; nothing submitted). No live divergence found; HOST_DRAW_Z_MAX already fixed 0.5->1.5; all 4 historical divergence classes mutually consistent. - CONVENTIONS-MANIFEST.md: 8-row skeleton of convention-bearing quantities (incompleteness declared) - COVERAGE.md: coverage-map/v1; finding C-MTC-20260712-001 (first missing-coverage finding), C-003 pp_coverage depth (WATCH — under active exploration, not a blocker), C-004 M_z injection invariant test (APPROVED FOR IMPLEMENTATION — a future session should add a CI assertion that the injection catalog "M" column == M_source*(1+z)) - tracer-verdict-2026-07-12.md: per-quantity verdict + refuter pass; ADVISORY, ratified 2026-07-12 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016thJ2geJxGSRFmQ3FRPyyu

…lly FALSIFIED Implements the approved Part 1 shallow-venue correction behind a new, isolated normalization_mode="volume_trunc": the calibrated volume kernel with the in-catalogue NUMERATOR integrated over the per-host galaxy window [z_g-4sigma, z_g+4sigma] (shared with Z_g/D_g) and the lower z-limit floored at 0 instead of 1e-6. Scalar and batched kernels; volume_deconv/local_ratio stay BYTE-IDENTICAL (golden regen additions-only), batch==scalar bit-identical on all volume_trunc cases, full CPU suite 889 passed. The decisive seed600 494-event shallow-venue A/B (scripts/volume_trunc_ab.py) REJECTS it: volume_trunc worsens the shallow bias — 1D mean 0.745 -> 0.800, MAP 0.73 -> 0.80, posterior collapses onto h=0.80 (moved AWAY from truth 0.73 by ~4x the +0.013 residual it was meant to remove). The volume_deconv arm reproduces the established reference exactly (1D 0.745, 2D 0.768), so the driver/data are sound. Mechanism (results/volume_trunc_ab_20260712/): (1) the shared fixed_quad(n=50) is numerically invalid over the wide host window — the sparse Gauss-Legendre nodes alias the narrow GW peak (n=50 -> 0.0 vs exact 0.24-0.65), h-dependently; (2) even the exact host-window numerator tilts high in the shallow regime. Both push H0 high. Verdict: the numerator-window unification is NOT the +0.013 shallow lever and, as specified, is numerically broken. volume_trunc is retained as an EXPERIMENTAL / FALSIFIED mode (not CLI-wired, not for production) so the finding is reproducible; volume_deconv remains the golden default. Tests: volume_trunc pins + sigma_z->0 limiting case + prior-shape h-independence. Ref: Gray 2020 A.10; G2b §1.4; scoping §7b (.planning/PRODUCTION-KERNEL-FIX-SCOPING-20260712.md). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QDfQjz3Bv6oKNUQycTSDis

…(2026-07-12) STATE.md + DECISIONS §PROD-Part-1-OUTCOME + new HANDOFF-VOLUME-TRUNC-FALSIFIED: the approved Part 1 volume_trunc kernel worsens the shallow venue (1D mean 0.745->0.800, posterior collapses to h=0.80); numerator-window is not the +0.013 lever and is numerically broken (fixed_quad n=50 aliases the GW peak over the wide host window). Redirect to Candidate B soft membership + [L7] coupling with a peak-aware quadrature (user-gated). Code/finding committed in c4a1c7d. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QDfQjz3Bv6oKNUQycTSDis

Records the volume_trunc host-z kernel (Part 1) under Unreleased: implemented behind an isolated normalization_mode, volume_deconv byte-identical, but FALSIFIED at the seed600 shallow A/B (worsens the bias) and retained only as an experimental/falsified diagnostic. Not CLI-wired, not for production. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QDfQjz3Bv6oKNUQycTSDis

…2 verified) User insight: the catalogue BH-mass error is ~60% (Reines-Volonteri 0.24 dex intrinsic scatter, linear sigma_M/M floor 0.58), not 10% — so the LINEAR-Gaussian host-mass kernel hits the same untruncated-vs-truncated inconsistency as the low-z photo-z redshift kernel, at the [M_min,M_max]=[1e4,1e7] EMRI bounds. Mass is 2D-only => candidate for the 2D +0.025 residual + the info-monotonicity violation. Two probes (results/mass_kernel_truncation_20260713/): - mass_trunc_probe.py: P(M<0)=4.8% under the linear kernel; G2d effective mass is <1% off in the interior but 15% low near M_min (wrong sign) / 22% high near M_max; 65% of R_eff-weighted EMRI hosts sit in the M_max boundary zone. - mass_kernel_h0_toy.py: controlled 2D mass<->z<->H0 toy, production (linear+G2d) vs correct (lognormal x R_eff truncated). Mass-kernel H0 shift = HIGH, seed-robust: +0.0027+/-0.0002 (sigma_z/z=0.15), +0.0089+/-0.0008 (sigma_z/z=0.30); grows with photo-z leverage; control (correct arm) ~unbiased at near-spec-z. Sign puzzle RESOLVED (marginalisation sees the kernel shape -> HIGH, not the naive LOW). Unified: 1D = redshift effect (+0.013); 2D adds the mass effect => 2D bias > 1D bias. Indicated fix (user-gated /physics-change): lognormal x R_eff truncated mass kernel. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QDfQjz3Bv6oKNUQycTSDis

…rrection Adds a behavior-preserving config.clamp_zgal knob to the pp_coverage harness (default True = bit-identical; 29 harness tests green) and a clamp-isolation diagnostic. At the shallow seed600-matched venue (d50=0.23, z_med~0.044, sigma_z=0.035), volume kernel, multi-seed: clamp ON : map_bias +0.0240 +/- 0.0022, cov68 ~0.61 (the [L8] +0.030) clamp OFF : map_bias -0.0056 +/- 0.0020, cov68 ~0.68 (vanishes, cov recovers) Deep venue is clamp-independent (control). => the shallow high bias is the boundary clamp on the OBSERVED photo-z + a naive Gaussian kernel that does not model the clamp; the volume/Eddington correction is a red herring (present both ways). Refines [L8]: the fix is a censored-measurement likelihood, not a re-derived volume kernel. Production relevance is OPEN: the reduced catalogue redshift is NOT hard-clamped (min -0.0003, 17/500k <=0), so production may not fully share the harness artifact; needs a low-z photo-z pileup/censoring inspection to settle whether the seed600 +0.013 1D residual is this effect. results/h1_zclamp_20260713/FINDINGS.md. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QDfQjz3Bv6oKNUQycTSDis

…+0.030 is largely an artifact Direct check of the reduced catalogue z<0.10 shell (871k gal): no pileup/spike at 0 (n(z==0)=0, min -0.0003, smooth rising histogram), only 3.7% of low-z hosts have the kernel crossing 0. => production does NOT reproduce the harness generative clamp, so the +0.030 shallow bias is substantially a harness artifact. Weakens [L8]'s seed600 +0.013 attribution and suggests the redshift-half production fix is largely unnecessary; mass-channel (H2) bias is independent and stands. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QDfQjz3Bv6oKNUQycTSDis

…egime Wide h-grid (no railing) mass-kernel H0 differential vs photo-z leverage (3 seeds): sigma_z/z=0.15 -> +0.0025, 0.30 -> +0.0081, 0.50 -> +0.0165, 0.75 -> +0.0214. At the real shallow-shell leverage (sigma_z/z~0.5-0.65) the mass-kernel bias is +0.016 to +0.02 HIGH -- a large fraction of the +0.025 2D residual. Combined with the H1 clamp finding (redshift half largely a harness artifact), the MASS kernel is the primary production-relevant driver of the 2D residual; the lognormal x R_eff truncated mass kernel is the load-bearing production fix. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QDfQjz3Bv6oKNUQycTSDis

…ted as 2D bias driver New isolated normalization_mode="mass_trunc": the 2D (with-BH-mass) channel's host-mass prior replaced from the linear-Gaussian G2d moment match (eddington_shifted_host_mass) to the truncated lognormal × R_eff prior on [M_MIN, M_MAX] — Reines & Volonteri (2015) lognormal error × Babak et al. (2017) R_eff population weight, renormalised on the physical EMRI mass window. Numerator: Gauss-Hermite on the narrow GW M_z peak (peak-aware — the fix for the fixed_quad aliasing that falsified volume_trunc). Denominator: Gauss-Legendre in ln M over a per-host peak-aware window (the erf-sum closed form is Gaussian-prior-only). volume_deconv / local_ratio / volume_trunc stay byte-identical (kernel-parity golden regenerated additions-only: 9 new *_mt_* pins, zero existing pins changed; mt_3d == vd_3d exactly; single_host_likelihood_batch bit-identical to scalar; limiting cases in test_mass_trunc_kernel.py). Full CPU suite 940 passed; mypy/ruff clean. Decisive seed600 494-event shallow-venue A/B (scripts/mass_trunc_ab.py): Δ2D mean = +0.0029 (small, WRONG sign), Δ1D = 0 exact → the mass-kernel truncation is NOT the 2D +0.025 residual driver. The isolated single-host toy over-stated it by omitting the selection denominator, which cancels the numerator shift in the full ratio-of-sums pipeline; the linear-Gaussian G2d approximation is thereby empirically validated as adequate for the 2D channel. Retained as an experimental/exonerated mode (not CLI-wired); volume_deconv stays the golden default. Refs: Reines & Volonteri (2015) arXiv:1508.06274 §4.1; Babak et al. (2017) arXiv:1703.09722; Abramowitz & Stegun 25.4.46. Finding: results/mass_trunc_ab_20260713/. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QDfQjz3Bv6oKNUQycTSDis

…ingle venue Both "X is NOT the 2D-bias driver" conclusions rest on the SAME seed600 494-event shallow subsample — no cross-venue confirmation, so a shared venue idiosyncrasy would fool both. Also separate the CLEAN A/B delta (same events both arms) from the cross-venue extrapolation: the A/Bs run on the SUBSAMPLE (2D mean 0.768 / +0.038), not the full venue (0.7546 / +0.025) the "+0.025 residual" refers to. Add a binding rule to the plan of record (§5): negative conclusions are venue-scoped and provisional pending the campaign (D4/§4b), not universal facts. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QDfQjz3Bv6oKNUQycTSDis

JasperSeehofer · 2026-07-16T18:03:18Z

Subsumed by #34 (physics/consolidate-to-main), which consolidated the cumulative GPD-verified physics onto main. All commits from this branch are contained in that merge.

JasperSeehofer and others added 11 commits July 10, 2026 19:22

docs(260710-sjm): pre-dispatch plan for pp_coverage catalogue-support…

fa50ad5

…-truncated mode (L-A)

chore: merge quick task worktree (260710-sjm pp_coverage deep-venue m…

cfce571

…ode)

docs(quick-260710-sjm): pp_coverage catalogue-support-truncated mode …

e0c429e

…(L-A) — summary, verification (passed), state

JasperSeehofer mentioned this pull request Jul 10, 2026

[PHYSICS BUG][CRITICAL] Zero-host events silently dropped from the joint likelihood — should contribute pure-completion p_i = B_num/D (58% of deep-campaign events) #29

Closed

JasperSeehofer and others added 15 commits July 10, 2026 23:12

docs: correct A/B analysis date stamps (all runs completed 2026-07-10…

711c1b7

… evening)

docs: next-session kickoff prompt — floor noise-model probe, N-4 shal…

66ff434

…low, debrief + model discipline Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QDfQjz3Bv6oKNUQycTSDis

JasperSeehofer and others added 24 commits July 11, 2026 12:59

JasperSeehofer mentioned this pull request Jul 16, 2026

[PHYSICS] Consolidate current pipeline to main (GPD-verified sound; cluster closure pending) #34

Merged

JasperSeehofer closed this Jul 16, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[PHYSICS] zero-host pure-completion fallback (#29) + selection z-caps (#30) + deep-incompleteness validation#32

[PHYSICS] zero-host pure-completion fallback (#29) + selection z-caps (#30) + deep-incompleteness validation#32
JasperSeehofer wants to merge 51 commits into
perf/eval-vectorizationfrom
physics/zero-host-completion-fallback

JasperSeehofer commented Jul 10, 2026

Uh oh!

JasperSeehofer commented Jul 10, 2026

Uh oh!

JasperSeehofer commented Jul 10, 2026

Uh oh!

JasperSeehofer commented Jul 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

JasperSeehofer commented Jul 10, 2026

Stack position

What

⚠ Validation verdict (read before campaign decisions)

Verification

Uh oh!

JasperSeehofer commented Jul 10, 2026

Uh oh!

JasperSeehofer commented Jul 10, 2026

Uh oh!

JasperSeehofer commented Jul 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant