CG tail transfer: field-local donor quarantine + mass-conserving own-tail stratum + E01000 receipt (#567) - #568
Conversation
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Confirm round 1 (HOLD, two blockers on the evidence commit): - The persisted weight vector had no record identity: positional floats cannot be reattached to households once the run is gone, because the failed run minted no H5 and diagnostics record only n_records. The writer now persists the ORDERED household ids beside the weights (second npy), hashes the ordering, and binds both to the run's target-frame identity in the metadata. - The write was unconditional at export_frame construction, so green runs wrote evidence files that no manifest accounts for. The call now lives inside the terminal_gate_failures branch only — the no-H5 failure class this evidence exists for; late gates (reform smoke, take-up contract) raise after the H5 write, so their failures retain weights in the written dataset itself. Enforced structurally with the #443 AST-guard pattern: the sole main() call site must sit inside the terminal_gate_failures branch. - Review test-quality finding folded: the SSI clone-2 test now mints a real weight-split clone — a new household carrying part of the source household's weight with total mass asserted conserved, duplicating the PUF-channel row the tail stage actually clones — instead of relabeling a full-weight row. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Round 2 pushed for both blockers: (1) the evidence writer now persists the ordered household ids beside the weight vector (second npy) with an ordering hash, and binds the pair to the run's target-frame identity — record-level reattachment is possible from the evidence alone; (2) the write moved inside the |
Confirm round 2 (HOLD, one blocker + two mediums): - A green retry reusing --out/--release-id inherited the prior failed attempt's evidence files and certified an H5 whose manifest knows nothing about them. The green continuation now unlinks all three evidence files immediately after the batched gates pass, before the certified dataset write — evidence can only ever describe the run that is failing, never coexist with a certified release. - The e2e identity assertion pinned only a nonempty base hash and the period; a truncated identity would have passed. It now rebuilds the expected identity from the captured materializer context (the established cache_context pattern) and asserts full equality. - AST guard v2: the call must sit in the terminal_gate_failures IF BODY (an else-branch call would run on green), strictly before the branch's raise, and the green continuation must unlink stale evidence before write_dataset — ordering enforced structurally, matching how production is laid out. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Round 3 pushed: (1) blocker — the green continuation now unlinks all three evidence files immediately after the batched gates pass and before the certified dataset write, so a green retry reusing |
…lenames Confirm round 3 (HOLD, certified-first blocker + guard advisory): - The inverted sequence — certified green run, then a failed retry into the same --out/--release-id — wrote failed-attempt evidence beside the prior run's manifest and H5 (mkdir exist_ok retained them; the failure path never reaches the green cleanup). The run now refuses at setup when release_dir already carries release_manifest.json: certified ids are immutable, reruns mint a new id (every launcher stamps a fresh UTC timestamp) or the operator removes the directory deliberately. The failed-then-green cleanup path stands as adjudicated. - The guard's cleanup check accepted any unlink anywhere after the branch; it now requires a For loop whose iterable names ALL THREE evidence filename constants with an unlink in its body, before write_dataset — decoy unlinks and dropped tuple entries fail. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Round 4: the certified-first sequence is closed by refusal per your prescription — |
…coys Confirm round 4 (HOLD, placement completeness + guard advisory): - Refusal ran after checkpoint_root/artifact_root creation, so a hostile --checkpoint-root beneath releases/<id> mutated a certified directory before the raise, and the base download preceded the check entirely. Refusal now runs twice: once immediately when --release-id is given (before the base download — every launcher passes an id; the auto-generated id derives from the base digest, so that path's refusal is necessarily post-download) and once unconditionally the moment release_dir is computed, before any output mkdir. An AST ordering test pins both placements — the helper-only test cannot prove placement, per review. - Guard v4: the cleanup iterable must BE a tuple naming all three evidence filename constants (subscript slices fail), and the unlink must be called on the loop target — in-loop decoys fail. The Low TOCTOU advisory (concurrent same-id attempts racing exists()) is an accepted residual per review. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Round 5: refusal now precedes every side effect — an early call the moment |
* Adult-care imputation refuses implausible donor knots Base-P3 attempt 2 failed the adult-care signal gate with 12 pre_subsidy_care_expenses values above the $250,000 plausibility ceiling. Checkpoint bisection traced the cause to two measured ASEC childcare values ($730,000 and $360,000) that have sat latent in every build's donor pool — their provenance is measured ASEC originals (their mortgage values are trivial, ruling out the #568 quarantine), and whether any recipient's quantile position reached those top knots was grid luck. Build P3's tail clones shifted the recipient grid and 12 draws landed in the $360k-$730k interpolation region; base-P2 passed the same gate over the same pool by geometry alone. The ceiling's charter is refusing corrupted magnitudes, and a corrupt magnitude is corrupt at the source: donors above the ceiling are now refused as interpolation knots, with a logged receipt (count, values, weight). The measured frame values stay untouched per the raw-only doctrine — they are only barred from becoming other units' imputed expenses. Draws are donor-value-bounded, so outputs can no longer exceed the ceiling under any grid. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Round 2: binding e2e, corrected narrative, receipt completeness, truthful docs Confirm round 1 (HOLD — coverage and audit documentation): - The poisoned-donor end-to-end binds the production call (mutation- checked: fails against the unscreened code): a $600k measured donor flows through with_us_adult_care_inputs; outputs stay bounded by the CLEAN donor maximum, the measured childcare column keeps its poisoned value untouched, incidence survives (usage computed pre-screen), and the receipt line is asserted. clean_mask2 asserted in the unit test. - The narrative no longer claims all 12 draws landed between $360k and $730k — three were between the ceiling and $360k; the corrected wording is 'entered the >$250k top-tail interpolation region'. - The receipt carries excluded_weight_share and retained_maximum, and states plainly that it is a deterministic build-log record, not a persisted artifact. - The ceiling comment no longer claims $250k exceeds every measured donor value (it does not: $730k/$360k are measured), and the module docstring says the draw preserves the SCREENED level distribution. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Fold round-2 advisories: complete log receipt; exact narrative; incidence-bound e2e The build-log line now carries excluded_weight_share and retained_maximum; the unit-test docstring states the 3/9 split; the e2e asserts exactly two expense carriers, so a screening-before-usage mutation (usage 2/3 -> 1/3, one selected unit) fails the count. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…name #568 added structural tests that locate the build body by AST, selecting the function literally named main. This branch made main a thin entry point and moved the body to _main, so those tests walked the four-line wrapper, found none of the calls they check, and failed. Neither change is wrong alone and the two merge cleanly, since they touch different lines. Only the combination breaks, which is why it surfaced on CI rather than in either branch. All three by-name lookups now select _main: the certified-release-dir refusal ordering, the final-household-weight evidence placement, and its bound cleanup loop. Searched for every by-name reference rather than only the two CI named, so the third could not fail on the next run. Behavioral callers of main() are unaffected -- the wrapper delegates. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Phase 2 of the #567 charter, implementing the design adjudicated from the phase-1 diagnostic (receipts on #567): the dense netCG gap is a provable infeasibility — the QRF transfer truncates the donor CG tail at the ASEC topcode (frame max $1,999,998 vs donor max $22.1B), leaving design-weight CG × the hard 5× cap = $1.206T < the $1.271T target.
Four commits, in review order:
synthetic_tail_support_eligible(parsed since the disaggregation landed; first production consumer). Joint donor vectors — signed ST/LT together with linked Schedule-D quantities and lineage — never independently scaled legs (base-m collapses partnership_self_employment_net_earnings to signed near-cancellation (+47.7B/−48.1B legs) #432). Stage manifest with donor ids, boundary, masses, sha.Verification: full
packages/populace-build/tests/shard green (verified independently by the dispatching session after rebase onto current main — the lane's stale-main mirror commit dropped cleanly as byte-identical to #565). +4,460/−88 across 26 files including two new test files (588-line tail-transfer suite, 339-line reconciliation suite).Anti-charter honored: no loss shaping, no multipliers, no gate/envelope changes, no ratio changes, no re-disaggregation.
Exit criterion (build-level, next after merge): base-P3 rebuild → dense attempt where the 5× row ceiling exceeds the target with headroom and netCG passes the existing 0.25 gate on truthful inputs.
Fixes the netCG half of the dense-P3 publication path (#566 take-up design is the companion).
🤖 Generated with Claude Code