Dense arm fails input-mass parity on miscellaneous_income (−58%); Build M dense needs a target-side remedy

[MaxGhenis]

Summary

The Build J-vintage dense arm (337,704 households, full pool, no-L0) fails exactly one hard gate: input mass parity on miscellaneous_income at −58.0% (export frame $19.92B vs reference $47.40B; band ±50%, $1B floor). All 34 other export columns pass; the release tooling correctly refused to emit release_manifest.json/H5. Build M's dense re-run must carry a remedy — this issue is the runbook entry.

Evidence (staging run 2026-07-10, release id populace-us-2024-buildj-dense-warmstart-c10f7bb-20260710T231332Z)

• Calibration quality is NOT the problem: final_loss 0.039657 (beats Build H dense 0.04018), within-10% 87.03%, ESS 65,959, coverage gate 70/70 PASS including the restored SCF asset and SNAP work-requirement columns.
• The failure is a robust property of the optimum, not a convergence artifact: cold direct solve −54.1% → 1500 warm-start epochs improve total loss but push misc further out to −58.0%.
• Dense-vs-sparse inversion vs Build H: Build J sparse holds misc +11.46% (in band) while dense fails; Build H was the reverse. The Build-J-specific driver is the newly restored input columns adding competing structure the full-pool solve satisfies at misc's expense — misc's SOI Table 1.4 target ("other income less loss", ~$52.84B aged) is sign-unstable and low-weight in the 5,534-spec surface.

Non-remedies (ruled out)

• Reviewed exclusion: not eligible — the reference mass ($47.4B) sits near the SOI target ($52.84B), so the band is achievable in principle.
• --allow-input-mass-drift: gate bypasses are exactly what the campaign exists to eliminate.

Candidate remedies for Build M dense

1. Raise the loss weight on the miscellaneous_income target family so the full-pool solve cannot trade it away (calibration-config change; measure the cost to other targets).
2. Band-aware penalty: add the export-mass band itself as a soft constraint for sign-unstable SOI net concepts.
3. If neither holds the band, split the net target into gross/loss legs so the solver stops exploiting sign cancellation.

Full run report (stage timings, RSS profile, shas) is preserved in the staging release dir alongside input_mass_parity.json.

[MaxGhenis]

Final determination — input-construction defect; weight path formally closed

The full remedy experiment bracket + ceiling math + source trace (Build J dense checkpoints, 7 cold-1500 solves, seed 0, ratio 5, λ=0):

Weight multipliers cannot fix this. Income-leg multipliers plateau at −52% through 20× (the income leg is already on-target at $66.12B — pushing it only over-shoots a real SOI target). Loss-leg multipliers are inert through 10× (the loss legs saturate the loss cap at rel_err +234%, so the solver abandons them; misc-loss carriers are pinned up by collateral targets — partnership_income, home_mortgage_interest). No candidate reaches the $23.70B band floor; no candidate causes whack-a-mole.

Support-bound ceiling math (decisive): at design weights the pool's misc is net −$8.15B (1.87M hh × +$21.32B vs 1.00M hh × −$29.47B). Even the most aggressive de-biasing the ratio-5 cap permits — zeroing every net-negative household — leaves $21.32B, still below the floor. The band is unreachable from this input distribution. This refutes this issue's original premise that the band is "achievable in principle."

Root cause: policyengine-us-data/policyengine_us_data/datasets/puf/puf.py:374 — puf["miscellaneous_income"] = puf.E01200. E01200 is Form 4797 "Other gains (or losses)" (business-property sales, loss-heavy), not the SOI Table 1.4 line-21 "other income (less loss)" concept the variable and target reference. Raw E01200: 1.26M loss returns / $31.8B loss (vs SOI 319k / $13.3B). The tax-detail QRF then amplifies the skew (net +$16.6B raw → −$8.15B on the pool at design weights). Also uprated as the wrong concept via uprate_puf.py:75. Blast radius of a fix is narrow (E01200 feeds only this column) but second-order: 40 co-imputed QRF outputs need parity re-verification after any rebuild.

Build M disposition

1. Dense arm ships with a documented reviewed-exclusion for miscellaneous_income in the export-mass register, citing this determination (source concept mismatch; band empirically and mathematically unreachable). Under the gate doctrine this is now a legitimate exclusion — the evidence standard is met, unlike when this issue was filed.
2. No loss-weight multiplier in the Build M dense invocation — formally closed.
3. The input fix is post-launch, first-class: remap miscellaneous_income to a PUF composition matching line-21 (or zero + documented-absent if no clean proxy exists — E01200 belongs with capital/business-gains-adjacent concepts, cf. non_sch_d_capital_gains = E01100), rebuild the processed PUF, rebuild the base, re-verify all 35 parity columns + the 40 co-imputed outputs. Requires PUF codebook diligence — a wrong remap is worse than the documented exclusion. Scheduled for architecture week. The ACS spine inherits the same defect via QRF transfer; the v2 ACS re-materialization picks up the fix.
4. The sparse certified arm is unaffected (selection holds misc in-band: +11.46% at Build J).

Full experiment artifacts under _buildj-runtime/experiments/ and out/buildj-run/misc_experiments/.

[MaxGhenis]

Remap research complete (memo: _buildl-runtime/e01200_remap_memo.md)

E01200 = Form 4797 / 1040 line 14 ("Sales of Property Other Than Capital Assets"), confirmed three ways: the SOI TY2015 Explanation of Terms, the PSL Tax-Calculator codebook, and numerically — raw PUF E01200 reproduces SOI Table 1.4's line-14 legs within 2–4% on every leg (net-gain 1,087k/$33.42B vs 1,063,576/$33.04B; net-loss 1,179k/$21.95B vs 1,153,117/$21.09B).

The fix:

1. other_net_gain ← E01200 — the correct destination already exists in the model (TaxUnit, "Other net gain/loss from Form 4797"), currently unmapped/zero in the PUF pipeline. Exact concept, correct entity.
2. miscellaneous_income → zero + documented-absent: line-21 "other income" has NO PUF field and no clean composition (all 222 PUF columns enumerated; income E-codes jump line 14 → line 15). The complete fix is an imputation against the SOI line-21 target — never a re-point to another E-code. TY2015 published legs: +$46.769B / 6,121,770 returns; −$6.694B / 332,708 returns.

Blast radius (all mapped): miscellaneous_income is in gross-income sources (AGI), so the loss leg currently reduces AGI unlimited; routed to other_net_gain it becomes §461(l)-limited via loss_ald — a correctness gain. market_income must gain other_net_gain as a component (else ~$11.9B drops). other_net_gain joins the QRF imputed-variable lists → re-verify the ~40 co-imputed outputs' parity. The unmerged branch add-misc-income-soi-targets (468bb6a2) wires replication to puf.E01200 > 0 and perpetuates the defect — re-source it with this fix. non_sch_d_capital_gains = E01100 verified correct; leave it.

One discrepancy to resolve before setting the loss-leg target: direct SOI TY2015 read gives the line-21 loss leg as $6.694B / 332,708 returns, while this issue's registry-derived figure was $13.28B / 319k. Counts agree within ~4%; the amount differs ~2× — check the ledger fact's vintage/composition (possible NOL-memo-column fold-in) before calibrating to it.

Uprating cross-check: the issue's "raw E01200 1.26M / $31.8B" reconciles exactly from raw 2015 via the weight (×1.068) and AGI-growth (×1.449) paths — the aging pipeline is not the problem.