You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
QRF donor contains aggregate-scale PUF rows (single-record home_mortgage_interest up to $236B); support_clip donor_realized is vacuous against them #516
Found during #511 forensics; latent hazard, not the realized driver of that row (base-O pool is clean of it — receipts below).
What's in the donor
Measured via puf_tax_unit_donor_from_arrays on the pinned artifact the certified O base consumed (base-o summary puf_h5 → policyengine_us_data/storage/puf_2024.h5, sha 7669f5b5…, 211,677 donor rows):
3,066 rows carry home_mortgage_interest ≥ $10M — up to $235.97B on a single record (three identical $236.0B rows at weight 0.13 each; $177.0B ×2; $22.0B × w1.08; $3.7B × w11). Weighted, these rows hold $2,947B of phantom mass against $418B for the other ~94k carriers. TY2015 E19200 for ALL returns was $304.5B — single-record values of this scale are not tax data; they look like IRS disclosure-aggregate rows (or their clones) whose amounts were never brought back to per-return scale.
The same rows poison the structural columns: donor-wide implied first-mortgage rate = 31.9% of balance (34% for 2005-2010 originations, 50% for 2010-2016) vs ~4% reality.
The pool's structural columns are rebuilt downstream (mortgage_conversion), so the donor's broken balances don't ship.
Why it still deserves a fix
support_clip range: donor_realized is vacuous for every column an aggregate-scale row touches — the realized range extends to $236B, so the clip can never catch a runaway draw. That defeats the backstop exactly where it's most needed.
Every puf_tax_detail output column shares the donor, so the exposure is not mortgage-specific.
Ask
Either screen aggregate-lineage rows out of the QRF donor path (they're identifiable by scale/weight; the disaggregate_aggregate_records op already knows the RECIDs upstream), or make them sane at ETL (per-return-scale values after disaggregation), and make support_clip robust to them (e.g., clip range from the non-aggregate donor support). Relates #481 (weight-loss lottery), #487 (donor uprating class), #511 (discovery context).
Found during #511 forensics; latent hazard, not the realized driver of that row (base-O pool is clean of it — receipts below).
What's in the donor
Measured via
puf_tax_unit_donor_from_arrayson the pinned artifact the certified O base consumed (base-osummarypuf_h5→policyengine_us_data/storage/puf_2024.h5, sha7669f5b5…, 211,677 donor rows):home_mortgage_interest≥ $10M — up to $235.97B on a single record (three identical $236.0B rows at weight 0.13 each; $177.0B ×2; $22.0B × w1.08; $3.7B × w11). Weighted, these rows hold $2,947B of phantom mass against $418B for the other ~94k carriers. TY2015 E19200 for ALL returns was $304.5B — single-record values of this scale are not tax data; they look like IRS disclosure-aggregate rows (or their clones) whose amounts were never brought back to per-return scale.Why nothing burned on O-1
home_mortgage_interest$3.09M, zero rows >$10M, pool implied first-mortgage rate 4.00%.Why it still deserves a fix
support_clip range: donor_realizedis vacuous for every column an aggregate-scale row touches — the realized range extends to $236B, so the clip can never catch a runaway draw. That defeats the backstop exactly where it's most needed.puf_tax_detailoutput column shares the donor, so the exposure is not mortgage-specific.Ask
Either screen aggregate-lineage rows out of the QRF donor path (they're identifiable by scale/weight; the
disaggregate_aggregate_recordsop already knows the RECIDs upstream), or make them sane at ETL (per-return-scale values after disaggregation), and makesupport_cliprobust to them (e.g., clip range from the non-aggregate donor support). Relates #481 (weight-loss lottery), #487 (donor uprating class), #511 (discovery context).