Impute SSN card type and immigration status from CPS ASEC citizenship

[MaxGhenis]

Fixes #225.

Problem

The published US dataset stored zero SSN/immigration/ITIN columns, so policyengine-us defaulted every person to a citizen with a valid SSN. Every SSN- or citizenship-conditioned policy was a no-op: OBBBA's CTC SSN tightening scored ≈ $0 (vs +$3.3B in PolicyEngine's official OBBBA model), EITC SSN requirements excluded nobody, and ACA/Medicaid/SNAP/SSI immigration axes treated everyone as eligible.

Approach

A new immigration_status source stage (manifest entry in source_stages.json + a derive_immigration_status US runtime handler), designed fresh rather than porting the incumbent wholesale:

1. Citizenship is measured, not imputed — PRCITSHP 1–4 → CITIZEN, 5 → non-citizen.
2. ASEC-UA residual method (Van Hook et al., SSRN 4662801): non-citizens with any legal-status indicator (pre-1982 IRCA arrival, naturalization eligibility, Medicare/Medicaid/SSI/Social Security, federal pension, IHS/CHAMPVA/military coverage, government employment, subsidized housing, veteran status) → OTHER_NON_CITIZEN.
3. Work/study authorization split — the CPS has no work-authorization variable, so residual non-citizen workers/students spill to NON_CITIZEN_VALID_EAD in deterministic seeded order until the remaining undocumented worker/student counts match published controls (Pew 8.3M workers; Presidents' Alliance ~408k students). These are the only forced margins, and each control carries a citation in the manifest.
4. The total undocumented population is emergent, not forced — 13.3M on the full 2024 ASEC, inside the range of published 2023–24 estimates. A release gate checks it against Pew's cited 11.0M (2022) anchor with a coarse [0.5, 1.6] band. Reconciling the level is the calibration lane's job (follow-up: a Ledger fact with SE).
5. Status tags carry only statutory tests the data supports — DACA (arrival-cohort test among EAD holders), CUBAN_HAITIAN_ENTRANT (nativity + post-1980 arrival); other documented non-citizens stay LEGAL_PERMANENT_RESIDENT.

Deliberate divergences from the incumbent (policyengine-us-data)

• No family-correlation step. The incumbent flipped OTHER_NON_CITIZEN members of mixed households back to undocumented to hit a 13M total — reclassifying people with affirmative legal-status indicators (Medicaid, Social Security) and corrupting exactly the program-participation↔status correlation that benefit analysis needs. It was also a no-op on current data (the emergent total already exceeded the target). Representation belongs to calibration per the charter.
• No blanket REFUGEE/TPS labels. The incumbent tagged every recently-arrived documented non-citizen REFUGEE and every leftover EAD holder TPS — mislabeling millions (true stocks are under 1M each) and over-granting refugee-class benefit exemptions. LPR treatment gives near-identical means-tested eligibility for those populations and is honest about modal composition.
• No np.random state. Selection draws are blake2b hashes keyed by source person identity (source_year/source_person_id), so support-channel clones of one source person always land on the same side of a selection threshold and reruns are bit-reproducible.
• Below-control counts spill nothing (the incumbent moved a share to EAD even when already below target).

Wiring

• build_us_puf_support_base.py runs the stage after derive_cps_carried, before channel cloning; the summary records the composition.
• build_us_fiscal_refresh_release.py runs it idempotently after the census mass repair (so absolute controls bind at full population scale) and blocks the release on a new immigration_composition gate: columns present, non-constant, enum-domain-valid, cross-column-consistent (CITIZEN⟺CITIZEN, NONE⟺UNDOCUMENTED), non-citizen share in [3%, 12%], undocumented total within the anchor band. The gate lands in calibration_diagnostics.json, build_manifest.json, and release_manifest.json alongside the existing gates.
• The L0/refit exporter now requires both person columns to carry signal, so a sparse release can't ship the Dataset imputes 100% of the population as citizens with valid SSNs (breaks SSN/citizenship-conditioned policies, e.g. OBBBA CTC) #225 failure mode.
• New stage name in US_STAGE_NAMES + US_DONORS citation entry.

Validation

• 51 new behavioral tests (test_us_immigration.py): residual conditions, control binding, weight-awareness, determinism/seed-sensitivity, clone consistency, statutory tags, idempotence, loud failures, manifest citation discipline, gate bands.
• Full suite: 1,126 passed, 10 skipped; ruff check clean.
• Full real-data run on the 2024 ASEC (326.0M persons):

ssn_card_type	this PR	incumbent usdata
CITIZEN	299.37M	299.37M
NONE (undocumented)	13.34M	13.24M
NON_CITIZEN_VALID_EAD	4.67M	4.76M
OTHER_NON_CITIZEN	8.59M	8.59M

Status tags: 360k DACA, 660k Cuban/Haitian entrants, 12.2M LPR. Composition gate passes.

Notes

• The published artifact needs a rebuild to pick this up; the next fiscal-refresh release derives the columns idempotently from any base H5 that still carries the raw ASEC columns (all current ones do).
• Follow-ups filed after merge: an undocumented-population calibration target in the Ledger facts lane, and a conditional model for the EAD split if a status-observed donor ever materializes.

🤖 Generated with Claude Code