US PUMA-anchored geography ladder (ACS/ASEC spine)

[MaxGhenis]

US PUMA-anchored geography ladder

Ports the UK output-area ladder (#354) to the US ACS/ASEC spine, the US analogue
of the block ladder (geography_ladder.py). The anchor is the 2020 Public Use
Microdata Area — the finest geography the ACS PUMS publishes — so an ACS record
already knows its anchor, and every coarser geography derives from it. One
national dataset, filterable at any grain, never per-area files (#275).

The ladder

• An ACS-spine record carries its puma; the ladder keeps it.
• An ASEC-spine record knows only state_fips; the ladder draws a PUMA within
  that state proportional to 2020 PUMA population (the two-stage discipline of the
  UK ladder: sample the anchor within the known marginal, then derive).
• Every coarser layer derives from the PUMA by a population-weighted draw over the
  PUMA's block-population overlap with that layer:
  • congressional district (119th) — (PUMA, CD) overlap
  • county — (PUMA, county) overlap
  • tract (behind assign_tract, default off) — (PUMA, tract) overlap; when
    assigned, the county derives structurally from the tract (county = tract // 10**6)
    so the two never disagree.

Congressional district, county and state are the launch requirement; tract is
implemented but flagged off. Draws are consumed from one seeded generator in a
fixed sorted order (states, then PUMAs), so an assignment is reproducible from its
seed. The dense ACS-spine artifact is filtered to state / CD / county for local
analysis because every record carries all three.

PUMAs do not nest in congressional districts or counties, so each derived layer is
a genuine population-weighted draw over the PUMA's overlap distribution — not a
structural prefix as in the block ladder. That is the substantive difference from
the block ladder, and why the artifact is a set of overlap tables rather than one
row per anchor.

One block pass, exact conservation

2020 blocks nest in 2020 tracts nest in 2020 PUMAs, so tract = block // 10**4,
county = block // 10**10, and tract → PUMA is a lookup. Summing 2020 block
populations by (PUMA, CD), (PUMA, county), (PUMA, tract) and by PUMA yields
the three overlap tables and the anchor population from a single block pass. Every
populated block contributes to exactly one PUMA, CD, county and tract, so each
overlap table conserves its PUMA's population exactly — the loader refuses an
artifact that violates conservation, omits a derived-layer vintage
(vintage_policy: error, the #205 lesson), or carries an overlap PUMA absent from
the anchor.

The block → CD and block → population parsers are reused unchanged from the
block ladder (block_ladder_sources.py, #288's lineage). The only source this
ladder adds is the small Census tract-to-PUMA relationship file.

Sources (pinned: URL + SHA-256 in us_puma_ladder.provenance.json)

Role	Census source	Vintage
tract → PUMA (county via the tract prefix)	2020 Census Tract to 2020 PUMA relationship file	2020
block → 119th CD	119th Congressional District BEF (NationalCD119.txt)	119th Congress
block → population (the weight)	2020 P.L. 94-171 geographic headers, per state	2020

tools/build_us_puma_ladder_artifact.py downloads/caches the sources, runs the
block pass, self-checks the artifact by loading it back through
load_us_puma_ladder, and records every source's URL + SHA-256, the artifact
SHA-256, and the national totals in the committed manifest. Alignment with #288:
this reuses #288's Census-source lineage (same CD BEF and P.L. block populations)
rather than duplicating it; #288 remains the place the 117th→119th vintage
crosswalk lives.

National build (validation)

Built over all 51 states from the pinned sources, conserving exactly:

• 2,462 PUMAs, 331,449,281 people — the 2020 apportionment population
• 436 congressional districts, 3,143 counties
• 4,043 CD overlaps, 4,620 county overlaps, 83,848 tract overlaps, from 5,769,942
  populated blocks
• byte-identical across runs; SHA-256 pinned in the manifest

The built NPZ lands in the gitignored build/ tree (no per-area files, not
committed) and is reproducible in ~46s via the one command in US_PUMA_LADDER.md.

Implemented vs flagged-off

• Implemented (launch): PUMA anchor (ACS keep / ASEC state-weighted draw),
  congressional district, county, state; the loader, assignment, Frame wrapper,
  provenance summary, and a release gate (structure, state consistency, and the
  NYC-collapse check — the permanent form of the Fix in_nyc before enforcing no-formula exports #34 in_nyc regression).
• Implemented but flagged off (assign_tract=False): the tract rung. The
  overlap table ships in the artifact; assignment only draws it when the flag is
  set, and then re-derives county from the tract for consistency.

Tests (35 new; 72 green including block-ladder regression)

• source parse/assemble + exact per-PUMA conservation; territory filtering; header,
  width, and conflicting-PUMA refusals
• load refusals: missing key, missing layer vintage (vintage_policy: error),
  non-positive population, unsorted/duplicate PUMAs, non-conserving overlap,
  overlap PUMA absent from anchor, CD/county/tract state mismatch
• deterministic (same seed → same assignment) and population-weighted
  assignment: CD ~90/10 within a two-district PUMA, and the ASEC state→PUMA draw
  matches the 1000/1500 population split within tolerance
• ACS keep / ASEC draw / mixed frame; the tract flag deriving a consistent county;
  CD stability across the tract flag; coverage and CD-vintage refusals
• Frame integration (mass preserved), assignment summary, and gate (pass +
  NYC-collapse / prefix / missing-column failures)

uv run ruff check and ruff format --check clean.

Wire-up (documented, not rewired)

load_us_puma_ladder → assign_us_puma_ladder / with_household_us_puma_ladder
→ us_puma_ladder_gate is the drop-in for the ACS/ASEC spine's geography stage.
Wiring it behind the country-spec geography-spine schema (a PUMA-anchored spine
method reading the built artifact from the release bundle) is the follow-up, kept
out of this change to stay reviewable — the same posture #354 took for the UK
ladder.

Refs #275, #205, #34, #354, #288.

🤖 Generated with Claude Code