Add congressional district vintage crosswalk support - #207
Merged
Conversation
This was referenced Jun 27, 2026
MaxGhenis
added a commit
that referenced
this pull request
Jul 19, 2026
The CD geography-vintage translation (#207/#208/#209) consumed a crosswalk passed as an external CLI path with no versioned, reproducible, or cited artifact in the repo. This adds that artifact and its generator, and makes it the packaged default. The crosswalk is built by a single-vintage block overlay, so no 2010<->2020 block bridge is needed: - old (117th) district of each 2020 block: the 2020 Block Assignment File CD layer (BlockAssign_ST{fips}_{usps}_CD.txt), which carries the 116th-Congress plan (identical district geography to the 117th) on 2020 tabulation blocks; - current (119th) district of each 2020 block: the 119th BEF (NationalCD119.txt), the same source the block ladder already uses; - weight: 2020 P.L. 94-171 POP100 per block (the block ladder's parse_pl_geo_blocks convention). Both district assignments are read on the same 2020 blocks weighted by the same 2020 block populations, so each old district's population is redistributed across the current districts it overlaps and never invented. Population is the correct default basis (apportionment and equal-population redistricting are population operations); ACS income/tax proxy weights for fiscal targets are a documented future refinement. The committed national crosswalk covers all 436 current 119th-Congress districts from 436 source districts, with exact per-state population conservation over all 331,449,281 people in the 50 states + DC (zero unmatched or cross-state). The apportionment-shrunk districts (CA-53, IL-18, MI-14, NY-27, OH-16, PA-18, WV-03) appear only as sources; the new districts (CO-08, FL-28, MT-02, NC-14, OR-06, TX-37, TX-38) appear as populated targets. Montana's at-large district splits ~50/50 into MT-01/MT-02, as equal-population districts require. The derived crosswalk is a regenerable build artifact, not a Ledger fact -- the fact-vs-computed boundary of PolicyEngine/chronicle#71; the same declared-consumer- side-transform pattern applies to the Belgian NIS-code vintage work in PolicyEngine/chronicle#69. Files: - congressional_district_vintage_crosswalk.py: pure parsers (BAF CD layer, CD BEF) and the population-weighted join with conservation diagnostics. - tools/build_us_congressional_district_vintage_crosswalk.py: download + cache + provenance orchestration, mirroring build_us_block_ladder_artifact. - us/congressional_district_vintage_crosswalk.csv (+ .provenance.json + .md): the committed artifact, per-source SHA-256s, and the data-source doc. - congressional_district_vintage.py: packaged-default loader helpers. - build_us_fiscal_refresh_release.py: default to the packaged crosswalk when CD targets are requested and no path is passed. Closes #205. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
MaxGhenis
added a commit
that referenced
this pull request
Jul 19, 2026
* Build the 117th->119th CD vintage crosswalk from Census sources The CD geography-vintage translation (#207/#208/#209) consumed a crosswalk passed as an external CLI path with no versioned, reproducible, or cited artifact in the repo. This adds that artifact and its generator, and makes it the packaged default. The crosswalk is built by a single-vintage block overlay, so no 2010<->2020 block bridge is needed: - old (117th) district of each 2020 block: the 2020 Block Assignment File CD layer (BlockAssign_ST{fips}_{usps}_CD.txt), which carries the 116th-Congress plan (identical district geography to the 117th) on 2020 tabulation blocks; - current (119th) district of each 2020 block: the 119th BEF (NationalCD119.txt), the same source the block ladder already uses; - weight: 2020 P.L. 94-171 POP100 per block (the block ladder's parse_pl_geo_blocks convention). Both district assignments are read on the same 2020 blocks weighted by the same 2020 block populations, so each old district's population is redistributed across the current districts it overlaps and never invented. Population is the correct default basis (apportionment and equal-population redistricting are population operations); ACS income/tax proxy weights for fiscal targets are a documented future refinement. The committed national crosswalk covers all 436 current 119th-Congress districts from 436 source districts, with exact per-state population conservation over all 331,449,281 people in the 50 states + DC (zero unmatched or cross-state). The apportionment-shrunk districts (CA-53, IL-18, MI-14, NY-27, OH-16, PA-18, WV-03) appear only as sources; the new districts (CO-08, FL-28, MT-02, NC-14, OR-06, TX-37, TX-38) appear as populated targets. Montana's at-large district splits ~50/50 into MT-01/MT-02, as equal-population districts require. The derived crosswalk is a regenerable build artifact, not a Ledger fact -- the fact-vs-computed boundary of PolicyEngine/chronicle#71; the same declared-consumer- side-transform pattern applies to the Belgian NIS-code vintage work in PolicyEngine/chronicle#69. Files: - congressional_district_vintage_crosswalk.py: pure parsers (BAF CD layer, CD BEF) and the population-weighted join with conservation diagnostics. - tools/build_us_congressional_district_vintage_crosswalk.py: download + cache + provenance orchestration, mirroring build_us_block_ladder_artifact. - us/congressional_district_vintage_crosswalk.csv (+ .provenance.json + .md): the committed artifact, per-source SHA-256s, and the data-source doc. - congressional_district_vintage.py: packaged-default loader helpers. - build_us_fiscal_refresh_release.py: default to the packaged crosswalk when CD targets are requested and no path is passed. Closes #205. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Move crosswalk artifact to us_runtime/data; default builder crosswalk CI (wheels job) surfaced two issues: 1. The `us/` country package is a spec-only directory: the governance tests (test_spec_only_country_packages) and country_spec.py require it to contain only .json resources, all declared in country_package.json. Move the crosswalk CSV, its provenance JSON, and the data-source doc to a new us_runtime/data/ package (us_runtime is exempt from the spec-only rule), and re-anchor the packaged-default loader at populace.build.us_runtime.data. 2. test_cd_targets_require_vintage_crosswalk asserted the release builder errors when CD targets are requested without a crosswalk; the builder now defaults to the packaged crosswalk, so rename/rewrite the test to assert the default is applied and an explicit path still overrides it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Correct crosswalk help text to us_runtime.data location Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Normalize crosswalk weights; keep population masses; strict packaged contract Cross-model review findings (gpt-5.6-sol adversarial pass on #288): 1. The committed CSV's weight column held raw pair populations (per-source sums 543k-1.08M, not 1.0) while the PR text and column name promised weights — correct only because the loader silently re-normalized any positive mass. The artifact is now self-describing: pair_population carries the 2020 P.L. 94-171 mass (conserving state totals exactly) and weight is that mass over the source district's total, summing to 1.0 per source in the raw file (worst deviation 2.2e-16). 2. The generic loader accepted duplicate pairs (silently aggregated) and arbitrary positive magnitudes. Duplicates now raise; when pair_population is present the weight column is cross-checked against it (1e-9), so a corrupted weight column cannot load. 3. Nothing enforced the published redistricting geometry on the packaged artifact. Default-path loads now run a strict contract: exactly 436 districts per side across 51 jurisdictions, contiguous per-state rosters, weights in (0,1] summing to 1 per source, and identity rows (weight 1.0) for the seven both-vintage at-large jurisdictions (AK/DE/DC/ND/SD/VT/WY). External override files keep the permissive any-positive-mass semantics. Tests pin the raw-CSV contract (not just loader-derived shares), the MT 50/50 split as shares, duplicate and inconsistency rejection, and the packaged validator's failure modes. Docs and provenance sha updated; generator emits both columns. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #205.
Verification
uv run --python 3.13 ruff check packages/populace-build/src/populace/build/us_runtime/congressional_district_vintage.py packages/populace-build/src/populace/build/us_runtime/__init__.py tools/build_us_puf_support_base.py tools/build_us_fiscal_refresh_release.py tools/score_us_fiscal_targets.py packages/populace-build/tests/test_us_congressional_district_vintage.py packages/populace-build/tests/test_us_fiscal_refresh_builder.py packages/populace-build/tests/test_us_puf_support_base_builder.pyuv run --python 3.13 --package populace-build --group dev python -m pytest packages/populace-build/tests/test_us_congressional_district_vintage.py packages/populace-build/tests/test_us_congressional_district_geography.py packages/populace-build/tests/test_us_fiscal_targets.py packages/populace-build/tests/test_us_fiscal_refresh_builder.py packages/populace-build/tests/test_us_puf_support_base_builder.py -qgit diff --cached --checkbefore commit /git diff --checkafter edits