Lanes: packages under ledger-source-ingestor; the profile PR under ledger-target-profile-author (judges: ledger-target-profile + ledger-boundary)
Depends on: #132 (parity fixture established); #131 for the slc_repayments package; Stat-Xplore extracts follow the existing api_response artifact pattern — no new policy needed
Context
policyengine-uk-data (pinned ref for this migration: ebf733c, 2026-07-26) compiles its national calibration registry from 24 source modules under policyengine_uk_data/targets/sources/. Wave 1 ports the three families production last calibrated against; this issue ports the remainder of the national registry, one publication table per package, and then declares the uk_national target profile over them.
Ruling on vintages (2026-08-05): every source-backed original vintage is eventually portable — porting a fact never activates it; Populace owns the active subset. Tranche 1 = current vintages below. Historical vintages (earlier EFO rounds, prior SPI editions, …) land later through a standing backfill lane; the vintage-suffixed package-ID convention (obr/efo_march_2026_*) guarantees no collisions. Uprated/aged/PolicyEngine-computed values never enter Ledger (docs/adr-ledger-facts-only.md).
Package inventory (uk-data module → package)
| uk-data module (lines / Target() ctors) |
package(s) |
notes |
obr.py (730 / 13, ~28 realized targets × FY2024–30) |
obr/efo_march_2026_receipts, obr/efo_march_2026_expenditure |
assertion: source_projection, period_type: fiscal_year. uk-data commits the EFO workbooks under storage/obr_efo/ because obr.uk 403s CI runners — R2 archival is the proper fix for the same problem; ingest from the archived workbooks. |
dwp.py (323 / 21) |
dwp/benefit_stats_feb_2026, dwp/pip_stats_jan_2026, dwp/benefit_cap_nov_2025, dwp/two_child_limit_apr_2025, dwp/uc_payment_distribution |
Values are hand-transcribed in uk-data with reference URLs — the resolve_pending.py first-print lane fits most of these. UC payment distribution comes via Stat-Xplore: capture the API response as the artifact (sheet_name: api_response precedent, 5 existing packages; Census-CD API precedent for the extraction_method wording). A free DWP Stat-Xplore key is needed only by whoever runs the extract. |
hmrc_spi.py (366 / 4) |
hmrc/spi_collated_2324 |
Same collated ODS populace's national build parses natively (hmrc_income.py); one package serves both eventually, but do not touch populace's HMRC stage in this issue. |
hmrc_cgt.py (268 / 4) |
hmrc/cgt_statistics_2024 |
Single-homing: populace's native UK_CGT_TARGET_SPECS (populace/build/uk_runtime/fiscal_targets.py) and uk-data's module both retire onto this one package (populace retirement happens in PolicyEngine/populace#622). |
hmrc_salary_sacrifice.py (129 / 3) |
hmrc/salary_sacrifice_tables_6_1_6_2 |
Publisher CSV already. |
slc_repayments.py (100 / 1) |
slc/loan_repayments (or extend the wave-1 SLC package) |
Academic-year periods — needs #131. |
voa_council_tax.py (300 / 4) |
voa/council_tax_stock_2025 |
National band counts; LA-level stock goes to wave 3. |
ons_households.py, ons_tenure.py, housing.py (275 / 5) |
ons/families_households_2024, ons/tenure_2024 |
Curated extracts; the ONS families & households datasets also lack stable URLs — same lane as SNPP. |
ons_savings.py (61 / 1) |
ons/savings_interest_haxv |
ONS time series (HAXV, UKEA). |
ons_public_sector_employment.py (51 / 1) |
ons/public_sector_employment_2026 |
New module since June. |
nts_vehicles.py (49 / 3) |
dft/nts_vehicle_ownership_2024 |
Publisher is DfT (National Travel Survey); new agency directory. |
scottish_government.py (37 / 1) |
scottish_government/budget_2026_27 |
|
mhclg_regional_land.py, ons_land_values.py, _land.py (169 / 4) |
ons/land_values, mhclg/regional_land |
Verify the actual publisher/table for each before packaging — _land.py is a shared helper and the land series have moved between ONS/MHCLG publications. |
Survey-derived aggregates (any APS/LCF-based table encountered) additionally name survey_instrument per the README provenance rules.
The uk_national profile
After the packages above land: one PR in the ledger-target-profile-author lane adding policyengine_ledger/target_profiles/uk_national.json —
- selectors referencing the raw facts above (deterministic check: selector references raw Ledger facts),
measurement + bindings.policyengine mirroring uk-data's targets/compute/* semantics (entity, map_to, filters),
- no values (
FORBIDDEN_VALUE_KEYS in target_profiles/model.py), no runtime hooks,
- counterfactual/cross-tab targets (two-child limit, UC family-type splits, benefit cap) get declarative binding payloads only — the metric providers live in populace, following the JCT
neutralize_variable precedent in populace's us/fiscal_target_references.json.
Acceptance
- Every package: manifest
source_url + sha256 + R2 URI; cell-geometry guards; concept_evidence_url per measure; deterministic checks green; judge lanes as headed.
- The
uk_national profile validates with no embedded values and resolves every selector against built facts.
- A consumer artifact covering waves 1+2 resolves every national-registry row that uk-data compiles at
ebf733c for the current calibration year, with any intentional differences (vintage updates, dropped rows) enumerated in the PR description — no silent gaps. (The mechanical compile-parity gate itself lands populace-side.)
Out of scope
Sizing
~13–14 packages, batched per agency ≈ 5–6 PRs + 1 profile PR.
Lanes: packages under
ledger-source-ingestor; the profile PR underledger-target-profile-author(judges:ledger-target-profile+ledger-boundary)Depends on: #132 (parity fixture established); #131 for the
slc_repaymentspackage; Stat-Xplore extracts follow the existingapi_responseartifact pattern — no new policy neededContext
policyengine-uk-data(pinned ref for this migration:ebf733c, 2026-07-26) compiles its national calibration registry from 24 source modules underpolicyengine_uk_data/targets/sources/. Wave 1 ports the three families production last calibrated against; this issue ports the remainder of the national registry, one publication table per package, and then declares theuk_nationaltarget profile over them.Ruling on vintages (2026-08-05): every source-backed original vintage is eventually portable — porting a fact never activates it; Populace owns the active subset. Tranche 1 = current vintages below. Historical vintages (earlier EFO rounds, prior SPI editions, …) land later through a standing backfill lane; the vintage-suffixed package-ID convention (
obr/efo_march_2026_*) guarantees no collisions. Uprated/aged/PolicyEngine-computed values never enter Ledger (docs/adr-ledger-facts-only.md).Package inventory (uk-data module → package)
obr.py(730 / 13, ~28 realized targets × FY2024–30)obr/efo_march_2026_receipts,obr/efo_march_2026_expenditureassertion: source_projection,period_type: fiscal_year. uk-data commits the EFO workbooks understorage/obr_efo/because obr.uk 403s CI runners — R2 archival is the proper fix for the same problem; ingest from the archived workbooks.dwp.py(323 / 21)dwp/benefit_stats_feb_2026,dwp/pip_stats_jan_2026,dwp/benefit_cap_nov_2025,dwp/two_child_limit_apr_2025,dwp/uc_payment_distributionresolve_pending.pyfirst-print lane fits most of these. UC payment distribution comes via Stat-Xplore: capture the API response as the artifact (sheet_name: api_responseprecedent, 5 existing packages; Census-CD API precedent for the extraction_method wording). A free DWP Stat-Xplore key is needed only by whoever runs the extract.hmrc_spi.py(366 / 4)hmrc/spi_collated_2324hmrc_income.py); one package serves both eventually, but do not touch populace's HMRC stage in this issue.hmrc_cgt.py(268 / 4)hmrc/cgt_statistics_2024UK_CGT_TARGET_SPECS(populace/build/uk_runtime/fiscal_targets.py) and uk-data's module both retire onto this one package (populace retirement happens in PolicyEngine/populace#622).hmrc_salary_sacrifice.py(129 / 3)hmrc/salary_sacrifice_tables_6_1_6_2slc_repayments.py(100 / 1)slc/loan_repayments(or extend the wave-1 SLC package)voa_council_tax.py(300 / 4)voa/council_tax_stock_2025ons_households.py,ons_tenure.py,housing.py(275 / 5)ons/families_households_2024,ons/tenure_2024ons_savings.py(61 / 1)ons/savings_interest_haxvons_public_sector_employment.py(51 / 1)ons/public_sector_employment_2026nts_vehicles.py(49 / 3)dft/nts_vehicle_ownership_2024scottish_government.py(37 / 1)scottish_government/budget_2026_27mhclg_regional_land.py,ons_land_values.py,_land.py(169 / 4)ons/land_values,mhclg/regional_land_land.pyis a shared helper and the land series have moved between ONS/MHCLG publications.Survey-derived aggregates (any APS/LCF-based table encountered) additionally name
survey_instrumentper the README provenance rules.The
uk_nationalprofileAfter the packages above land: one PR in the
ledger-target-profile-authorlane addingpolicyengine_ledger/target_profiles/uk_national.json—measurement+bindings.policyenginemirroring uk-data'stargets/compute/*semantics (entity, map_to, filters),FORBIDDEN_VALUE_KEYSintarget_profiles/model.py), no runtime hooks,neutralize_variableprecedent in populace'sus/fiscal_target_references.json.Acceptance
source_url+ sha256 + R2 URI; cell-geometry guards;concept_evidence_urlper measure; deterministic checks green; judge lanes as headed.uk_nationalprofile validates with no embedded values and resolves every selector against built facts.ebf733cfor the current calibration year, with any intentional differences (vintage updates, dropped rows) enumerated in the PR description — no silent gaps. (The mechanical compile-parity gate itself lands populace-side.)Out of scope
db/etl_obr.py/db/etl_ons.pydicts — superseded by these packages; their cleanup is tracked in UK enablement tooling: source-inventory de-hardcode, UK consumer-artifact suite, legacy ETL cleanup #135.Sizing
~13–14 packages, batched per agency ≈ 5–6 PRs + 1 profile PR.