Skip to content

UK source packages, wave 2: national fiscal/benefit families + uk_national target profile #133

Description

@juaristi22

Lanes: packages under ledger-source-ingestor; the profile PR under ledger-target-profile-author (judges: ledger-target-profile + ledger-boundary)
Depends on: #132 (parity fixture established); #131 for the slc_repayments package; Stat-Xplore extracts follow the existing api_response artifact pattern — no new policy needed

Context

policyengine-uk-data (pinned ref for this migration: ebf733c, 2026-07-26) compiles its national calibration registry from 24 source modules under policyengine_uk_data/targets/sources/. Wave 1 ports the three families production last calibrated against; this issue ports the remainder of the national registry, one publication table per package, and then declares the uk_national target profile over them.

Ruling on vintages (2026-08-05): every source-backed original vintage is eventually portable — porting a fact never activates it; Populace owns the active subset. Tranche 1 = current vintages below. Historical vintages (earlier EFO rounds, prior SPI editions, …) land later through a standing backfill lane; the vintage-suffixed package-ID convention (obr/efo_march_2026_*) guarantees no collisions. Uprated/aged/PolicyEngine-computed values never enter Ledger (docs/adr-ledger-facts-only.md).

Package inventory (uk-data module → package)

uk-data module (lines / Target() ctors) package(s) notes
obr.py (730 / 13, ~28 realized targets × FY2024–30) obr/efo_march_2026_receipts, obr/efo_march_2026_expenditure assertion: source_projection, period_type: fiscal_year. uk-data commits the EFO workbooks under storage/obr_efo/ because obr.uk 403s CI runners — R2 archival is the proper fix for the same problem; ingest from the archived workbooks.
dwp.py (323 / 21) dwp/benefit_stats_feb_2026, dwp/pip_stats_jan_2026, dwp/benefit_cap_nov_2025, dwp/two_child_limit_apr_2025, dwp/uc_payment_distribution Values are hand-transcribed in uk-data with reference URLs — the resolve_pending.py first-print lane fits most of these. UC payment distribution comes via Stat-Xplore: capture the API response as the artifact (sheet_name: api_response precedent, 5 existing packages; Census-CD API precedent for the extraction_method wording). A free DWP Stat-Xplore key is needed only by whoever runs the extract.
hmrc_spi.py (366 / 4) hmrc/spi_collated_2324 Same collated ODS populace's national build parses natively (hmrc_income.py); one package serves both eventually, but do not touch populace's HMRC stage in this issue.
hmrc_cgt.py (268 / 4) hmrc/cgt_statistics_2024 Single-homing: populace's native UK_CGT_TARGET_SPECS (populace/build/uk_runtime/fiscal_targets.py) and uk-data's module both retire onto this one package (populace retirement happens in PolicyEngine/populace#622).
hmrc_salary_sacrifice.py (129 / 3) hmrc/salary_sacrifice_tables_6_1_6_2 Publisher CSV already.
slc_repayments.py (100 / 1) slc/loan_repayments (or extend the wave-1 SLC package) Academic-year periods — needs #131.
voa_council_tax.py (300 / 4) voa/council_tax_stock_2025 National band counts; LA-level stock goes to wave 3.
ons_households.py, ons_tenure.py, housing.py (275 / 5) ons/families_households_2024, ons/tenure_2024 Curated extracts; the ONS families & households datasets also lack stable URLs — same lane as SNPP.
ons_savings.py (61 / 1) ons/savings_interest_haxv ONS time series (HAXV, UKEA).
ons_public_sector_employment.py (51 / 1) ons/public_sector_employment_2026 New module since June.
nts_vehicles.py (49 / 3) dft/nts_vehicle_ownership_2024 Publisher is DfT (National Travel Survey); new agency directory.
scottish_government.py (37 / 1) scottish_government/budget_2026_27
mhclg_regional_land.py, ons_land_values.py, _land.py (169 / 4) ons/land_values, mhclg/regional_land Verify the actual publisher/table for each before packaging — _land.py is a shared helper and the land series have moved between ONS/MHCLG publications.

Survey-derived aggregates (any APS/LCF-based table encountered) additionally name survey_instrument per the README provenance rules.

The uk_national profile

After the packages above land: one PR in the ledger-target-profile-author lane adding policyengine_ledger/target_profiles/uk_national.json

  • selectors referencing the raw facts above (deterministic check: selector references raw Ledger facts),
  • measurement + bindings.policyengine mirroring uk-data's targets/compute/* semantics (entity, map_to, filters),
  • no values (FORBIDDEN_VALUE_KEYS in target_profiles/model.py), no runtime hooks,
  • counterfactual/cross-tab targets (two-child limit, UC family-type splits, benefit cap) get declarative binding payloads only — the metric providers live in populace, following the JCT neutralize_variable precedent in populace's us/fiscal_target_references.json.

Acceptance

  • Every package: manifest source_url + sha256 + R2 URI; cell-geometry guards; concept_evidence_url per measure; deterministic checks green; judge lanes as headed.
  • The uk_national profile validates with no embedded values and resolves every selector against built facts.
  • A consumer artifact covering waves 1+2 resolves every national-registry row that uk-data compiles at ebf733c for the current calibration year, with any intentional differences (vintage updates, dropped rows) enumerated in the PR description — no silent gaps. (The mechanical compile-parity gate itself lands populace-side.)

Out of scope

Sizing

~13–14 packages, batched per agency ≈ 5–6 PRs + 1 profile PR.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions