Skip to content

ensure_datasets: consume Microcosm-published UK releases (manifest-verified), so analyses can swap bases by URI #495

Description

@MaxGhenis

Doctrine says policyengine.py is the permanent interface and Microcosm is where UK data construction is migrating (PolicyEngine/microcosm#665). The consumption path is currently missing: pe.uk.ensure_datasets materialises per-year files from hf://policyengine/policyengine-uk-data/..., and there is no way to point it at a Microcosm-published UK release (certified single-year artifact + release manifest with pinned SHA-256s).

Concrete need, with a live consumer waiting: PolicyEngine/uk-cgt-paper#6 and PolicyEngine/uk-equalising-cgt#1 — both reform analyses should rerun on the Microcosm UK build once it certifies (currently gated on microcosm#630), ideally changing nothing but a dataset URI.

Scope sketch:

  • accept a Microcosm release identifier (HF repo + release id) in ensure_datasets;
  • verify artifact SHA-256 against the release manifest before use (Microcosm releases pin these; the interface should check, matching Microcosm's own runtime verification discipline);
  • resolve the release's compatible_model_packages / compatible_core_packages constraints and fail loudly on mismatch with the installed bundle;
  • forward-year materialisation (uprating 2023 → simulation years) with the same semantics the uk-data path applies today, so results are comparable across the swap.

Related: #491 (publisher claim). Happy to pair on the design — the UK CGT arc (microcosm#552) is the first artifact that wants this path.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions