Skip to content

Release contract needs dataset-role classes: national-default checks reject valid non-default local-area artifacts #398

Description

@MaxGhenis

Finding (from publishing the first non-default artifact)

--no-latest (#397) made the pointer safe, but the release-contract validator remains hard-wired to the national certified default: it requires the US critical-target set (premium tax credit, taxable Social Security, itemized/SALT/medical deductions, …) in calibration_diagnostics.json and enumerates fiscal_target_sources against a national family expectation. The Build L ACS local-area artifact — calibrated deliberately to state/CD population marginals + state administrative families — can never satisfy that contract, by design rather than by defect.

Publishing it therefore had to route around the tracked path (direct Hub commit + tag, no pointer, all applicable gates passing and recorded). That's the wrong long-term state: non-default artifacts deserve a contract, just not the national one.

Proposal

Add a dataset-role dimension to the contract:

  • release_manifest.json already carries dataset_role (non_default_local_area on the Build L artifact) — key validation off it.
  • national_default (current behavior, unchanged): full US critical-target checks, national family enumeration, pointer update allowed.
  • local_area / research (non-default): require gates object + pass, calibration diagnostics shape (targets list ≡ n_targets, surface + registry refs, options), source coverage keyed to the families actually calibrated (derived from the diagnostics, not a national list), reviewed-limitations register non-empty or explicitly empty, and forbid the pointer op entirely for these roles (defense in depth beyond --no-latest).
  • publish CLI: refuse --no-latest absent on non-default roles and refuse pointer updates for them even if requested.

First consumer to migrate: populace-us-2024-buildl-acs-local-36de5d9a-20260712T104640Z (published directly 2026-07-12; files under releases/<id>/, tagged, pointer untouched and verified).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions