Gate validation-config input coverage against source-stage outputs

[MaxGhenis]

Fixes #272.

Why

The reform-validation configs (obbba_reforms.json, tax_expenditure_reforms.json, soi_baseline_levels.json) score policy provisions on the calibrated dataset. A provision's simulated effect is driven by pure-input leaves — PolicyEngine-US variables with no formula, whose values must come from a source stage. When a provision's critical leaf is produced by no stage, the provision is a structural zero and the row validates as a silent zero until an external benchmark exposes it. Nothing checked this, which is exactly how the recent gaps stayed invisible:

• education credits depend on qualified_tuition_expenses — populated on 641 of 160,858 person records, so education credits validate ~40% low (qualified_tuition_expenses essentially unimputed (641 records, $4.5B) — education credits understated ≥40% #253); and
• the OBBBA auto-loan-interest deduction depends on qualified_passenger_vehicle_loan_interest — never imputed, so the provision is structurally $0 (qualified_passenger_vehicle_loan_interest is never imputed — OBBBA auto-loan deduction is structurally $0 #252).

What

A release gate that fails the build when a validation row's provision-critical input leaf is neither produced by a source stage nor allowlisted with a reason.

• source_stage_input_coverage_gate (pure, in gates.py): given {leaf -> validation rows that need it}, the declared source-stage outputs, and reviewed exclusions, it passes iff every required leaf is a declared output or carries a reason. Mirrors the reviewed-exclusion, stale/dormant, and named-failure conventions of the surrounding gates (a failure names the leaf and every affected row).
• us_validation_input_coverage_gate + the US_VALIDATION_PROVISION_INPUT_LEAVES registry (in us_runtime/validation_input_coverage.py): resolves the declared surface from source_stages.json (typed SourceManifest read) and runs the pure gate. Seeded with the two invisible-gap instances above, each allowlisted with its open issue.
• assert_validation_leaf_registry_current — anti-rot: each registry entry is checked against the live PolicyEngine-US graph via the engine tracer (the leaf must be a pure input and an input-leaf dependency of its provision), the reverse-direction check that keeps the other engine-derived guards from rotting. A no-op without the [us] extra.
• Wired into build_us_fiscal_refresh_release.py as an early preflight in main (before the expensive calibration, next to the CD-vintage preflight), and recorded in calibration_diagnostics.json under release_gates.validation_input_coverage.

Scope decision — curated critical leaves, not the transitive closure

The issue's literal phrasing is "every pure-input leaf a config variable depends on." Applied as the full transitive closure this is intractable as a gate: education_tax_credits/cdcc each touch 195 input leaves, and even after unioning every packaged input surface (source-stage outputs + CPS-carried + PUF defaults + structural) the residual undeclared set is 145 leaves — energy-credit expenditures, clean-vehicle flags, state-specific earnings, take-up flags. A gate failing on 145 leaves behind a 145-entry allowlist would drown the real signal and would never have singled out #252/#253. The registry instead names, per validation concern, the leaf whose absence zeroes that provision — the small high-signal set the issue's evidence table describes — and is kept honest by the live-graph consistency check rather than being a static list that silently rots. A new validation row whose provision keys on an un-imputed, un-allowlisted leaf fails CI here instead of shipping a silent zero.

Open question for review: this catches gaps for registered provisions. If you'd rather catch every new un-imputed leaf automatically (at the cost of a ~150-entry reasoned allowlist and churn as PolicyEngine-US adds variables), the pure gate already supports that mode — say the word and I'll widen the registry to the full closure.

Tests

TDD, failing-first. The plant-a-missing-leaf tests prove both the pure gate and the public entry point fail loudly (with the variable and affected row named) when a validation row keys on an un-imputed, un-allowlisted input leaf; passing cases and stale/dormant-exclusion handling are covered too. The live-graph registry-consistency test runs where the [us] extra is installed.

Local, reproducing each CI job:

• test job (uv sync --all-packages, no [us] extra): uv run pytest — 1348 passed, 48 skipped; uv run ruff check . — clean.
• wheels job (built wheels installed into a clean venv, no [us] extra): pytest packages/*/tests — 1309 passed, 57 skipped; the new module is present in the built populace_build wheel and importable.
• With the [us] extra installed, the live-graph registry check passes against real PolicyEngine-US.

Note on #307

This branch originally also carried a deflake for test_refit_l0_selection_threads_l2_lambda_to_refit (#307). PR #309 (l2-anchor-semantics) already fixes #307 at the root cause — it replaces the noisy ESS-direction assertion with the threading contract and shows the direction this branch's epsilon asserted actually inverts at production scale — so that change was dropped and this PR is #272-only, with no overlap on test_solve.py.

🤖 Generated with Claude Code