Python ↔ Rust parity harness: cell-for-cell diff on shared FRS slice

[vahid-ahmadi]

Gap

A grep of the repo finds no comparison or parity-validation code between the Rust engine and Python policyengine-uk. There is currently no automated way to detect when a Rust output drifts from Python.

Why it matters

• Every variable port (issues Universal Credit: port full programme structure (43 Python files → ~1 Rust fn) #40, National Insurance: split into Class 1 / 2 / 3 / 4 with employee, employer, self-employed detail #43–46) is a chance to introduce a silent regression that nobody catches until a downstream user reports a divergent number
• The Python model is the source of truth for 'is this output correct?' — without a parity check, the Rust port is flying blind
• Calibration (src/data/calibrate.rs) is meaningless if individual-variable outputs diverge from Python pre-aggregation

Proposal

A script (Python preferred, can live in scripts/ or interfaces/python/policyengine_uk_compiled/parity.py) that:

1. Loads the same FRS slice into Python policyengine-uk and Rust policyengine_uk_compiled
2. Computes a fixed list of key outputs in both: income_tax, national_insurance, universal_credit, child_benefit, tax_credits, housing_benefit, state_pension, household_net_income, total_tax, total_benefits
3. Diffs cell-for-cell at the household level (not just aggregates — aggregates can mask offsetting errors)
4. Reports max abs diff, mean abs diff, and the top-N largest-diff households per variable
5. Runs in CI weekly (full FRS) and on PRs (small sampled slice) and fails if diffs exceed configurable tolerances

Why this is high-leverage

This is S–M effort but pays for itself on every subsequent port. Recommend landing this before the next batch of variable ports so they land with parity already verified.

References

• Python wrapper: interfaces/python/policyengine_uk_compiled/engine.py
• Python truth: policyengine_uk/microsimulation.py, policyengine_uk/simulation.py
• Rust outputs: src/main.rs simulate endpoint, src/engine/

Effort

Small to medium. Most of the wiring already exists in interfaces/python/policyengine_uk_compiled/; the new code is the pairing + diff logic + CI integration.