Actionable reform-input validation for the MCP/CLI tools

[vahid-ahmadi]

Fixes two developer-experience defects found in live end-to-end testing of the hosted MCP server.

1. Date-range reform keys returned a raw pydantic dump

{"gov.hmrc.income_tax.rates.uk[0].rate": {"2026-01-01.2029-12-31": 0.21}} returned:

Error executing tool score_reform: 1 validation error for Simulation — Value error, unconverted data remains: .2029-12-31

Root cause (upstream, confirmed in the installed package). policyengine/tax_benefit_models/common/reform.py compiles each reform entry to ParameterValue(start_date=datetime.strptime(effective_date, "%Y-%m-%d"), end_date=None, ...). Every reform value is applied from an effective date and stays in force indefinitely — end_date is hardcoded to None, and the reform-dict API exposes no field for it. strptime("2026-01-01.2029-12-31", "%Y-%m-%d") reproduces the reported error exactly.

Decision: reject explicitly, do not fake. Accepting the range and silently using only the start date would score a permanent reform while the caller asked for a temporary one — a wrong number returned confidently, the worst failure mode for a costing tool. The error now names the end date it cannot express, shows the correct single-date form, and gives a workaround:

reform['gov.hmrc.income_tax.rates.uk[0].rate'] uses the date-range key
'2026-01-01.2029-12-31', which the PolicyEngine reform-dict API does not
support: a reform value is applied from an effective date and stays in force
indefinitely (end_date is always None), so the end date 2029-12-31 cannot be
expressed.
Use the start date alone to score the reform as permanent: {"2026-01-01": 0.21}.
To approximate a time-limited reform, score the affected years individually
(e.g. one population_reform_impact call per year in the window) and treat later
years as baseline.
Supported reform shapes (per parameter path):
  - flat value:      {"gov.hmrc.income_tax.rates.uk[0].rate": 0.21}
  - effective date:  {"gov.hmrc.income_tax.rates.uk[0].rate": {"2026-01-01": 0.21}}

If upstream ever exposes end_date through the reform dict, validate_reform is the single place to change.

2. calculate_household returned a raw pydantic dump on missing arguments

Omitting country produced 2 validation errors for calculate_householdArguments: country Field required... people Field required. The MCP arguments are now optional at the schema level and validated in core, so callers get a sentence:

country is required and must be 'uk' or 'us' — there is no default, because the
two are different tax-benefit models and guessing would silently score the wrong
country. Example: country="uk".

No default for country. pe_population_impact keeps its historical country="uk" default for compatibility, but defaulting the household tools would silently run a US household through UK rules. The error message is made good instead of guessing a default.

Changes

• core.validate_reform() — one validator behind every tool that accepts a reform (pe_household, pe_household_impact, pe_population_impact, score_reform, and the OG/OBR bridges). Accepts flat scalars and single-date keys, returns the normalised dict, and raises ValueError naming the supported shapes with a worked example. _validate_reform kept as a thin alias.
• _validate_country / _validate_people — actionable messages for the other required arguments. Validation runs before the heavy PolicyEngine import, so bad input fails in milliseconds.
• _og_build_policy — now honours dated values (previously assumed scalars), emitting one ParameterValue per effective date.
• score_reform — a missing/unknown model names the three valid options and what each is for.
• Docstrings / CLI help — MCP tool docs and --reform help document the two supported shapes and state that ranges are unsupported, so agents get it right on the first call.

Tests

New integration/tests/test_reform_input.py (28 tests): flat value, integer/bool values, single-date key, multiple dates on one parameter, mixed flat+dated; date-range rejection (both . and : separators) asserting the message names the end date, shows the correct form, and leaks no upstream text; and each malformed input's message (empty, non-dict, bad date key, non-numeric value, non-string path, missing country/people, unknown model). Also asserts PolicyEngine is never imported when input is invalid.

87 passed, 45 skipped, 1 failed

The single failure is test_core.py::test_summary_parses, pre-existing and environmental — boe_var is not installed locally and MACROMOD_BOE_VAR_REPO is unset. Untouched by this change; passes in CI where the model extras install.

ruff check src tests is clean (also fixed one pre-existing F541 in cli.py).

Not deployed to Modal — CI handles that on merge.

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
macromod	Ready	Preview, Comment	Jul 18, 2026 6:04pm