You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
PolicyBench draws its households from the certified US build and scores frontier models against PolicyEngine references computed from exactly the inputs the build carries. On 2026-09-05 the 39-model board had to exclude eleven outputs from scoring because their references depend on inputs the June 2026 build (populace-us-2024-5da5a95-20260611) never carried, so the prompt could not list them (PolicyEngine/policybench#164, reference_exclusions.json). The v2 household draw needs a build that carries them.
What the June build lacks
Input
State in the June build
Consequence
meets_ssi_disability_criteria
false for all 160,858 persons
SSI payable only through age or blindness; six outputs excluded
months_receiving_social_security_disability
absent in every build to date
Medicare's under-65 route unreachable; five outputs excluded
is_permanently_and_totally_disabled
never populated
IRC 152 disabled-dependent age exemption and the IRC 22 credit unreachable
is_incapable_of_self_care
absent until the 2026-07-23 builds (2,067 from PEDISDRS)
IRC 21 CDCC disabled-dependent route
Builds from 2026-07-17 carry the SIPP-imputed SSI criterion (7,275 to 7,515 persons). #361 documents the sparse export dropping 94 no-formula inputs, including is_disabled, is_blind, employer_sponsored_insurance_premiums and the asset columns; the benchmark build must carry every no-formula input the engine reads.
Populate is_permanently_and_totally_disabled (impute from SSI/SSDI receipt or the SIPP donor) or document that the tax code's disability pathways are out of scope.
Carry SSDI entitlement duration or start date (months_receiving_social_security_disability) with a defined reference date, if the SIPP donor can supply it; otherwise record the assumption explicitly so the benchmark can state it.
Record provenance per disability fact (observed, imputed, unknown). An unknown must not export as an observed negative.
Keep person_marital_unit_id in the export; the benchmark now builds marital units itself (head and spouse) after finding policyengine-core lumps a household into one unit when none is given.
Summary
PolicyBench draws its households from the certified US build and scores frontier models against PolicyEngine references computed from exactly the inputs the build carries. On 2026-09-05 the 39-model board had to exclude eleven outputs from scoring because their references depend on inputs the June 2026 build (
populace-us-2024-5da5a95-20260611) never carried, so the prompt could not list them (PolicyEngine/policybench#164,reference_exclusions.json). The v2 household draw needs a build that carries them.What the June build lacks
meets_ssi_disability_criteriamonths_receiving_social_security_disabilityis_permanently_and_totally_disabledis_incapable_of_self_careBuilds from 2026-07-17 carry the SIPP-imputed SSI criterion (7,275 to 7,515 persons). #361 documents the sparse export dropping 94 no-formula inputs, including
is_disabled,is_blind,employer_sponsored_insurance_premiumsand the asset columns; the benchmark build must carry every no-formula input the engine reads.Asks
is_incapable_of_self_care.is_permanently_and_totally_disabled(impute from SSI/SSDI receipt or the SIPP donor) or document that the tax code's disability pathways are out of scope.months_receiving_social_security_disability) with a defined reference date, if the SIPP donor can supply it; otherwise record the assumption explicitly so the benchmark can state it.person_marital_unit_idin the export; the benchmark now builds marital units itself (head and spouse) after finding policyengine-core lumps a household into one unit when none is given.Related: #424, #453 (SSI candidate undercount, child SSI absent), PolicyEngine/policyengine-us#9330 (disability input wiring audit), PolicyEngine/policybench#139 (v2 board).