Skip to content

Remove all SSI special handling: random take-up at prior among eligible, one-shot ordering, ordinary targets #469

Description

@MaxGhenis

Directive (Max, 2026-07-21): SSI gets zero special handling in the release pipeline. Categorical eligibility is not an excuse — Medicaid also confers CE, and tax variables carry equally complex interdependencies with no bespoke machinery. The standard program treatment applies: assign take-up randomly in the microdata at the documented prior among the engine-eligible, recipients get the benefit and CE flows through the ordinary dependency order, every column is calculated once, and calibration rips. Whatever residual error the special handling addressed shrinks with pool size — the ACS-spine main dataset culls from a much bigger pool, so machinery justified pre-ACS has no remaining rationale.

Deletion set (release builder; anchors from the current tree)

  1. _reconcile_ssi_take_up_and_refit (tools/build_us_fiscal_refresh_release.py:5452) and every call site — already plan-of-record on Stage the release builder: bound the dense reconcile working set (~0.5TB measured) via cache spill + checkpointed passes #463; this issue widens it.
  2. The greedy count-match take-up assignment machinery: age-band take-up goals rescaled onto ssi_recipients, fresh-pair re-assignment, and the December-SSI candidate-amount materialization pass that feeds it (_ssi_person_uncapped_amount, the "December SSI candidate amounts" batches) — replaced by seeded random draws at the prior rate among engine-eligible units. Prior source, in order of preference: the engine's SSI take-up parameter if one exists; else SSA recipient count over modeled eligible, computed once and recorded in diagnostics with its inputs.
  3. SSI_TAKE_UP_SWAP_SANITY_CAP_RATIO / _DENSE (Dense-arm SSI swap runaway cap at 0.12 × fresh national (#447) #452), the arm-discrimination plumbing (Key the SSI swap cap on the selection arm, not the export-mode flag #455), and pass_history telemetry (Carry the SSI reconcile per-pass trajectory through raise and record (#447) #448) — instrumentation of a loop that no longer exists.
  4. The SSI-conditional "dependency replay" structure — with assign-once ordering (flags frozen → dependents materialized once → calibrate), replay is just ordinary ordering; no SSI-keyed branch remains.
  5. Any remaining ssi-named special case in the release path the implementer finds by exhaustive grep — enumerate in the PR; the bar is that ssi appears in the builder only where any other program's name could appear.

What stays

Acceptance

Supersedes the conditional-columns fallback on #463 for SSI specifically (that pattern remains documented for any future program where a data↔solution coupling genuinely survives this treatment — none known today). #463 owns the implementation alongside the staging work; #449 sequences it into N.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions