The canonical operator ordering on the multispine
Max's architecture directive (2026-07-11): build stages are operators on ONE assembled multispine frame, in this order — not per-source pipelines that each re-implement enrichment:
- Harmonize (per-source, the only per-source layer): map each survey's raw observations into shared harmonized columns with universe metadata (e.g. the six-question disability battery from both ASEC PEDIS and ACS DEAR/DEYE/DREM/DPHY/DDRS/DOUT).
- Assemble the multispine: all survey spines (ASEC, ACS 1-yr, future: ACS 5-yr, SIPP) into one frame with spine tags and mass allocation.
- PUF cloning as a frame operator: fused tax-detail twins for records on EVERY spine, conditioned on the harmonized demographic+income surface — the ACS spine gets direct fusion instead of QRF transfer from the ASEC donor (retires the two-hop imputation for PUF-origin detail).
- Cross-source imputations (SCF wealth, SIPP tips, MEPS, …) once, on the pooled frame.
- Deterministic derivations (is_disabled via the SSI alignment rule over the harmonized battery, structural own_children_in_household, …) once, spine-agnostically.
- Runtime seeds (take-up draws) — already frame-level today.
- Engine materialization / microsimulation for target measures.
- Calibration, then selection/export, with gates at each stage boundary.
Where today deviates
The ASEC×PUF donor pool is built first (steps 3–4 run on the ASEC spine only), and the ACS spine attaches afterward via QRF transfer from that donor (acs_transfer.py). This is the shipped v1 (launch) architecture; it is honest and provenance-tracked, but PUF-origin detail on ACS carries two modeling hops, and per-source derivation risks rule drift (being fixed frame-level for disability/children on the buildl-acs-spine branch).
Scope of this issue
- Migrate PUF cloning from the ASEC base-pool build to a multispine frame operator (design: conditioning surface, clone mass semantics per spine, memory plan — ACS 1-yr doubles to ~2.6M rows).
- Migrate SCF/SIPP/MEPS imputation stages to run post-assembly on the pooled frame.
- Certified-path guard throughout: each migrated stage must prove bit-identical output on the ASEC spine vs the current pipeline before the per-source implementation is retired (same guard pattern as the disability-derivation refactor).
- Sequencing: post-launch; the v1 transfer architecture ships Monday with its provenance register and deferred-inputs list documenting exactly what the reorder improves.
The canonical operator ordering on the multispine
Max's architecture directive (2026-07-11): build stages are operators on ONE assembled multispine frame, in this order — not per-source pipelines that each re-implement enrichment:
Where today deviates
The ASEC×PUF donor pool is built first (steps 3–4 run on the ASEC spine only), and the ACS spine attaches afterward via QRF transfer from that donor (
acs_transfer.py). This is the shipped v1 (launch) architecture; it is honest and provenance-tracked, but PUF-origin detail on ACS carries two modeling hops, and per-source derivation risks rule drift (being fixed frame-level for disability/children on the buildl-acs-spine branch).Scope of this issue