Doctrine: the scale ladder — dev-scale runs before production spend ("boring before billed")

[MaxGhenis]

Chartered from the run 5–7 retrospective (#578). Total cost of the missing middle tier: ~$350 of Modal spend and ~5 days, with every major lesson (the PERIDNUM export bug, the 58-target silent skip, preemption/timeout economics) discoverable at 10% scale locally in under an hour.

The standing rules for every data pipeline in this repo:

1. Scale ladder from day one. An identity-bound sample_fraction (uniform across survey spines, composition-preserving; donors always full) with three rungs: fixture (seconds — code paths), dev-scale real data run locally (sub-hour — mechanism + integration, including the export/publication tail), full scale (distributions + shipping). A pipeline without a sub-hour full-path rung is not chartered for cloud spend.
2. Boring before billed. No full-scale dispatch until N consecutive dev-scale runs complete end-to-end clean — gates green or receipted. The first integration run's purpose is to reach the last phase; bugs concentrate in the least-exercised tail.
3. Fixtures from real-data samples. Test fixtures preserve real pathology (nulls, dtypes, string encodings) — the week's misses were pathology-shaped, not logic-shaped.
4. Gates at phase boundaries, not only terminal (the pre-simulation completeness gate is the pattern).
5. Stated P(pass) before every gate, logged and resolved (_buildo-runtime/predictions/). Calibrated high dev-scale P(pass) is the productionization signal.

Antecedent: this is the encode-ladder discipline (fixture → single case → battery → full conformance) ported to data builds.

First consumer: the #578 stacked-spine adoption lane (single sample_fraction, 10% local builds, clone_attachment_fraction as a production-only memory lever).

🤖 Generated with Claude Code