Fix SyntheticDiD catastrophic cancellation at extreme Y scale

[igerber]

Summary

• Internally centers and scales Y once in SyntheticDiD.fit() using mean/std of Y_pre_control; threads normalized Y and normalized zetas through weight optimization, the estimator, and all three variance procedures; then rescales τ/SE/CI/effect vectors back to original outcome units.
• Mathematically a no-op (τ is location-invariant and scale-equivariant; Frank-Wolfe argmin is invariant under (Y, ζ) → (Y/s, ζ/s)), but prevents ~6-digit precision loss in the SDID double-difference when outcomes span millions-to-billions. Pre-fix, p-values clustered near 0.5 regardless of true effect size. Matches pre-existing results on normally-scaled data to float epsilon (rel ≲ 1e-14).
• zeta_omega, zeta_lambda, and noise_level on the results object are reported on the user's original outcome scale, preserving the behavior of in_time_placebo() and sensitivity_to_zeta_omega() which operate on the stored original-scale fit snapshot.

Methodology references (required if estimator / math changes)

• Method: Synthetic Difference-in-Differences (Arkhangelsky et al., 2021) — Algorithms 3 (jackknife) and 4 (placebo), plus the bootstrap path matching synthdid::vcov.
• Intentional deviation: internal Y normalization is not performed by R's synthdid; the R package has the same extreme-scale vulnerability (Sensitivity to the scaling of variables synth-inference/synthdid#71). Documented in docs/methodology/REGISTRY.md under the SyntheticDiD edge-case list with a **Note:** label.
• Scope limitation: in_time_placebo() and sensitivity_to_zeta_omega() continue to run on original-scale snapshot and zetas (pre-fix behavior preserved). Documented in the same REGISTRY note as a tracked follow-up.

Validation

• Tests added in tests/test_methodology_sdid.py::TestScaleEquivariance:
  • test_baseline_parity_small_scale[placebo|bootstrap|jackknife] — hard-coded literal baselines asserted at rel=1e-8 to catch any drift from the no-op guarantee.
  • test_scale_equivariance[placebo|bootstrap|jackknife] — refits on a*Y + b for (a, b) ∈ {(1e-6,0), (1,0), (1e6,1e9), (1e9,-1e6), (1,1e9)} and asserts len(placebo_effects) parity, att/a, se/|a|, p_value, and noise_level/zeta_omega scaling.
  • test_detects_true_effect_at_extreme_scale[...] — generates a panel with Y ~ 1e9 and known ATT = 5e6 (0.5% of baseline, below pre-fix 6-digit precision floor); asserts detectable z-stat and correct p-value.
• All 116 pre-existing SDID methodology tests pass unchanged — no tolerance relaxation.
• 156 SDID-related tests across methodology, survey phases 5/6, SE accuracy, and power all green; 203 additional estimator/alias/visualization tests also green.

Security / privacy

• Confirm no secrets/PII in this PR: Yes

[github-actions]

Overall Assessment

✅ Looks good

Executive Summary

• Affected methodology: SyntheticDiD.fit() plus all three variance paths (bootstrap, jackknife, placebo).
• The internal Y normalization is explicitly documented as a deviation in docs/methodology/REGISTRY.md#L1518, so it is informational rather than a defect.
• I did not find an unmitigated mismatch with the SDID method mapping: fixed-weight jackknife is preserved, placebo still re-estimates omega/lambda, and user-facing ATT/SE/CI/effect vectors are rescaled back to original units.
• The unchanged original-scale behavior of in_time_placebo() and sensitivity_to_zeta_omega() is also documented, so it is not a blocker.
• I could not execute the tests locally because this environment does not have numpy or pytest; the review below is based on static inspection.

Methodology

• P3 Informational: The PR changes the SDID estimation path in diff_diff/synthetic_did.py#L419 and threads normalized Y/zeta_* through unit-weight optimization, time-weight optimization, ATT computation, and all three variance procedures before rescaling outputs at diff_diff/synthetic_did.py#L517, diff_diff/synthetic_did.py#L571, and diff_diff/synthetic_did.py#L688. Cross-checking against the SDID docstring references and the registry, I do not see an undocumented methodology deviation, incorrect variance/SE formula, or missing assumption check. Concrete fix: none.
• P3 Informational: Algorithm 3/4 behavior is still intact. The jackknife path remains fixed-weight at diff_diff/synthetic_did.py#L590, and the placebo path still re-estimates both omega and lambda at diff_diff/synthetic_did.py#L605. Impact: no P0/P1 methodology issue found. Concrete fix: none.
• P3 Informational: in_time_placebo() and sensitivity_to_zeta_omega() intentionally continue to use the original-scale fit snapshot in diff_diff/results.py#L1136 and diff_diff/results.py#L1313, matching the documented scope note in docs/methodology/REGISTRY.md#L1518. Impact: those diagnostics do not yet gain the new stabilization, but the limitation is documented and therefore non-blocking. Concrete fix: none required for approval.

Code Quality

• No findings. The diff still routes inference through safe_inference at diff_diff/synthetic_did.py#L624, so it does not introduce the inline-inference or partial-NaN anti-patterns.

Performance

• No findings. The temporary normalized matrices created at diff_diff/synthetic_did.py#L439 increase peak memory during fit(), but I do not see a material algorithmic regression in scope.

Maintainability

• No findings. The implementation comment at diff_diff/synthetic_did.py#L419 and the registry note at docs/methodology/REGISTRY.md#L1518 make the deviation and its intended scope easy to audit.

Tech Debt

• No findings. I did not identify new untracked deferred work in the changed code.

Security

• No findings. The diff adds no new security surface, and I did not see secrets or PII in the changed files.

Documentation/Tests

• P3 Informational: The added tests in tests/test_methodology_sdid.py#L2148 cover baseline parity, scale equivariance, and extreme-scale detectability across placebo/bootstrap/jackknife, which is the right test surface for this change. Concrete fix: none.
• Reviewer note: test execution was not possible here because numpy and pytest are unavailable in the current environment.