diff --git a/CHANGELOG.md b/CHANGELOG.md index 2c413ba8..990e8759 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] ### Added +- **`ChaisemartinDHaultfoeuille.by_path` and `paths_of_interest` now compose with `heterogeneity=""`** (Web Appendix Section 1.5, Lemma 7). Per-path heterogeneity coefficient is computed by re-running the Lemma 7 regression on each path-restricted switcher subsample. The path filter (`path_groups: Optional[Set[int]]`) restricts eligibility to switchers ON path `p` inside the inner regression; the variance machinery (standard WLS vcov for non-survey, cell-period IF allocator for Binder TSL, group-level allocator for Rao-Wu replicate) is unchanged from the global heterogeneity path. Cohort dummies in the design matrix absorb baseline by construction, so multi-baseline switcher panels do not produce R-divergence (no parallel `UserWarning` like `controls` / `trends_linear`). Surfaces on `results.path_heterogeneity_effects` keyed `{path: {l: {beta, se, t_stat, p_value, conf_int, n_obs}}}` and on `results.to_dataframe(level="by_path")` via new always-present `het_*` columns (`het_beta`, `het_se`, `het_t_stat`, `het_p_value`, `het_conf_int_lower`, `het_conf_int_upper`), populated for positive-horizon rows when `heterogeneity` is set and NaN otherwise (mirrors the `cband_*` and `cumulated_*` always-present convention). Composes with `survey_design` (analytical Binder TSL + replicate-weight bootstrap) via the existing PR #408 IF allocator path; under replicate weights, every per-(path, horizon) fit appends `n_valid` to the shared `_replicate_n_valid_list` accumulator and the final `_effective_df_survey` recomputation reflects all per-path appends. R parity verified against `did_multiplegt_dyn(..., by_path=3, predict_het=list("het_x", c(1,2,3)))` on the new `multi_path_reversible_by_path_predict_het` golden-value scenario; a sibling global anchor `multi_path_reversible_predict_het` introduces the FIRST `predict_het` R-parity baseline in the repo (no prior `TestDCDHDynRParityHeterogeneity` existed). Both R calls use `dont_drop_larger_lower=TRUE` to match the Python `drop_larger_lower=False` requirement and to provide cohort variation at every horizon under reversal paths. Per-path SE matches global SE bit-exactly on a single-path panel (telescope invariant, `atol=rtol=1e-14`). Multiplier bootstrap (`n_bootstrap > 0`) under `by_path + heterogeneity + survey_design` inherits the existing per-path multiplier-bootstrap-survey gate from PR #408. The `NotImplementedError` gate at `chaisemartin_dhaultfoeuille.py:1230-1234` is removed; `heterogeneity` precondition mutex with `controls` / `trends_linear` / `trends_nonparam` stays in place. Cross-surface invariants regression-tested at `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathHeterogeneity` (~13 tests across gate dispatch, behavior, single-path telescope, zero-signal anti-regression, multi-baseline UserWarning anti-regression, DataFrame integration, edge cases) + `tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityHeterogeneity` (global anchor) + `::TestDCDHDynRParityByPathHeterogeneity` (per-path). See `docs/methodology/REGISTRY.md` §`ChaisemartinDHaultfoeuille` `Note (Phase 3 by_path ...)` → "Per-path heterogeneity testing" for the full contract. - **Tutorial 21: HAD Pre-test Workflow** (`docs/tutorials/21_had_pretest_workflow.ipynb`) — composite pre-test walkthrough for `HeterogeneousAdoptionDiD` building on Tutorial 20's brand-campaign framing. Uses a 60-DMA × 8-week panel close in shape to T20's but with the dose distribution drawn from `Uniform[$0.01K, $50K]` (vs T20's `[$5K, $50K]`); the true support is strictly positive but very near zero, chosen so the QUG step in `did_had_pretest_workflow` fails-to-reject `H0: d_lower = 0` in this finite sample and the verdict text fires the load-bearing "Assumption 7 deferred" pivot for the upgrade-arc narrative. (HAD's `design="auto"` selector — a separate min/median heuristic at `had.py::_detect_design`, NOT the QUG p-value — independently lands on the `continuous_at_zero` identification path with target `WAS` on this panel because `d.min() < 0.01 * median(|d|)`. The QUG test and the design selector are independent rules that point to the same identification path here.) Walks through three surfaces: (a) `did_had_pretest_workflow(aggregate="overall")` on a two-period collapse, where the verdict explicitly flags Step 2 (Assumption 7 pre-trends) as not run because a single pre-period structurally cannot support a pre-trends test, and the structural fields `pretrends_joint` / `homogeneity_joint` are both `None`; (b) `did_had_pretest_workflow(aggregate="event_study")` on the full multi-period panel, where the verdict reads "TWFE admissible under Section 4 assumptions" because all three testable diagnostics (QUG + joint pre-trends Stute over 3 horizons + joint homogeneity Stute over 4 horizons) fail-to-reject — non-rejection evidence under finite-sample power and test specification, not proof that the identifying assumptions hold; and (c) a side panel exercising both `yatchew_hr_test` null modes — `null="linearity"` (default, paper Theorem 7) vs `null="mean_independence"` (Phase 4 R-parity with R `YatchewTest::yatchew_test(order=0)`) — on the within-pre-period first-difference paired with post-period dose, illustrating the stricter null's larger residual variance (`sigma2_lin` 7.01 vs 6.53) and smaller p-value (0.29 vs 0.49). Companion drift-test file `tests/test_t21_had_pretest_workflow_drift.py` (16 tests pinning panel composition, both verdict pivots, structural anchors on both paths, deterministic QUG / Yatchew statistics, bootstrap p-value tolerance bands per `feedback_bootstrap_drift_tests_need_backend_tolerance`, and `HAD(design="auto")` resolution to `continuous_at_zero` on this panel). T20's "Composite pretest workflow" Extensions bullet updated with a forward-pointer to T21. T22 weighted/survey HAD tutorial remains queued as a separate notebook PR. - **`ChaisemartinDHaultfoeuille.by_path` and `paths_of_interest` now compose with `survey_design`** for analytical Binder TSL SE and replicate-weight bootstrap variance. The `NotImplementedError` gate at `chaisemartin_dhaultfoeuille.py:1233-1239` is replaced by a per-path multiplier-bootstrap-only gate (`survey_design + n_bootstrap > 0` under by_path / paths_of_interest still raises, since the survey-aware perturbation pivot for path-restricted IFs is methodologically underived). Per-path SE routes through the existing `_survey_se_from_group_if` cell-period allocator: the per-period IF (`U_pp_l_path`) is built with non-path switcher-side contributions skipped (control contributions are unchanged, matching the joiners/leavers IF convention; preserves the row-sum identity `U_pp.sum(axis=1) == U`), cohort-recentered via `_cohort_recenter_per_period`, then expanded to observations as `psi_i = U_pp[g_i, t_i] · (w_i / W_{g_i, t_i})`. Replicate-weight designs unconditionally use the cell allocator (Class A contract from PR #323). New `_refresh_path_inference` helper post-call refreshes `safe_inference` on every populated entry across `multi_horizon_inference`, `placebo_horizon_inference`, `path_effects`, and `path_placebos` so all four surfaces use the same final `df_survey` after per-path replicate fits append `n_valid` to the shared accumulator. Path-enumeration ranking under `survey_design` remains unweighted (group-cardinality, not population-weight mass). Lonely-PSU policy stays sample-wide, not per-path. Telescope invariant: on a single-path panel, per-path SE matches the global non-by_path survey SE bit-exactly. **No R parity** — R `did_multiplegt_dyn` does not support survey weighting; this is a Python-only methodology extension. The global non-by_path TSL multiplier-bootstrap path is unaffected (anti-regression test `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathSurveyDesignAnalytical::test_global_survey_plus_n_bootstrap_still_works` locks the per-path-only scope of the new gate). Cross-surface invariants regression-tested at `TestByPathSurveyDesignAnalytical` (~17 tests across gate / dispatch / analytical SE / replicate-weight SE / per-path placebos / `trends_linear` composition / unobserved-path warnings / final-df refresh regressions) and `TestByPathSurveyDesignTelescope`. See `docs/methodology/REGISTRY.md` §`ChaisemartinDHaultfoeuille` `Note (Phase 3 by_path ...)` → "Per-path survey-design SE" for the full contract. - **Inference-field aliases on staggered result classes** for adapter / external-consumer compatibility. Read-only `@property` aliases expose the flat `att` / `se` / `conf_int` / `p_value` / `t_stat` names (matching `DiDResults` / `TROPResults` / `SyntheticDiDResults` / `HeterogeneousAdoptionDiDResults`) on every result class that previously only carried prefixed canonical fields: `CallawaySantAnnaResults`, `StackedDiDResults`, `EfficientDiDResults`, `ChaisemartinDHaultfoeuilleResults`, `StaggeredTripleDiffResults`, `WooldridgeDiDResults`, `SunAbrahamResults`, `ImputationDiDResults`, `TwoStageDiDResults` (mapping to `overall_*`); `ContinuousDiDResults` (mapping to `overall_att_*`, ATT-side as the headline, ACRT-side accessible unchanged via `overall_acrt_*`); `MultiPeriodDiDResults` (mapping to `avg_*`). `ContinuousDiDResults` additionally exposes `overall_se` / `overall_conf_int` / `overall_p_value` / `overall_t_stat` aliases for naming consistency with the rest of the staggered family. Aliases are pure read-throughs over the canonical fields — no recomputation, no behavior change — so the `safe_inference()` joint-NaN contract (per CLAUDE.md "Inference computation") is inherited automatically (NaN canonical → NaN alias, locked at `tests/test_result_aliases.py::test_pattern_b_aliases_propagate_nan`). The native `overall_*` / `overall_att_*` / `avg_*` fields remain canonical for documentation and computation. Motivated by the `balance.interop.diff_diff.as_balance_diagnostic()` adapter (`facebookresearch/balance` PR #465) which calls `getattr(res, "se", None)` / `getattr(res, "conf_int", None)` without a fallback chain — pre-alias, every staggered result class returned `None` on those keys, silently dropping `se` and `conf_int` from the adapter's diagnostic dict. 23 alias-mechanic + balance-adapter regression tests at `tests/test_result_aliases.py`. Patch-level (additive on stable surfaces). diff --git a/benchmarks/R/generate_dcdh_dynr_test_values.R b/benchmarks/R/generate_dcdh_dynr_test_values.R index e6119720..a5caa5ee 100644 --- a/benchmarks/R/generate_dcdh_dynr_test_values.R +++ b/benchmarks/R/generate_dcdh_dynr_test_values.R @@ -618,6 +618,61 @@ extract_dcdh_by_path <- function(res, n_effects, n_placebos = 0) { list(by_path = out) } +# Helper: extract global predict_het results. R's predict_het slot is at +# res$results$predict_het, a data.frame with columns +# {effect, covariate, Estimate, SE, t, LB, UB, N, pF}. Estimate is the +# WLS coefficient on the heterogeneity covariate. +extract_dcdh_predict_het <- function(res, n_effects) { + ph <- res$results$predict_het + horizons <- list() + if (is.null(ph) || nrow(ph) == 0) return(list(predict_het = horizons)) + for (h in seq_len(min(n_effects, nrow(ph)))) { + horizons[[as.character(ph$effect[h])]] <- list( + beta = as.numeric(ph$Estimate[h]), + se = as.numeric(ph$SE[h]), + t = as.numeric(ph$t[h]), + ci_lo = as.numeric(ph$LB[h]), + ci_hi = as.numeric(ph$UB[h]), + n_obs = as.numeric(ph$N[h]), + p_value = as.numeric(ph$pF[h]) + ) + } + list(predict_het = horizons) +} + +# Helper: extract per-path predict_het results. Under by_path=k + +# predict_het, R's per-by_level dispatcher writes a predict_het table to +# each res$by_level_i$results$predict_het. Output mirrors +# extract_dcdh_by_path's shape with a horizons dict keyed by horizon. +extract_dcdh_by_path_predict_het <- function(res, n_effects) { + by_levels <- res$by_levels + out <- list() + for (i in seq_along(by_levels)) { + slot <- res[[paste0("by_level_", i)]] + ph <- slot$results$predict_het + horizons <- list() + if (!is.null(ph) && nrow(ph) > 0) { + for (h in seq_len(min(n_effects, nrow(ph)))) { + horizons[[as.character(ph$effect[h])]] <- list( + beta = as.numeric(ph$Estimate[h]), + se = as.numeric(ph$SE[h]), + t = as.numeric(ph$t[h]), + ci_lo = as.numeric(ph$LB[h]), + ci_hi = as.numeric(ph$UB[h]), + n_obs = as.numeric(ph$N[h]), + p_value = as.numeric(ph$pF[h]) + ) + } + } + out[[i]] <- list( + path = by_levels[i], + frequency_rank = i, + horizons = horizons + ) + } + list(by_path_predict_het = out) +} + # Scenario 13: mixed_single_switch + by_path=2 (basic 2-path case). # The mixed_single_switch DGP produces joiners (path 0,1,1,1) and # leavers (path 1,0,0,0) as its only two observed paths at L_max=3, so @@ -1018,6 +1073,136 @@ cat(" Scenario 19: multi_path_reversible_by_path_non_binary\n") ) } +# Scenarios 20 + 21: predict_het R-parity (Wave 5 #11). Scenario 20 is +# the global anchor (predict_het without by_path); scenario 21 is the +# per-path version (by_path + predict_het). Both use the same DGP so the +# Python-side parity tests can calibrate atol on the global scenario and +# inherit the same atol for the per-path test. R's predict_het syntax +# requires a 2-element list: list(covariate_name, horizons_vec). +# +# DGP shape: 90 switchers + 30 never-treated controls, 10 periods, 3 +# paths (0,1,1,1) / (0,1,0,0) / (0,1,1,0), F_g varies in {3,4,5} +# INDEPENDENTLY of path so each path has multiple cohorts. het_x is +# binary {0,1}, balanced across both switchers and controls. Effect = +# 5.0 + 3.0 * het_x to produce a detectable heterogeneity signal. +# Never-treated controls are required for R's predict_het to compute +# horizons l>=2 under reversal paths (otherwise R returns +# "max effects = 2" or empty cohort dummies). `dont_drop_larger_lower +# = TRUE` matches the Python by_path requirement that +# `drop_larger_lower=False`. +cat(" Scenarios 20/21: multi_path_reversible_predict_het + by_path version\n") +{ + set.seed(120L) + n_switchers20 <- 90L + n_controls20 <- 30L + n_groups20 <- n_switchers20 + n_controls20 + n_periods20 <- 10L + paths20 <- list(c(0L, 1L, 1L, 1L), c(0L, 1L, 0L, 0L), c(0L, 1L, 1L, 0L)) + D20 <- matrix(0L, nrow = n_groups20, ncol = n_periods20) + het_x20 <- integer(n_groups20) + group_fe20 <- rnorm(n_groups20, 0, 2.0) + # Switchers (groups 1..n_switchers20) + for (g in seq_len(n_switchers20)) { + F_g_choice <- ((g - 1L) %/% 3L) %% 3L + F_g <- 3L + F_g_choice + path_idx <- ((g - 1L) %% 3L) + 1L + pp <- paths20[[path_idx]] + het_x20[g] <- if (g <= n_switchers20 %/% 2L) 1L else 0L + for (j in seq_along(pp)) { + t <- F_g - 1L + j + if (t >= 1L && t <= n_periods20) D20[g, t] <- pp[j] + } + if (F_g - 1L + length(pp) <= n_periods20) { + tail_t <- (F_g - 1L + length(pp)):n_periods20 + D20[g, tail_t] <- pp[length(pp)] + } + } + # Never-treated controls (groups n_switchers20+1..n_groups20). + # Balanced het_x assignment so the heterogeneity covariate has + # variation in the control pool too. + for (g in (n_switchers20 + 1L):n_groups20) { + k <- g - n_switchers20 + het_x20[g] <- if (k <= n_controls20 %/% 2L) 1L else 0L + } + noise20 <- matrix(rnorm(n_groups20 * n_periods20, 0, 0.5), + nrow = n_groups20, ncol = n_periods20) + period_arr20 <- 0:(n_periods20 - 1L) + effect20 <- 5.0 + 3.0 * het_x20 # heterogeneity signal + Y20 <- matrix(group_fe20, nrow = n_groups20, ncol = n_periods20) + + matrix(0.5 * period_arr20, nrow = n_groups20, ncol = n_periods20, byrow = TRUE) + + matrix(effect20, nrow = n_groups20, ncol = n_periods20) * D20 + + noise20 + d20 <- data.frame( + group = rep(seq_len(n_groups20) - 1L, each = n_periods20), + period = rep(period_arr20, n_groups20), + treatment = as.vector(t(D20)), + outcome = as.vector(t(Y20)), + het_x = rep(het_x20, each = n_periods20) + ) + + # Scenario 20: global predict_het anchor (no by_path). + # `dont_drop_larger_lower = TRUE` preserves multi-switch cohorts so the + # heterogeneity regression has cohort variation at every horizon. + # Without this, R drops off-switch paths at l>=2, leaving a single + # cohort and triggering the `prod_het_l_XX ~ het_x + ` empty-cohort + # error. dCDH `by_path` requires `drop_larger_lower=False` on the Python + # side anyway, so this flag is consistent with the per-path scope. + res20 <- did_multiplegt_dyn( + df = d20, outcome = "outcome", group = "group", time = "period", + treatment = "treatment", effects = 3, + dont_drop_larger_lower = TRUE, + predict_het = list("het_x", c(1, 2, 3)), + ci_level = 95, graph_off = TRUE + ) + scenarios$multi_path_reversible_predict_het <- list( + data = list( + group = as.numeric(d20$group), + period = as.numeric(d20$period), + treatment = as.numeric(d20$treatment), + outcome = as.numeric(d20$outcome), + het_x = as.numeric(d20$het_x) + ), + params = list(pattern = "multi_path_reversible_predict_het", + n_switchers = n_switchers20, n_controls = n_controls20, + n_groups = n_groups20, n_periods = n_periods20, + seed = 120L, effects = 3, + predict_het_var = "het_x", + predict_het_horizons = c(1, 2, 3), + ci_level = 95, + dont_drop_larger_lower = TRUE), + results = extract_dcdh_predict_het(res20, n_effects = 3) + ) + + # Scenario 21: by_path + predict_het (per-path version on same DGP). + # `dont_drop_larger_lower = TRUE` matches scenario 20 + Python + # by_path's `drop_larger_lower=False` requirement. + res21 <- did_multiplegt_dyn( + df = d20, outcome = "outcome", group = "group", time = "period", + treatment = "treatment", effects = 3, by_path = 3, + dont_drop_larger_lower = TRUE, + predict_het = list("het_x", c(1, 2, 3)), + ci_level = 95, graph_off = TRUE + ) + scenarios$multi_path_reversible_by_path_predict_het <- list( + data = list( + group = as.numeric(d20$group), + period = as.numeric(d20$period), + treatment = as.numeric(d20$treatment), + outcome = as.numeric(d20$outcome), + het_x = as.numeric(d20$het_x) + ), + params = list(pattern = "multi_path_reversible_by_path_predict_het", + n_switchers = n_switchers20, n_controls = n_controls20, + n_groups = n_groups20, n_periods = n_periods20, + seed = 120L, effects = 3, by_path = 3, + predict_het_var = "het_x", + predict_het_horizons = c(1, 2, 3), + ci_level = 95, + dont_drop_larger_lower = TRUE), + results = extract_dcdh_by_path_predict_het(res21, n_effects = 3) + ) +} + # --------------------------------------------------------------------------- # Write output # --------------------------------------------------------------------------- diff --git a/benchmarks/data/dcdh_dynr_golden_values.json b/benchmarks/data/dcdh_dynr_golden_values.json index 38b6a847..fe4e6f44 100644 --- a/benchmarks/data/dcdh_dynr_golden_values.json +++ b/benchmarks/data/dcdh_dynr_golden_values.json @@ -1437,6 +1437,185 @@ } ] } + }, + "multi_path_reversible_predict_het": { + "data": { + "group": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 37, 37, 37, 37, 37, 37, 37, 37, 37, 37, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 39, 39, 39, 39, 39, 39, 39, 39, 39, 39, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 43, 43, 43, 43, 43, 43, 43, 43, 43, 43, 44, 44, 44, 44, 44, 44, 44, 44, 44, 44, 45, 45, 45, 45, 45, 45, 45, 45, 45, 45, 46, 46, 46, 46, 46, 46, 46, 46, 46, 46, 47, 47, 47, 47, 47, 47, 47, 47, 47, 47, 48, 48, 48, 48, 48, 48, 48, 48, 48, 48, 49, 49, 49, 49, 49, 49, 49, 49, 49, 49, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 51, 51, 51, 51, 51, 51, 51, 51, 51, 51, 52, 52, 52, 52, 52, 52, 52, 52, 52, 52, 53, 53, 53, 53, 53, 53, 53, 53, 53, 53, 54, 54, 54, 54, 54, 54, 54, 54, 54, 54, 55, 55, 55, 55, 55, 55, 55, 55, 55, 55, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 57, 57, 57, 57, 57, 57, 57, 57, 57, 57, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 61, 61, 61, 61, 61, 61, 61, 61, 61, 61, 62, 62, 62, 62, 62, 62, 62, 62, 62, 62, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 66, 66, 66, 66, 66, 66, 66, 66, 66, 66, 67, 67, 67, 67, 67, 67, 67, 67, 67, 67, 68, 68, 68, 68, 68, 68, 68, 68, 68, 68, 69, 69, 69, 69, 69, 69, 69, 69, 69, 69, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 71, 71, 71, 71, 71, 71, 71, 71, 71, 71, 72, 72, 72, 72, 72, 72, 72, 72, 72, 72, 73, 73, 73, 73, 73, 73, 73, 73, 73, 73, 74, 74, 74, 74, 74, 74, 74, 74, 74, 74, 75, 75, 75, 75, 75, 75, 75, 75, 75, 75, 76, 76, 76, 76, 76, 76, 76, 76, 76, 76, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 78, 78, 78, 78, 78, 78, 78, 78, 78, 78, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 82, 82, 82, 82, 82, 82, 82, 82, 82, 82, 83, 83, 83, 83, 83, 83, 83, 83, 83, 83, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 85, 85, 85, 85, 85, 85, 85, 85, 85, 85, 86, 86, 86, 86, 86, 86, 86, 86, 86, 86, 87, 87, 87, 87, 87, 87, 87, 87, 87, 87, 88, 88, 88, 88, 88, 88, 88, 88, 88, 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 90, 90, 90, 90, 90, 90, 90, 90, 90, 90, 91, 91, 91, 91, 91, 91, 91, 91, 91, 91, 92, 92, 92, 92, 92, 92, 92, 92, 92, 92, 93, 93, 93, 93, 93, 93, 93, 93, 93, 93, 94, 94, 94, 94, 94, 94, 94, 94, 94, 94, 95, 95, 95, 95, 95, 95, 95, 95, 95, 95, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 97, 97, 97, 97, 97, 97, 97, 97, 97, 97, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 101, 101, 101, 101, 101, 101, 101, 101, 101, 101, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 106, 106, 106, 106, 106, 106, 106, 106, 106, 106, 107, 107, 107, 107, 107, 107, 107, 107, 107, 107, 108, 108, 108, 108, 108, 108, 108, 108, 108, 108, 109, 109, 109, 109, 109, 109, 109, 109, 109, 109, 110, 110, 110, 110, 110, 110, 110, 110, 110, 110, 111, 111, 111, 111, 111, 111, 111, 111, 111, 111, 112, 112, 112, 112, 112, 112, 112, 112, 112, 112, 113, 113, 113, 113, 113, 113, 113, 113, 113, 113, 114, 114, 114, 114, 114, 114, 114, 114, 114, 114, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 116, 116, 116, 116, 116, 116, 116, 116, 116, 116, 117, 117, 117, 117, 117, 117, 117, 117, 117, 117, 118, 118, 118, 118, 118, 118, 118, 118, 118, 118, 119, 119, 119, 119, 119, 119, 119, 119, 119, 119], + "period": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9], + "treatment": [0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], + "outcome": [-0.38947445306, -0.23568361568, 0.21162698298, 9.4874432555, 9.5272086491, 9.8732152644, 10.4966606852, 10.7269218628, 11.3041347822, 12.3075808547, -0.53540224144, -0.85896484972, 0.9039919656, 8.4216226165, 0.32926420824, 1.5187015075, 2.2466260038, 2.3752794078, 2.5489603023, 3.8839539563, -1.6531086427, -0.48172021176, -1.1588810115, 7.5449719691, 8.2423444411, 0.16054588328, 1.34512733, 2.3267248703, 2.907302291, 2.7362604935, 3.9790870637, 5.1832195997, 4.9765720574, 5.9562138217, 14.7385510317, 14.1883967913, 14.9982098406, 16.5614484131, 16.3735973947, 17.2722352384, 2.4672139194, 3.181736844, 2.8966044002, 3.8788938746, 12.0094366558, 4.5284429927, 4.7586747034, 6.3688938717, 6.8655832555, 6.6200148023, 1.366346809, 2.2681542407, 1.0986738641, 1.3627616225, 10.60127055, 10.2334877677, 3.7661446414, 2.8049023737, 5.3088063004, 5.7352273543, -2.1462960524, -1.562262737, -0.68045628748, -0.39170645702, 0.51893991954, 8.2908071534, 9.041118565, 9.1952101148, 10.5230194116, 10.4974242438, 3.8654236028, 3.9905881505, 4.2598845829, 5.4183992861, 5.9314932187, 13.2206109279, 5.3935881935, 7.4081365763, 7.5828582314, 7.359520808, 0.32944110732, 1.3451564511, 0.47252530565, 1.86498985, 2.4477442051, 10.7154679806, 10.8414260484, 3.9189896786, 4.2352551524, 4.3157247407, 1.3062082738, 2.2840554655, 2.5999388791, 11.2750783874, 11.7106216584, 11.6742402808, 13.1041698079, 13.2031556633, 13.5015002952, 14.1776065603, 0.11158785236, 1.054725582, 2.5473237287, 9.9685092, 2.5553042659, 3.1672544975, 3.2044163382, 3.279807654, 4.9738691668, 6.0325408926, -0.080780006181, 2.1294224371, 1.8617613349, 9.4372859439, 10.6907812039, 3.0731760034, 3.9983448664, 3.9479302438, 4.5770928564, 4.759461957, 1.2233010078, 1.5007797084, 1.8169053771, 3.0011971575, 11.1825713053, 12.4024767365, 12.1157117449, 12.2510645066, 13.4088151694, 13.9401105065, -2.7956639463, -1.9178517996, -1.8381646645, -1.0116116427, 7.2114118991, -0.47283326103, -0.2375849953, -0.21983407988, 0.83491171322, 1.7623326366, -2.4904909619, -1.1251647276, -0.0021674062124, 0.20812688743, 9.9195669513, 9.5480968853, 2.3956781408, 2.1183487474, 3.0646069867, 3.808704102, -2.0779624805, -1.9780684504, -0.33525118838, -0.057425341904, 1.0012356804, 8.1271505508, 8.7896794462, 8.8251678355, 10.1813290206, 10.1022892304, 3.6502716683, 3.7201719177, 3.7674199724, 5.1980443964, 5.7139097866, 14.6401937191, 7.0192780984, 7.0612922863, 7.5443921558, 8.7556031236, -0.26132650677, -0.50936928911, 0.99992467827, 0.80556665801, 1.4907623966, 9.7831198927, 9.9980487229, 3.7279599817, 3.0358585022, 3.9339454859, 1.7055670675, 2.6615688448, 2.0238218508, 10.8780753336, 11.4600783337, 12.1587520022, 13.7945646254, 13.1589222575, 13.342547118, 13.9311259257, -0.98078405525, -0.63223319399, -1.929094269, 8.5579341992, -0.79975861867, 0.6987605024, 2.0093938808, 1.957238027, 2.5023783296, 2.9270198103, 3.616503637, 5.3290175157, 5.0181873208, 13.0651636363, 13.3235606524, 6.4172576786, 7.1614692798, 7.607304798, 8.2321568553, 8.7568274818, -0.55566389977, 0.36757246427, 0.28018998819, 1.261105748, 8.9955481188, 10.1666747189, 9.8200303073, 10.1898516286, 11.6332278043, 11.4423294544, 0.043742524857, 1.4317108525, 1.446779116, 1.7235245936, 9.6155857044, 2.6780724807, 3.8338588243, 3.0809574447, 3.8233716995, 5.2736169021, 0.23027752914, -0.26124341972, -0.046130533987, 1.0311827165, 10.2695519873, 9.1487918458, 1.7938108203, 3.856934424, 3.8212591017, 3.3125876647, 3.2444220758, 3.0145873594, 3.5693841475, 4.527761117, 5.100913301, 12.6708939616, 13.4343961383, 14.8168409647, 14.6450726832, 15.5364022794, -1.8347375996, -1.4876759328, -1.2163971975, -0.29855416949, 0.092900206374, 8.2832035096, 0.013082856576, 0.89557660897, 1.7138465012, 2.2530638019, 0.76113222743, 1.043661851, 1.7984771121, 3.0208714656, 2.8743377143, 11.8256263263, 12.6746604421, 3.7250139312, 5.4491969792, 5.7317854828, 0.074535831744, 1.4428167027, 1.1066800877, 9.2238521871, 10.5115502601, 11.3772966396, 10.9541345172, 12.5793579545, 12.3657504826, 13.0027281071, -2.4127593869, -2.3899477395, -1.5435450717, 7.6410352797, -0.072105997553, -0.44113885708, -0.14677767369, 0.52052741992, 1.2023035084, 2.5550904186, -5.2176366841, -4.7239899175, -4.077171198, 4.735683732, 5.1687376682, -1.7818546965, -1.4377782451, -1.6932244105, -0.76675510256, 0.24508795491, 2.8905378674, 4.0481598896, 2.4793004476, 3.9117013242, 14.0021662289, 13.3506459755, 14.3634437928, 14.0247306384, 14.2254401205, 15.8489624363, 1.4579466797, 3.0292382559, 2.1459296903, 2.4742188578, 11.8428716637, 4.1913754533, 4.9542969898, 4.7906239328, 5.4839864806, 6.3573411246, 1.5502157812, 2.7273867786, 3.6829326297, 3.2500867309, 12.1787740115, 12.8266235821, 5.2117032225, 5.5086421172, 6.4287441774, 7.1957751897, 1.5717105195, 2.1034404835, 2.9853482577, 1.9922540467, 3.1024341108, 12.0144685484, 11.3633060429, 12.4330985545, 13.0966613228, 13.9743134509, 2.9326237068, 2.5054078142, 4.389182362, 4.6156128698, 5.1185159097, 13.6500763459, 5.7165365624, 6.8283484181, 7.2726425166, 7.1139668596, -2.7510252383, -1.7881562897, -0.7569242708, -0.66254116073, -1.0130202411, 8.2760144771, 7.6821223629, 0.55272479404, 0.88966618002, 1.6990596386, -1.1075252055, -1.0653834352, -0.43050860649, 7.9330480831, 8.497039261, 8.4097442995, 8.6939911965, 9.203283756, 10.6833768049, 11.668019098, 5.2983061789, 5.4612207703, 5.8971662603, 15.8508309236, 7.6326584579, 7.8929180033, 9.3436098284, 8.0966602405, 9.6916022556, 10.8633361911, 0.80205529395, 0.20107471189, 0.69348362951, 8.9602497569, 10.456290067, 3.2517643718, 3.3978156634, 3.1825880623, 4.0252656414, 3.81567534, -1.7209120673, -1.4829479201, -1.163442575, -1.790056039, 8.1935086645, 8.1984467002, 8.6224411669, 9.1043002177, 10.2683555294, 9.8689868745, 0.29561696587, 0.8554221211, 2.0339642264, 2.6531321176, 10.1813390641, 3.6104252101, 2.9532447436, 4.7696134091, 4.8295133364, 5.0527991602, 0.58705806724, 1.4180117625, 1.4244496058, 2.6400076277, 10.8060851593, 11.4310602002, 3.3611866291, 4.2666006678, 5.7305407586, 4.6888574948, -1.0815272324, -0.23665106862, 0.28232657309, 0.81491270698, 1.6876098526, 10.9938513032, 10.2613641269, 10.5614397285, 11.6892560277, 12.0693711136, -1.2744124685, -0.25160282178, 0.97134830573, 0.2398470695, 0.79407021378, 9.6954405575, 2.2737730981, 2.9067112029, 2.9139167009, 4.4474104736, 2.8877037347, 2.5995466583, 3.9802611614, 3.6293335277, 4.4413698643, 12.3784220024, 12.4012837999, 5.7105252229, 6.6018115119, 6.6402465829, 0.29239262974, 1.2220047033, 2.3159182171, 6.3110713658, 7.4969704119, 7.4467125172, 8.4492773221, 8.9485553238, 9.3296967175, 9.2584034658, 0.68581620777, 1.599784379, 1.1636924299, 6.6758938993, 1.702525925, 2.997307299, 4.1814522946, 4.5412303497, 4.6175597579, 4.2182195995, -1.5932447414, -0.68014300614, -0.6830571828, 4.3718025501, 4.3824694244, 0.098011860586, 0.77397454044, 0.73867374234, 2.4780776778, 1.453609304, 0.8477297338, -0.055091676225, 1.2279035812, 1.0540105073, 6.7232069649, 7.574659705, 8.6119120026, 8.965879519, 9.8579466434, 10.1375765135, 0.19069758436, 1.0157922636, 2.6483364818, 1.8677601978, 8.9249418047, 3.0859216384, 3.4905406714, 4.3722898374, 4.4431670254, 5.3993971226, -3.3670833707, -1.4697253234, -1.6150207132, -1.3246951404, 4.1797020113, 4.8251407527, 0.28412515654, 0.91461882204, 0.99263226799, 1.4414939648, -0.25098842145, -0.085274367311, -0.1373956682, 0.15144191374, 1.4329104513, 7.648234139, 6.8556590112, 6.5019130638, 8.3512621801, 8.8399798905, 1.8232880237, 1.0445447676, 1.6789949488, 2.2221513625, 2.9716852472, 8.8662419521, 3.5103702855, 4.1750677138, 5.5765981069, 6.1889800904, 0.37819640531, 2.3085907134, 2.2703432485, 2.0339267798, 2.4933037099, 7.2969353222, 8.6685670728, 3.9244198419, 4.1597588735, 4.7480514032, 2.3197377468, 2.9066032253, 3.0275584801, 9.0037335396, 7.9370045254, 9.4697266898, 10.1525937348, 10.5695573614, 10.9108106702, 10.5700879261, -2.0411429691, -1.730527702, -0.33798743459, 3.6419238489, 0.92486595565, 1.148748046, 1.2579016489, 2.753824807, 1.7992774598, 3.2821386074, 1.9269115148, 2.1414404672, 2.7275337912, 8.0888173297, 9.0665421161, 3.9158916232, 5.793690125, 5.6687501361, 5.5801231283, 5.5029519768, 1.9462441818, 2.587106672, 2.2657292461, 3.725756554, 8.8071198087, 8.8919966727, 10.4295998433, 11.2295422809, 10.992419767, 12.3528988981, 0.51858760917, 0.91477075195, 1.0080317414, 2.3184150701, 8.311700509, 3.6885278127, 3.4003729172, 4.6786132486, 4.0686884409, 4.915805778, -0.51524152113, -0.58645941226, -0.18816426967, -0.1146746434, 5.2180432031, 6.362684678, 1.9185794208, 2.758895588, 3.2557682801, 3.5754196536, -1.4128881009, -1.1117636987, -0.22622997905, 0.34854714406, 0.53510934436, 6.3081790903, 6.8667069477, 7.1597174792, 7.0436647989, 8.1714976953, 1.8119864596, 1.5601421163, 2.4154945667, 2.8617976566, 4.0783180801, 9.5651932194, 5.270553974, 5.0677497909, 5.684842441, 6.0440030115, -4.0527566651, -4.3064178115, -3.4440934523, -3.2411382271, -1.5680894026, 3.5066065231, 4.0222421096, -1.8773911565, -0.26988754805, 0.26043138596, 2.8361206552, 3.8286696348, 3.7203584628, 9.7198598921, 9.0413047461, 11.5538392884, 11.30481579, 12.1053735548, 11.4067468353, 12.8609925775, -0.45512969419, -0.10209549375, 0.68755271212, 5.988299857, 0.79090267123, 3.0182794574, 2.6212420009, 2.6720411405, 3.2406228495, 3.6639011017, 3.277243578, 4.5561359837, 4.1140656646, 10.0312637383, 10.6523891786, 6.5488403692, 6.367970651, 7.1791200758, 7.5460429143, 8.020449256, -2.6950947559, -3.0761058934, -2.4352818908, -1.8047061817, 2.8460391719, 4.387045321, 4.9861574992, 5.6441444576, 5.4262427911, 5.9915006491, -2.7025124396, -2.0006968552, -1.4619125359, -0.78214005868, 4.7204380238, 0.14475544389, 0.31247146682, 0.81393622044, 2.1864289018, 1.2845520714, -1.6017611223, -0.63409687164, -0.61757075983, 1.3645308869, 5.0822395083, 5.5951077433, 1.861764545, 1.8682365106, 2.7495963329, 3.4486828226, 1.4498521375, 2.252985795, 2.5042359162, 3.3001439812, 3.7402133884, 9.1921958482, 10.0247020114, 9.9641095372, 10.5841137951, 10.7239198288, -3.508178256, -2.9298772963, -2.7993204407, -2.0102210017, -2.1316030015, 4.4468867886, -0.49269407384, 0.68321152785, 0.098691593493, 0.44118759045, -2.0920700385, -1.590574067, -0.86172015186, -1.2531938485, 0.33645216964, 5.7867090665, 5.3428352061, 1.4600703271, 1.456727736, 3.0492924763, -0.48978688819, -0.18337661381, 0.72365944284, 6.6551141084, 6.9412464716, 7.2842518826, 7.5096654312, 7.9465656395, 8.6876180297, 9.0223244405, -4.0331048686, -4.6608913806, -3.2781301701, 1.6831261139, -3.439395894, -2.0717054903, -1.6038605293, -1.4175370871, -0.90858805401, -1.038290352, -2.7774091386, -2.3177608925, -0.85938575013, 5.5146980102, 5.7348025667, 0.5754581037, 0.65891969572, 1.2389167582, 2.0435927323, 1.8646853236, 1.13044316, 1.6152067423, 1.8433539367, 2.5184079189, 8.1693710149, 9.1298558756, 9.4690362017, 10.4772665744, 9.9178710379, 11.2768003797, -4.2201928233, -3.1296750739, -2.1963975207, -2.4282653157, 2.6559008893, -1.345311643, -0.55777399722, 0.039359733318, -0.091148987272, 1.1577176451, 2.1214982795, 2.1074229551, 3.2838730945, 5.122385666, 8.4827558178, 9.5363576609, 4.3805908855, 5.6180574956, 5.4885979348, 6.6574463826, 3.3044452322, 3.7907846417, 3.5434672216, 4.5968076373, 5.0973756463, 11.0498816954, 11.3102519271, 12.1880907787, 12.6804100484, 12.3694441975, -1.0776008576, -0.80639205059, -0.25968633449, -0.27319955367, -0.4020016233, 6.1943632529, 2.2952204585, 2.2180117426, 2.1132930971, 2.3116309818, -1.8467022583, -1.8965759207, -1.4672865528, -1.4475057995, -0.16632255853, 4.5370957083, 5.6290440436, 1.4496676538, 1.8695318093, 1.6356947361, -0.89664436868, 0.37852159348, 1.4254957278, 6.2304811947, 5.9777317, 7.1459413811, 7.5235867108, 8.6613019746, 8.1321174316, 9.5279854166, -1.4589153635, -0.78306139038, -0.20238002646, 5.5104827626, 0.49152010598, 0.86744374768, 1.9286384636, 2.5806103308, 2.8496930824, 2.4348250895, -0.036533972497, 0.207004254, -0.49018076779, 6.0529719853, 6.3484880533, 1.5371985143, 2.6551831784, 2.7127862584, 2.6929283426, 3.0481687887, 0.41533608345, 0.19412270959, 0.44662839578, 1.3063368129, 6.5002944896, 6.2187332039, 7.9403458529, 8.6118875247, 8.4482050796, 9.1856316845, 0.18053704127, -0.10682316028, 0.9128127945, 1.5004444945, 6.7936053535, 2.0289864485, 2.982605235, 2.8026053225, 3.5232163892, 4.8206101575, 3.2125051652, 3.9434954038, 4.5812676634, 5.181856726, 11.5547003011, 11.3794504772, 6.4794442193, 7.0864431319, 7.3520479742, 7.6067573977, 0.45400730084, 0.50469593165, -0.15334942425, 1.6789373293, 1.1334069731, 6.8475748307, 7.0292239561, 7.339011245, 8.4666051676, 8.5136696916, -1.2573536105, -0.13785707747, 0.58761560397, 1.4980853561, 1.6776613045, 7.8164304743, 1.8078839037, 2.3531997482, 3.4236838727, 3.719405626, 0.074032633417, 1.814538819, 1.3581908836, 2.3461222296, 3.1857112741, 8.0960479271, 7.5620221866, 3.9524023678, 5.0265240703, 4.8961197126, 1.2242853074, 1.4319331965, 2.1865840446, 1.2445095092, 3.2472439405, 3.5478312994, 4.3347472449, 3.7025538593, 3.7343055628, 5.2520598565, 0.42261622489, 1.5341546878, 2.1091275475, 2.6454517731, 2.9877288173, 3.7105099568, 3.371385426, 4.9902149205, 5.6603785771, 5.5144729766, 0.83738592206, 1.1336031968, 1.2955173463, 2.4042237907, 2.0025159071, 3.6017998716, 3.0922754444, 4.0010512866, 3.3814397228, 5.3317601623, -2.7224373238, -1.6068537104, -1.8182638402, -1.3495740912, -0.5678388365, -0.17789129917, -0.25266487419, 0.65726082751, 1.1299674607, 2.685825718, 2.4294472048, 2.69679639, 3.5074766318, 3.5273647629, 4.9542552887, 6.234232526, 6.0380579643, 6.0894049413, 6.2517694637, 7.3639076133, 0.67674781791, 2.3785713361, 2.1147102458, 2.6698045443, 4.6512799113, 2.9411541538, 4.550284476, 6.3724810856, 6.307971804, 5.7232910315, -1.6492585942, -1.7539792047, -1.9123520004, -0.40869695192, 0.74677096825, 0.13807947305, 1.0984134077, 2.4333951981, 3.0935415539, 2.855910576, 2.7513777021, 2.2988540544, 3.4339824897, 4.0025603665, 4.9161784384, 5.3750996072, 5.0699472188, 5.9682120541, 6.3750415099, 6.8659144838, -0.25008999524, -0.36155368448, 1.0730390213, 0.92233794381, 1.4353629051, 3.4427390551, 2.4755656479, 3.8880553371, 3.1890537717, 4.6918300874, 2.114627159, 1.6309141333, 3.1799739141, 4.1875171173, 3.2056631079, 4.3838886062, 5.1727617959, 4.2613681872, 5.2715451911, 6.5128933687, 2.0483819212, 3.9674454647, 2.7202020758, 3.5510473395, 4.1734635461, 3.8062951613, 5.3111025233, 5.2316163492, 5.614005842, 6.9321329919, 0.90297804009, 2.483627712, 3.0084619546, 2.9788799322, 3.6033174203, 3.4533849012, 3.9437255353, 4.9397819987, 4.8915899111, 4.8730162756, 0.040066616557, 0.6590094332, 0.97007462142, 0.68855093457, 2.6142505976, 2.7121691125, 3.4645319517, 3.5547984743, 4.4703968352, 4.798246022, -0.33000579984, -0.51179645192, 2.0683298992, 1.5578004908, 2.9519912561, 2.3182333687, 2.6519502439, 5.0200643644, 4.3979462621, 4.697667554, -0.60852106376, 0.12819634273, 0.65851446779, 2.334157038, 1.2507354539, 2.8736625902, 2.856854623, 3.8526650288, 3.6036816207, 4.363825628, -0.73128646947, -0.87705949197, 0.2068024588, 0.8102192234, 0.49489152403, 1.5117635517, 1.391564736, 3.0038229366, 3.247422302, 3.0927705043, -4.1039807396, -2.9931419689, -3.2088313683, -2.9148890748, -2.5175390588, -2.2039553654, -1.5537585763, -1.5038679763, -0.77320784305, -0.065851868564, 1.0313142498, 2.9571659829, 2.0190895471, 3.7268996701, 4.6589797789, 4.6889558678, 4.9110222729, 5.7559852625, 5.5423346785, 6.2178089257, 2.9428533374, 4.757281362, 4.6941910741, 4.478268561, 6.180925968, 5.971211092, 6.6300126261, 7.8431354173, 7.6138885889, 8.2870484982, -4.6486500232, -4.3293962547, -3.3837629389, -3.5033584064, -2.7672857308, -1.7697210179, -1.5745726688, -0.67578078035, -0.082928399663, 0.011366511, -0.49283294835, -0.38636689804, 0.83643989751, 1.212473889, 1.5125671082, 2.7632103096, 1.7720701418, 2.4320263932, 3.9936373848, 3.4689586505, 0.91627601873, 0.98105976401, 1.6130113126, 2.2386443746, 2.7213773977, 3.7001782294, 2.8956455331, 3.8890665453, 4.5542578852, 6.1909092137, -3.1401737915, -3.3173951931, -2.614595799, -1.948952474, -1.3420523774, -0.51325227315, -0.89496354307, 0.69317522502, 0.83387600449, 1.3736895326, -1.3991665799, -0.88071166454, -1.7935199456, -1.0327456384, 0.22797073004, 0.4750719784, 0.99050757786, 0.47105082401, 2.2144708992, 1.0455693417, -0.23698467418, 1.5335503698, 2.7875031807, 1.3339827803, 2.6836804393, 4.1014726763, 4.4595980479, 4.010149303, 4.8514460328, 5.0520554245, 2.2453832707, 3.296747875, 2.5233817543, 3.2384413823, 3.5365389376, 4.7254040415, 4.8124189642, 4.4508694895, 5.6664761848, 6.6236985335, 1.0230133467, 0.76044973396, 2.5663115776, 3.2345561469, 2.9240503032, 3.1886070679, 3.9852755792, 4.7770707231, 4.9810713019, 5.6255235495, -0.90286019841, -0.076239112849, 0.3715132453, 2.1594074749, 2.5021690364, 1.8118486327, 2.2917715352, 3.1525505143, 3.5313555246, 2.9499473078, 0.50211334146, 1.242218949, 0.87806989916, 2.5472634181, 2.4091157411, 2.0542727525, 2.1935037832, 3.6388483348, 4.8883468351, 3.6319662274, -2.6163110762, -2.0306604353, -0.74956124941, -1.1605560257, 0.057877418549, 0.21359397186, 0.81043487155, 1.6319201969, 2.0516515842, 3.361952844], + "het_x": [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] + }, + "params": { + "pattern": "multi_path_reversible_predict_het", + "n_switchers": 90, + "n_controls": 30, + "n_groups": 120, + "n_periods": 10, + "seed": 120, + "effects": 3, + "predict_het_var": "het_x", + "predict_het_horizons": [1, 2, 3], + "ci_level": 95, + "dont_drop_larger_lower": true + }, + "results": { + "predict_het": { + "1": { + "beta": 3.1129329005, + "se": 0.16802462607, + "t": 18.5266467977, + "ci_lo": 2.7789109989, + "ci_hi": 3.4469548022, + "n_obs": 90, + "p_value": 9.0828759468e-32 + }, + "2": { + "beta": 2.0734964222, + "se": 0.69578948634, + "t": 2.9800628824, + "ci_lo": 0.69031270198, + "ci_hi": 3.4566801424, + "n_obs": 90, + "p_value": 0.0037461129962 + }, + "3": { + "beta": 1.0453185701, + "se": 0.69089010811, + "t": 1.5130026582, + "ci_lo": -0.32812550855, + "ci_hi": 2.4187626488, + "n_obs": 90, + "p_value": 0.13394537794 + } + } + } + }, + "multi_path_reversible_by_path_predict_het": { + "data": { + "group": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 37, 37, 37, 37, 37, 37, 37, 37, 37, 37, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 39, 39, 39, 39, 39, 39, 39, 39, 39, 39, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 43, 43, 43, 43, 43, 43, 43, 43, 43, 43, 44, 44, 44, 44, 44, 44, 44, 44, 44, 44, 45, 45, 45, 45, 45, 45, 45, 45, 45, 45, 46, 46, 46, 46, 46, 46, 46, 46, 46, 46, 47, 47, 47, 47, 47, 47, 47, 47, 47, 47, 48, 48, 48, 48, 48, 48, 48, 48, 48, 48, 49, 49, 49, 49, 49, 49, 49, 49, 49, 49, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 51, 51, 51, 51, 51, 51, 51, 51, 51, 51, 52, 52, 52, 52, 52, 52, 52, 52, 52, 52, 53, 53, 53, 53, 53, 53, 53, 53, 53, 53, 54, 54, 54, 54, 54, 54, 54, 54, 54, 54, 55, 55, 55, 55, 55, 55, 55, 55, 55, 55, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 57, 57, 57, 57, 57, 57, 57, 57, 57, 57, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 61, 61, 61, 61, 61, 61, 61, 61, 61, 61, 62, 62, 62, 62, 62, 62, 62, 62, 62, 62, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 66, 66, 66, 66, 66, 66, 66, 66, 66, 66, 67, 67, 67, 67, 67, 67, 67, 67, 67, 67, 68, 68, 68, 68, 68, 68, 68, 68, 68, 68, 69, 69, 69, 69, 69, 69, 69, 69, 69, 69, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 71, 71, 71, 71, 71, 71, 71, 71, 71, 71, 72, 72, 72, 72, 72, 72, 72, 72, 72, 72, 73, 73, 73, 73, 73, 73, 73, 73, 73, 73, 74, 74, 74, 74, 74, 74, 74, 74, 74, 74, 75, 75, 75, 75, 75, 75, 75, 75, 75, 75, 76, 76, 76, 76, 76, 76, 76, 76, 76, 76, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 78, 78, 78, 78, 78, 78, 78, 78, 78, 78, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 82, 82, 82, 82, 82, 82, 82, 82, 82, 82, 83, 83, 83, 83, 83, 83, 83, 83, 83, 83, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 85, 85, 85, 85, 85, 85, 85, 85, 85, 85, 86, 86, 86, 86, 86, 86, 86, 86, 86, 86, 87, 87, 87, 87, 87, 87, 87, 87, 87, 87, 88, 88, 88, 88, 88, 88, 88, 88, 88, 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 90, 90, 90, 90, 90, 90, 90, 90, 90, 90, 91, 91, 91, 91, 91, 91, 91, 91, 91, 91, 92, 92, 92, 92, 92, 92, 92, 92, 92, 92, 93, 93, 93, 93, 93, 93, 93, 93, 93, 93, 94, 94, 94, 94, 94, 94, 94, 94, 94, 94, 95, 95, 95, 95, 95, 95, 95, 95, 95, 95, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 97, 97, 97, 97, 97, 97, 97, 97, 97, 97, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 101, 101, 101, 101, 101, 101, 101, 101, 101, 101, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 106, 106, 106, 106, 106, 106, 106, 106, 106, 106, 107, 107, 107, 107, 107, 107, 107, 107, 107, 107, 108, 108, 108, 108, 108, 108, 108, 108, 108, 108, 109, 109, 109, 109, 109, 109, 109, 109, 109, 109, 110, 110, 110, 110, 110, 110, 110, 110, 110, 110, 111, 111, 111, 111, 111, 111, 111, 111, 111, 111, 112, 112, 112, 112, 112, 112, 112, 112, 112, 112, 113, 113, 113, 113, 113, 113, 113, 113, 113, 113, 114, 114, 114, 114, 114, 114, 114, 114, 114, 114, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 116, 116, 116, 116, 116, 116, 116, 116, 116, 116, 117, 117, 117, 117, 117, 117, 117, 117, 117, 117, 118, 118, 118, 118, 118, 118, 118, 118, 118, 118, 119, 119, 119, 119, 119, 119, 119, 119, 119, 119], + "period": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9], + "treatment": [0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], + "outcome": [-0.38947445306, -0.23568361568, 0.21162698298, 9.4874432555, 9.5272086491, 9.8732152644, 10.4966606852, 10.7269218628, 11.3041347822, 12.3075808547, -0.53540224144, -0.85896484972, 0.9039919656, 8.4216226165, 0.32926420824, 1.5187015075, 2.2466260038, 2.3752794078, 2.5489603023, 3.8839539563, -1.6531086427, -0.48172021176, -1.1588810115, 7.5449719691, 8.2423444411, 0.16054588328, 1.34512733, 2.3267248703, 2.907302291, 2.7362604935, 3.9790870637, 5.1832195997, 4.9765720574, 5.9562138217, 14.7385510317, 14.1883967913, 14.9982098406, 16.5614484131, 16.3735973947, 17.2722352384, 2.4672139194, 3.181736844, 2.8966044002, 3.8788938746, 12.0094366558, 4.5284429927, 4.7586747034, 6.3688938717, 6.8655832555, 6.6200148023, 1.366346809, 2.2681542407, 1.0986738641, 1.3627616225, 10.60127055, 10.2334877677, 3.7661446414, 2.8049023737, 5.3088063004, 5.7352273543, -2.1462960524, -1.562262737, -0.68045628748, -0.39170645702, 0.51893991954, 8.2908071534, 9.041118565, 9.1952101148, 10.5230194116, 10.4974242438, 3.8654236028, 3.9905881505, 4.2598845829, 5.4183992861, 5.9314932187, 13.2206109279, 5.3935881935, 7.4081365763, 7.5828582314, 7.359520808, 0.32944110732, 1.3451564511, 0.47252530565, 1.86498985, 2.4477442051, 10.7154679806, 10.8414260484, 3.9189896786, 4.2352551524, 4.3157247407, 1.3062082738, 2.2840554655, 2.5999388791, 11.2750783874, 11.7106216584, 11.6742402808, 13.1041698079, 13.2031556633, 13.5015002952, 14.1776065603, 0.11158785236, 1.054725582, 2.5473237287, 9.9685092, 2.5553042659, 3.1672544975, 3.2044163382, 3.279807654, 4.9738691668, 6.0325408926, -0.080780006181, 2.1294224371, 1.8617613349, 9.4372859439, 10.6907812039, 3.0731760034, 3.9983448664, 3.9479302438, 4.5770928564, 4.759461957, 1.2233010078, 1.5007797084, 1.8169053771, 3.0011971575, 11.1825713053, 12.4024767365, 12.1157117449, 12.2510645066, 13.4088151694, 13.9401105065, -2.7956639463, -1.9178517996, -1.8381646645, -1.0116116427, 7.2114118991, -0.47283326103, -0.2375849953, -0.21983407988, 0.83491171322, 1.7623326366, -2.4904909619, -1.1251647276, -0.0021674062124, 0.20812688743, 9.9195669513, 9.5480968853, 2.3956781408, 2.1183487474, 3.0646069867, 3.808704102, -2.0779624805, -1.9780684504, -0.33525118838, -0.057425341904, 1.0012356804, 8.1271505508, 8.7896794462, 8.8251678355, 10.1813290206, 10.1022892304, 3.6502716683, 3.7201719177, 3.7674199724, 5.1980443964, 5.7139097866, 14.6401937191, 7.0192780984, 7.0612922863, 7.5443921558, 8.7556031236, -0.26132650677, -0.50936928911, 0.99992467827, 0.80556665801, 1.4907623966, 9.7831198927, 9.9980487229, 3.7279599817, 3.0358585022, 3.9339454859, 1.7055670675, 2.6615688448, 2.0238218508, 10.8780753336, 11.4600783337, 12.1587520022, 13.7945646254, 13.1589222575, 13.342547118, 13.9311259257, -0.98078405525, -0.63223319399, -1.929094269, 8.5579341992, -0.79975861867, 0.6987605024, 2.0093938808, 1.957238027, 2.5023783296, 2.9270198103, 3.616503637, 5.3290175157, 5.0181873208, 13.0651636363, 13.3235606524, 6.4172576786, 7.1614692798, 7.607304798, 8.2321568553, 8.7568274818, -0.55566389977, 0.36757246427, 0.28018998819, 1.261105748, 8.9955481188, 10.1666747189, 9.8200303073, 10.1898516286, 11.6332278043, 11.4423294544, 0.043742524857, 1.4317108525, 1.446779116, 1.7235245936, 9.6155857044, 2.6780724807, 3.8338588243, 3.0809574447, 3.8233716995, 5.2736169021, 0.23027752914, -0.26124341972, -0.046130533987, 1.0311827165, 10.2695519873, 9.1487918458, 1.7938108203, 3.856934424, 3.8212591017, 3.3125876647, 3.2444220758, 3.0145873594, 3.5693841475, 4.527761117, 5.100913301, 12.6708939616, 13.4343961383, 14.8168409647, 14.6450726832, 15.5364022794, -1.8347375996, -1.4876759328, -1.2163971975, -0.29855416949, 0.092900206374, 8.2832035096, 0.013082856576, 0.89557660897, 1.7138465012, 2.2530638019, 0.76113222743, 1.043661851, 1.7984771121, 3.0208714656, 2.8743377143, 11.8256263263, 12.6746604421, 3.7250139312, 5.4491969792, 5.7317854828, 0.074535831744, 1.4428167027, 1.1066800877, 9.2238521871, 10.5115502601, 11.3772966396, 10.9541345172, 12.5793579545, 12.3657504826, 13.0027281071, -2.4127593869, -2.3899477395, -1.5435450717, 7.6410352797, -0.072105997553, -0.44113885708, -0.14677767369, 0.52052741992, 1.2023035084, 2.5550904186, -5.2176366841, -4.7239899175, -4.077171198, 4.735683732, 5.1687376682, -1.7818546965, -1.4377782451, -1.6932244105, -0.76675510256, 0.24508795491, 2.8905378674, 4.0481598896, 2.4793004476, 3.9117013242, 14.0021662289, 13.3506459755, 14.3634437928, 14.0247306384, 14.2254401205, 15.8489624363, 1.4579466797, 3.0292382559, 2.1459296903, 2.4742188578, 11.8428716637, 4.1913754533, 4.9542969898, 4.7906239328, 5.4839864806, 6.3573411246, 1.5502157812, 2.7273867786, 3.6829326297, 3.2500867309, 12.1787740115, 12.8266235821, 5.2117032225, 5.5086421172, 6.4287441774, 7.1957751897, 1.5717105195, 2.1034404835, 2.9853482577, 1.9922540467, 3.1024341108, 12.0144685484, 11.3633060429, 12.4330985545, 13.0966613228, 13.9743134509, 2.9326237068, 2.5054078142, 4.389182362, 4.6156128698, 5.1185159097, 13.6500763459, 5.7165365624, 6.8283484181, 7.2726425166, 7.1139668596, -2.7510252383, -1.7881562897, -0.7569242708, -0.66254116073, -1.0130202411, 8.2760144771, 7.6821223629, 0.55272479404, 0.88966618002, 1.6990596386, -1.1075252055, -1.0653834352, -0.43050860649, 7.9330480831, 8.497039261, 8.4097442995, 8.6939911965, 9.203283756, 10.6833768049, 11.668019098, 5.2983061789, 5.4612207703, 5.8971662603, 15.8508309236, 7.6326584579, 7.8929180033, 9.3436098284, 8.0966602405, 9.6916022556, 10.8633361911, 0.80205529395, 0.20107471189, 0.69348362951, 8.9602497569, 10.456290067, 3.2517643718, 3.3978156634, 3.1825880623, 4.0252656414, 3.81567534, -1.7209120673, -1.4829479201, -1.163442575, -1.790056039, 8.1935086645, 8.1984467002, 8.6224411669, 9.1043002177, 10.2683555294, 9.8689868745, 0.29561696587, 0.8554221211, 2.0339642264, 2.6531321176, 10.1813390641, 3.6104252101, 2.9532447436, 4.7696134091, 4.8295133364, 5.0527991602, 0.58705806724, 1.4180117625, 1.4244496058, 2.6400076277, 10.8060851593, 11.4310602002, 3.3611866291, 4.2666006678, 5.7305407586, 4.6888574948, -1.0815272324, -0.23665106862, 0.28232657309, 0.81491270698, 1.6876098526, 10.9938513032, 10.2613641269, 10.5614397285, 11.6892560277, 12.0693711136, -1.2744124685, -0.25160282178, 0.97134830573, 0.2398470695, 0.79407021378, 9.6954405575, 2.2737730981, 2.9067112029, 2.9139167009, 4.4474104736, 2.8877037347, 2.5995466583, 3.9802611614, 3.6293335277, 4.4413698643, 12.3784220024, 12.4012837999, 5.7105252229, 6.6018115119, 6.6402465829, 0.29239262974, 1.2220047033, 2.3159182171, 6.3110713658, 7.4969704119, 7.4467125172, 8.4492773221, 8.9485553238, 9.3296967175, 9.2584034658, 0.68581620777, 1.599784379, 1.1636924299, 6.6758938993, 1.702525925, 2.997307299, 4.1814522946, 4.5412303497, 4.6175597579, 4.2182195995, -1.5932447414, -0.68014300614, -0.6830571828, 4.3718025501, 4.3824694244, 0.098011860586, 0.77397454044, 0.73867374234, 2.4780776778, 1.453609304, 0.8477297338, -0.055091676225, 1.2279035812, 1.0540105073, 6.7232069649, 7.574659705, 8.6119120026, 8.965879519, 9.8579466434, 10.1375765135, 0.19069758436, 1.0157922636, 2.6483364818, 1.8677601978, 8.9249418047, 3.0859216384, 3.4905406714, 4.3722898374, 4.4431670254, 5.3993971226, -3.3670833707, -1.4697253234, -1.6150207132, -1.3246951404, 4.1797020113, 4.8251407527, 0.28412515654, 0.91461882204, 0.99263226799, 1.4414939648, -0.25098842145, -0.085274367311, -0.1373956682, 0.15144191374, 1.4329104513, 7.648234139, 6.8556590112, 6.5019130638, 8.3512621801, 8.8399798905, 1.8232880237, 1.0445447676, 1.6789949488, 2.2221513625, 2.9716852472, 8.8662419521, 3.5103702855, 4.1750677138, 5.5765981069, 6.1889800904, 0.37819640531, 2.3085907134, 2.2703432485, 2.0339267798, 2.4933037099, 7.2969353222, 8.6685670728, 3.9244198419, 4.1597588735, 4.7480514032, 2.3197377468, 2.9066032253, 3.0275584801, 9.0037335396, 7.9370045254, 9.4697266898, 10.1525937348, 10.5695573614, 10.9108106702, 10.5700879261, -2.0411429691, -1.730527702, -0.33798743459, 3.6419238489, 0.92486595565, 1.148748046, 1.2579016489, 2.753824807, 1.7992774598, 3.2821386074, 1.9269115148, 2.1414404672, 2.7275337912, 8.0888173297, 9.0665421161, 3.9158916232, 5.793690125, 5.6687501361, 5.5801231283, 5.5029519768, 1.9462441818, 2.587106672, 2.2657292461, 3.725756554, 8.8071198087, 8.8919966727, 10.4295998433, 11.2295422809, 10.992419767, 12.3528988981, 0.51858760917, 0.91477075195, 1.0080317414, 2.3184150701, 8.311700509, 3.6885278127, 3.4003729172, 4.6786132486, 4.0686884409, 4.915805778, -0.51524152113, -0.58645941226, -0.18816426967, -0.1146746434, 5.2180432031, 6.362684678, 1.9185794208, 2.758895588, 3.2557682801, 3.5754196536, -1.4128881009, -1.1117636987, -0.22622997905, 0.34854714406, 0.53510934436, 6.3081790903, 6.8667069477, 7.1597174792, 7.0436647989, 8.1714976953, 1.8119864596, 1.5601421163, 2.4154945667, 2.8617976566, 4.0783180801, 9.5651932194, 5.270553974, 5.0677497909, 5.684842441, 6.0440030115, -4.0527566651, -4.3064178115, -3.4440934523, -3.2411382271, -1.5680894026, 3.5066065231, 4.0222421096, -1.8773911565, -0.26988754805, 0.26043138596, 2.8361206552, 3.8286696348, 3.7203584628, 9.7198598921, 9.0413047461, 11.5538392884, 11.30481579, 12.1053735548, 11.4067468353, 12.8609925775, -0.45512969419, -0.10209549375, 0.68755271212, 5.988299857, 0.79090267123, 3.0182794574, 2.6212420009, 2.6720411405, 3.2406228495, 3.6639011017, 3.277243578, 4.5561359837, 4.1140656646, 10.0312637383, 10.6523891786, 6.5488403692, 6.367970651, 7.1791200758, 7.5460429143, 8.020449256, -2.6950947559, -3.0761058934, -2.4352818908, -1.8047061817, 2.8460391719, 4.387045321, 4.9861574992, 5.6441444576, 5.4262427911, 5.9915006491, -2.7025124396, -2.0006968552, -1.4619125359, -0.78214005868, 4.7204380238, 0.14475544389, 0.31247146682, 0.81393622044, 2.1864289018, 1.2845520714, -1.6017611223, -0.63409687164, -0.61757075983, 1.3645308869, 5.0822395083, 5.5951077433, 1.861764545, 1.8682365106, 2.7495963329, 3.4486828226, 1.4498521375, 2.252985795, 2.5042359162, 3.3001439812, 3.7402133884, 9.1921958482, 10.0247020114, 9.9641095372, 10.5841137951, 10.7239198288, -3.508178256, -2.9298772963, -2.7993204407, -2.0102210017, -2.1316030015, 4.4468867886, -0.49269407384, 0.68321152785, 0.098691593493, 0.44118759045, -2.0920700385, -1.590574067, -0.86172015186, -1.2531938485, 0.33645216964, 5.7867090665, 5.3428352061, 1.4600703271, 1.456727736, 3.0492924763, -0.48978688819, -0.18337661381, 0.72365944284, 6.6551141084, 6.9412464716, 7.2842518826, 7.5096654312, 7.9465656395, 8.6876180297, 9.0223244405, -4.0331048686, -4.6608913806, -3.2781301701, 1.6831261139, -3.439395894, -2.0717054903, -1.6038605293, -1.4175370871, -0.90858805401, -1.038290352, -2.7774091386, -2.3177608925, -0.85938575013, 5.5146980102, 5.7348025667, 0.5754581037, 0.65891969572, 1.2389167582, 2.0435927323, 1.8646853236, 1.13044316, 1.6152067423, 1.8433539367, 2.5184079189, 8.1693710149, 9.1298558756, 9.4690362017, 10.4772665744, 9.9178710379, 11.2768003797, -4.2201928233, -3.1296750739, -2.1963975207, -2.4282653157, 2.6559008893, -1.345311643, -0.55777399722, 0.039359733318, -0.091148987272, 1.1577176451, 2.1214982795, 2.1074229551, 3.2838730945, 5.122385666, 8.4827558178, 9.5363576609, 4.3805908855, 5.6180574956, 5.4885979348, 6.6574463826, 3.3044452322, 3.7907846417, 3.5434672216, 4.5968076373, 5.0973756463, 11.0498816954, 11.3102519271, 12.1880907787, 12.6804100484, 12.3694441975, -1.0776008576, -0.80639205059, -0.25968633449, -0.27319955367, -0.4020016233, 6.1943632529, 2.2952204585, 2.2180117426, 2.1132930971, 2.3116309818, -1.8467022583, -1.8965759207, -1.4672865528, -1.4475057995, -0.16632255853, 4.5370957083, 5.6290440436, 1.4496676538, 1.8695318093, 1.6356947361, -0.89664436868, 0.37852159348, 1.4254957278, 6.2304811947, 5.9777317, 7.1459413811, 7.5235867108, 8.6613019746, 8.1321174316, 9.5279854166, -1.4589153635, -0.78306139038, -0.20238002646, 5.5104827626, 0.49152010598, 0.86744374768, 1.9286384636, 2.5806103308, 2.8496930824, 2.4348250895, -0.036533972497, 0.207004254, -0.49018076779, 6.0529719853, 6.3484880533, 1.5371985143, 2.6551831784, 2.7127862584, 2.6929283426, 3.0481687887, 0.41533608345, 0.19412270959, 0.44662839578, 1.3063368129, 6.5002944896, 6.2187332039, 7.9403458529, 8.6118875247, 8.4482050796, 9.1856316845, 0.18053704127, -0.10682316028, 0.9128127945, 1.5004444945, 6.7936053535, 2.0289864485, 2.982605235, 2.8026053225, 3.5232163892, 4.8206101575, 3.2125051652, 3.9434954038, 4.5812676634, 5.181856726, 11.5547003011, 11.3794504772, 6.4794442193, 7.0864431319, 7.3520479742, 7.6067573977, 0.45400730084, 0.50469593165, -0.15334942425, 1.6789373293, 1.1334069731, 6.8475748307, 7.0292239561, 7.339011245, 8.4666051676, 8.5136696916, -1.2573536105, -0.13785707747, 0.58761560397, 1.4980853561, 1.6776613045, 7.8164304743, 1.8078839037, 2.3531997482, 3.4236838727, 3.719405626, 0.074032633417, 1.814538819, 1.3581908836, 2.3461222296, 3.1857112741, 8.0960479271, 7.5620221866, 3.9524023678, 5.0265240703, 4.8961197126, 1.2242853074, 1.4319331965, 2.1865840446, 1.2445095092, 3.2472439405, 3.5478312994, 4.3347472449, 3.7025538593, 3.7343055628, 5.2520598565, 0.42261622489, 1.5341546878, 2.1091275475, 2.6454517731, 2.9877288173, 3.7105099568, 3.371385426, 4.9902149205, 5.6603785771, 5.5144729766, 0.83738592206, 1.1336031968, 1.2955173463, 2.4042237907, 2.0025159071, 3.6017998716, 3.0922754444, 4.0010512866, 3.3814397228, 5.3317601623, -2.7224373238, -1.6068537104, -1.8182638402, -1.3495740912, -0.5678388365, -0.17789129917, -0.25266487419, 0.65726082751, 1.1299674607, 2.685825718, 2.4294472048, 2.69679639, 3.5074766318, 3.5273647629, 4.9542552887, 6.234232526, 6.0380579643, 6.0894049413, 6.2517694637, 7.3639076133, 0.67674781791, 2.3785713361, 2.1147102458, 2.6698045443, 4.6512799113, 2.9411541538, 4.550284476, 6.3724810856, 6.307971804, 5.7232910315, -1.6492585942, -1.7539792047, -1.9123520004, -0.40869695192, 0.74677096825, 0.13807947305, 1.0984134077, 2.4333951981, 3.0935415539, 2.855910576, 2.7513777021, 2.2988540544, 3.4339824897, 4.0025603665, 4.9161784384, 5.3750996072, 5.0699472188, 5.9682120541, 6.3750415099, 6.8659144838, -0.25008999524, -0.36155368448, 1.0730390213, 0.92233794381, 1.4353629051, 3.4427390551, 2.4755656479, 3.8880553371, 3.1890537717, 4.6918300874, 2.114627159, 1.6309141333, 3.1799739141, 4.1875171173, 3.2056631079, 4.3838886062, 5.1727617959, 4.2613681872, 5.2715451911, 6.5128933687, 2.0483819212, 3.9674454647, 2.7202020758, 3.5510473395, 4.1734635461, 3.8062951613, 5.3111025233, 5.2316163492, 5.614005842, 6.9321329919, 0.90297804009, 2.483627712, 3.0084619546, 2.9788799322, 3.6033174203, 3.4533849012, 3.9437255353, 4.9397819987, 4.8915899111, 4.8730162756, 0.040066616557, 0.6590094332, 0.97007462142, 0.68855093457, 2.6142505976, 2.7121691125, 3.4645319517, 3.5547984743, 4.4703968352, 4.798246022, -0.33000579984, -0.51179645192, 2.0683298992, 1.5578004908, 2.9519912561, 2.3182333687, 2.6519502439, 5.0200643644, 4.3979462621, 4.697667554, -0.60852106376, 0.12819634273, 0.65851446779, 2.334157038, 1.2507354539, 2.8736625902, 2.856854623, 3.8526650288, 3.6036816207, 4.363825628, -0.73128646947, -0.87705949197, 0.2068024588, 0.8102192234, 0.49489152403, 1.5117635517, 1.391564736, 3.0038229366, 3.247422302, 3.0927705043, -4.1039807396, -2.9931419689, -3.2088313683, -2.9148890748, -2.5175390588, -2.2039553654, -1.5537585763, -1.5038679763, -0.77320784305, -0.065851868564, 1.0313142498, 2.9571659829, 2.0190895471, 3.7268996701, 4.6589797789, 4.6889558678, 4.9110222729, 5.7559852625, 5.5423346785, 6.2178089257, 2.9428533374, 4.757281362, 4.6941910741, 4.478268561, 6.180925968, 5.971211092, 6.6300126261, 7.8431354173, 7.6138885889, 8.2870484982, -4.6486500232, -4.3293962547, -3.3837629389, -3.5033584064, -2.7672857308, -1.7697210179, -1.5745726688, -0.67578078035, -0.082928399663, 0.011366511, -0.49283294835, -0.38636689804, 0.83643989751, 1.212473889, 1.5125671082, 2.7632103096, 1.7720701418, 2.4320263932, 3.9936373848, 3.4689586505, 0.91627601873, 0.98105976401, 1.6130113126, 2.2386443746, 2.7213773977, 3.7001782294, 2.8956455331, 3.8890665453, 4.5542578852, 6.1909092137, -3.1401737915, -3.3173951931, -2.614595799, -1.948952474, -1.3420523774, -0.51325227315, -0.89496354307, 0.69317522502, 0.83387600449, 1.3736895326, -1.3991665799, -0.88071166454, -1.7935199456, -1.0327456384, 0.22797073004, 0.4750719784, 0.99050757786, 0.47105082401, 2.2144708992, 1.0455693417, -0.23698467418, 1.5335503698, 2.7875031807, 1.3339827803, 2.6836804393, 4.1014726763, 4.4595980479, 4.010149303, 4.8514460328, 5.0520554245, 2.2453832707, 3.296747875, 2.5233817543, 3.2384413823, 3.5365389376, 4.7254040415, 4.8124189642, 4.4508694895, 5.6664761848, 6.6236985335, 1.0230133467, 0.76044973396, 2.5663115776, 3.2345561469, 2.9240503032, 3.1886070679, 3.9852755792, 4.7770707231, 4.9810713019, 5.6255235495, -0.90286019841, -0.076239112849, 0.3715132453, 2.1594074749, 2.5021690364, 1.8118486327, 2.2917715352, 3.1525505143, 3.5313555246, 2.9499473078, 0.50211334146, 1.242218949, 0.87806989916, 2.5472634181, 2.4091157411, 2.0542727525, 2.1935037832, 3.6388483348, 4.8883468351, 3.6319662274, -2.6163110762, -2.0306604353, -0.74956124941, -1.1605560257, 0.057877418549, 0.21359397186, 0.81043487155, 1.6319201969, 2.0516515842, 3.361952844], + "het_x": [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] + }, + "params": { + "pattern": "multi_path_reversible_by_path_predict_het", + "n_switchers": 90, + "n_controls": 30, + "n_groups": 120, + "n_periods": 10, + "seed": 120, + "effects": 3, + "by_path": 3, + "predict_het_var": "het_x", + "predict_het_horizons": [1, 2, 3], + "ci_level": 95, + "dont_drop_larger_lower": true + }, + "results": { + "by_path_predict_het": [ + { + "path": "0,1,0,0", + "frequency_rank": 1, + "horizons": { + "1": { + "beta": 2.8301870449, + "se": 0.32413011552, + "t": 8.7316386517, + "ci_lo": 2.1639280505, + "ci_hi": 3.4964460393, + "n_obs": 30, + "p_value": 3.2987315589e-09 + }, + "2": { + "beta": -0.16062645702, + "se": 0.27377503227, + "t": -0.58670966336, + "ci_lo": -0.72337909543, + "ci_hi": 0.40212618138, + "n_obs": 30, + "p_value": 0.56245884519 + }, + "3": { + "beta": -0.16188978053, + "se": 0.25819780533, + "t": -0.62699905727, + "ci_lo": -0.69262297039, + "ci_hi": 0.36884340932, + "n_obs": 30, + "p_value": 0.53612734365 + } + } + }, + { + "path": "0,1,1,0", + "frequency_rank": 2, + "horizons": { + "1": { + "beta": 3.3963706812, + "se": 0.28651602615, + "t": 11.8540338799, + "ci_lo": 2.8074285548, + "ci_hi": 3.9853128076, + "n_obs": 30, + "p_value": 5.495025198e-12 + }, + "2": { + "beta": 3.1871911509, + "se": 0.26329034164, + "t": 12.1052338305, + "ci_lo": 2.6459901027, + "ci_hi": 3.728392199, + "n_obs": 30, + "p_value": 3.4532476901e-12 + }, + "3": { + "beta": 0.46828316092, + "se": 0.27123345051, + "t": 1.7264948701, + "ci_lo": -0.089245181354, + "ci_hi": 1.0258115032, + "n_obs": 30, + "p_value": 0.096124084653 + } + } + }, + { + "path": "0,1,1,1", + "frequency_rank": 3, + "horizons": { + "1": { + "beta": 3.1122409756, + "se": 0.28295503686, + "t": 10.9990654702, + "ci_lo": 2.5306185675, + "ci_hi": 3.6938633837, + "n_obs": 30, + "p_value": 2.8157393205e-11 + }, + "2": { + "beta": 3.1939245728, + "se": 0.23480746457, + "t": 13.6023127656, + "ci_lo": 2.711270917, + "ci_hi": 3.6765782286, + "n_obs": 30, + "p_value": 2.4833856221e-13 + }, + "3": { + "beta": 2.8295623299, + "se": 0.27042368033, + "t": 10.4634413913, + "ci_lo": 2.2736984941, + "ci_hi": 3.3854261657, + "n_obs": 30, + "p_value": 8.1857621596e-11 + } + } + } + ] + } } }, "generator": "generate_reversible_did_data v1", diff --git a/diff_diff/chaisemartin_dhaultfoeuille.py b/diff_diff/chaisemartin_dhaultfoeuille.py index 55bfb883..463f5b86 100644 --- a/diff_diff/chaisemartin_dhaultfoeuille.py +++ b/diff_diff/chaisemartin_dhaultfoeuille.py @@ -26,7 +26,7 @@ """ import warnings -from typing import Any, Dict, List, Optional, Sequence, Tuple +from typing import Any, Dict, List, Optional, Sequence, Set, Tuple import numpy as np import pandas as pd @@ -468,10 +468,22 @@ class ChaisemartinDHaultfoeuille(ChaisemartinDHaultfoeuilleBootstrapMixin): ``NotImplementedError``. Top-k path ranking under ``survey_design`` remains group-cardinality-based (unweighted), not population-weight-based — survey weights do not affect - which paths are selected as "top-k". Incompatible with - ``heterogeneity``, ``design2``, and ``honest_did`` (each - combination raises ``NotImplementedError`` in the current - release). + which paths are selected as "top-k". + + Compatible with ``heterogeneity=""`` — per-path + heterogeneity coefficient is computed by re-running the + Lemma 7 regression on each path-restricted switcher + subsample. Cohort dummies absorb baseline (no R-divergence + warning needed). Surfaces on + ``results.path_heterogeneity_effects`` keyed + ``{path: {l: {beta, se, t_stat, p_value, conf_int, n_obs}}}`` + and on ``to_dataframe(level="by_path")`` via ``het_*`` + columns. Mirrors R ``did_multiplegt_dyn(..., by_path, + predict_het)`` per-by_level. Composes with ``survey_design`` + (analytical Binder TSL + replicate-weight) via the existing + cell-period IF allocator path. Incompatible with + ``design2`` and ``honest_did`` (each combination raises + ``NotImplementedError`` in the current release). Mutually exclusive with ``paths_of_interest`` — use ``by_path=k`` for top-k automatic ranking by frequency, or @@ -633,10 +645,12 @@ class ChaisemartinDHaultfoeuille(ChaisemartinDHaultfoeuilleBootstrapMixin): Compatible with all downstream surfaces inherited by ``by_path``: bootstrap, per-path placebos, per-path joint sup-t bands, ``controls``, ``trends_linear``, - ``trends_nonparam``, and ``survey_design`` (analytical Binder + ``trends_nonparam``, ``survey_design`` (analytical Binder TSL + replicate-weight; multiplier bootstrap under survey - remains gated, same as ``by_path=k``). Mechanical extension - to path enumeration; no methodology change. + remains gated, same as ``by_path=k``), and ``heterogeneity`` + (per-path heterogeneity coefficient surfaces on + ``results.path_heterogeneity_effects``). Mechanical + extension to path enumeration; no methodology change. **Order semantics**: paths appear in ``results.path_effects`` in the user-specified order, modulo @@ -952,7 +966,11 @@ def fit( Partial implementation: post-treatment regressions only (no placebo regressions or joint null test). Cannot be combined with ``controls``, ``trends_linear``, or - ``trends_nonparam``. Requires ``L_max >= 1``. + ``trends_nonparam``. Requires ``L_max >= 1``. Under + ``by_path`` / ``paths_of_interest``, per-path + heterogeneity coefficients also surface on + ``results.path_heterogeneity_effects`` and on + ``to_dataframe(level="by_path")`` via ``het_*`` columns. design2 : bool, default=False If ``True``, identify and report switch-in/switch-out (Design-2) groups. Convenience wrapper (descriptive summary, @@ -1227,11 +1245,6 @@ def fit( f"[F_g-1, ..., F_g-1+L_max]); got path " f"{p!r} of length {len(p)}." ) - if heterogeneity is not None: - raise NotImplementedError( - "by_path / paths_of_interest combined with " - "heterogeneity testing is deferred to a future release." - ) if design2: raise NotImplementedError( "by_path / paths_of_interest combined with design2 " @@ -3861,6 +3874,36 @@ def fit( replicate_n_valid_list=_replicate_n_valid_list, ) + # Per-path heterogeneity (mirrors R `did_multiplegt_dyn(..., + # by_path, predict_het)` per-by_level dispatch). Empty-state + # contract: None when not requested (no `heterogeneity` kwarg + # or no `by_path`/`paths_of_interest` selector); `{}` when + # requested but no path is observed (mirrors `path_effects`). + path_heterogeneity_effects: Optional[Dict[Tuple[int, ...], Dict[int, Dict[str, Any]]]] = ( + None + ) + if heterogeneity is not None and ( + self.by_path is not None or self.paths_of_interest is not None + ): + path_heterogeneity_effects = _compute_path_heterogeneity_test( + Y_mat=Y_het, + N_mat=N_het, + baselines=baselines, + first_switch_idx=first_switch_idx_arr, + switch_direction=switch_direction_arr, + T_g=T_g_arr, + X_het=X_het, + L_max=L_max, + by_path=self.by_path, + paths_of_interest=self.paths_of_interest, + D_mat=D_mat, + alpha=self.alpha, + rank_deficient_action=self.rank_deficient_action, + group_ids_order=np.array(all_groups), + obs_survey_info=_obs_survey_info, + replicate_n_valid_list=_replicate_n_valid_list, + ) + twfe_weights_df = None twfe_fraction_negative = None twfe_sigma_fe = None @@ -4006,6 +4049,25 @@ def fit( _info_r2["t_stat"] = _t_r2 _info_r2["p_value"] = _p_r2 _info_r2["conf_int"] = _ci_r2 + # Per-path heterogeneity (Wave 5 #11): per-(path, l) entries + # snapshot df_inference at compute-time. Refresh with final df + # so t/p/CI match `survey_metadata.df_survey`. Schema differs + # from per-path event-study (`{path: {l: ...}}` vs + # `{path: {"horizons": {l: ...}}}`), so inline loop here + # rather than reusing `_refresh_path_inference`. + if path_heterogeneity_effects: + for _path_r2, _horizons_r2 in list(path_heterogeneity_effects.items()): + for _l_r2, _info_r2 in list(_horizons_r2.items()): + if np.isfinite(_info_r2["se"]): + _t_r2, _p_r2, _ci_r2 = safe_inference( + _info_r2["beta"], + _info_r2["se"], + alpha=self.alpha, + df=_final_inf_df, + ) + _info_r2["t_stat"] = _t_r2 + _info_r2["p_value"] = _p_r2 + _info_r2["conf_int"] = _ci_r2 # Normalized effects: another public surface built with the # pre-heterogeneity `_df_survey`. Recompute inference with # the final df so t/p/CI match the other surfaces (and the @@ -4116,6 +4178,7 @@ def fit( linear_trends_effects=linear_trends_effects, trends_linear=_is_trends_linear, heterogeneity_effects=heterogeneity_effects, + path_heterogeneity_effects=path_heterogeneity_effects, design2_effects=( _compute_design2_effects( D_mat=D_mat, @@ -4828,6 +4891,7 @@ def _compute_heterogeneity_test( group_ids_order: Optional[np.ndarray] = None, obs_survey_info: Optional[Dict[str, Any]] = None, replicate_n_valid_list: Optional[List[int]] = None, + path_groups: Optional[Set[int]] = None, ) -> Dict[int, Dict[str, Any]]: """Test for heterogeneous treatment effects (Web Appendix Section 1.5). @@ -4959,6 +5023,8 @@ def _compute_heterogeneity_test( cohort_keys = [] for g in range(n_groups): + if path_groups is not None and g not in path_groups: + continue f_g = first_switch_idx[g] if f_g < 0: continue # never-switcher @@ -6390,6 +6456,85 @@ def _compute_path_placebos( return path_placebos +def _compute_path_heterogeneity_test( + Y_mat: np.ndarray, + N_mat: np.ndarray, + baselines: np.ndarray, + first_switch_idx: np.ndarray, + switch_direction: np.ndarray, + T_g: np.ndarray, + X_het: np.ndarray, + L_max: int, + by_path: Optional[int], + paths_of_interest: Optional[List[Tuple[int, ...]]], + D_mat: np.ndarray, + alpha: float = 0.05, + rank_deficient_action: str = "warn", + group_ids_order: Optional[np.ndarray] = None, + obs_survey_info: Optional[Dict[str, Any]] = None, + replicate_n_valid_list: Optional[List[int]] = None, +) -> Optional[Dict[Tuple[int, ...], Dict[int, Dict[str, Any]]]]: + """Per-path heterogeneity test (Web Appendix Section 1.5, Lemma 7). + + For each selected path ``p``, runs ``_compute_heterogeneity_test`` on + the path-restricted switcher subsample. Cohort dummies absorb baseline + by construction, so the path-restricted regression is methodologically + well-posed even when path switchers span multiple baselines. + + Mirrors R ``did_multiplegt_dyn(..., by_path, predict_het)`` semantics: + the R per-path dispatcher re-runs ``did_multiplegt_main(..., + predict_het=...)`` on each path-restricted subsample, which is exactly + what this helper does in Python. + + The ``_enumerate_treatment_paths`` call here re-derives the path + enumeration (already computed elsewhere in fit() for ``path_effects``). + The call is wrapped in ``warnings.catch_warnings()`` to suppress + duplicate unobserved-path / by_path-exceeds-observed warnings; the + upstream ``_compute_path_effects`` call already surfaced them. + + Returns + ------- + Dict[Tuple[int, ...], Dict[int, Dict[str, Any]]] + ``{path: {l: {beta, se, t_stat, p_value, conf_int, n_obs}}}``. + Returns ``{}`` if ``selected_paths`` is empty. Return type is + ``Optional[...]`` for caller-contract symmetry; the helper itself + never produces ``None`` (the empty-state distinction + ``None`` not requested vs ``{}`` requested but empty lives at + the caller site in ``fit()``). + """ + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + selected_paths, path_to_group_mask, _ = _enumerate_treatment_paths( + D_mat=D_mat, + first_switch_idx=first_switch_idx, + N_mat=N_mat, + L_max=L_max, + by_path=by_path, + paths_of_interest=paths_of_interest, + ) + out: Dict[Tuple[int, ...], Dict[int, Dict[str, Any]]] = {} + for path in selected_paths: + mask = path_to_group_mask[path] + path_groups: Set[int] = {int(g) for g in np.flatnonzero(mask)} + out[path] = _compute_heterogeneity_test( + Y_mat=Y_mat, + N_mat=N_mat, + baselines=baselines, + first_switch_idx=first_switch_idx, + switch_direction=switch_direction, + T_g=T_g, + X_het=X_het, + L_max=L_max, + alpha=alpha, + rank_deficient_action=rank_deficient_action, + group_ids_order=group_ids_order, + obs_survey_info=obs_survey_info, + replicate_n_valid_list=replicate_n_valid_list, + path_groups=path_groups, + ) + return out + + def _collect_path_bootstrap_inputs( D_mat: np.ndarray, Y_mat: np.ndarray, diff --git a/diff_diff/chaisemartin_dhaultfoeuille_results.py b/diff_diff/chaisemartin_dhaultfoeuille_results.py index 9d907c83..b32ffd21 100644 --- a/diff_diff/chaisemartin_dhaultfoeuille_results.py +++ b/diff_diff/chaisemartin_dhaultfoeuille_results.py @@ -424,6 +424,20 @@ class ChaisemartinDHaultfoeuilleResults: cohort-sharing SE deviation from R documented for ``path_effects``. See REGISTRY.md ``Note (Phase 3 by_path ...)`` → "Per-path placebos". + path_heterogeneity_effects : dict, optional + Per-path heterogeneity test results (Web Appendix Section 1.5, + Lemma 7) when ``heterogeneity`` is set AND (``by_path=k`` or + ``paths_of_interest=[(...), ...]``) is set. Inner dict keyed by + horizon directly (no ``"horizons"`` wrapper); each entry holds + ``{"beta", "se", "t_stat", "p_value", "conf_int", "n_obs"}``, + where ``beta`` is the WLS coefficient on the heterogeneity + covariate on the path-restricted switcher subsample. Cohort + dummies in the design matrix absorb baseline by construction. + Empty-state contract mirrors ``path_effects``: ``None`` when not + requested; ``{}`` when requested but no path has eligible + switchers. Mirrors R ``did_multiplegt_dyn(..., by_path, + predict_het)`` per-by_level dispatch. See REGISTRY.md + ``Note (Phase 3 by_path ...)`` → "Per-path heterogeneity testing". path_cumulated_event_study : dict, optional Per-path cumulated level effects ``delta_{path, l} = sum_{l'=1..l} DID^{fd}_{path, l'}`` for ``l = 1..L_max``, @@ -580,6 +594,18 @@ class ChaisemartinDHaultfoeuilleResults: path_placebo_event_study: Optional[Dict[Tuple[int, ...], Dict[int, Dict[str, Any]]]] = field( default=None, repr=False ) + # Per-path heterogeneity test (Web Appendix Section 1.5, Lemma 7) + # under `by_path` / `paths_of_interest`. Inner dict keyed by horizon + # directly: `{path: {l: {beta, se, t_stat, p_value, conf_int, n_obs}}}`. + # Mirrors the simpler `path_placebo_event_study` shape — no metadata + # wrapper because frequency_rank / n_groups already live on + # `path_effects[path]` for the same path. Empty-state contract + # mirrors `path_effects`: None when not requested (no `heterogeneity` + # kwarg or no `by_path` / `paths_of_interest` selector); `{}` when + # requested but no path is observed. + path_heterogeneity_effects: Optional[Dict[Tuple[int, ...], Dict[int, Dict[str, Any]]]] = field( + default=None, repr=False + ) # Per-path cumulated event study (level effects under `trends_linear` # = True). `path_effects[path]["horizons"][l]` surfaces raw # `DID^{fd}_l` per path; this field surfaces the cumulated level @@ -1392,6 +1418,29 @@ def _render_path_effects_section( ce["p_value"], ) ) + # Per-path heterogeneity rows (under heterogeneity=col). + # Mirrors the global `_render_heterogeneity_section` block + # but scoped to this path. Skip silently when + # path_heterogeneity_effects is None or this path lacks an + # entry (e.g., when `heterogeneity` was not requested). + if ( + self.path_heterogeneity_effects is not None + and path in self.path_heterogeneity_effects + ): + het_horizons = self.path_heterogeneity_effects[path] + if het_horizons: + lines.append(" Heterogeneity Test (Section 1.5, partial):") + for l_h in sorted(het_horizons.keys()): + het = het_horizons[l_h] + lines.append( + _format_inference_row( + f" l={l_h}", + het["beta"], + het["se"], + het["t_stat"], + het["p_value"], + ) + ) # Per-path joint sup-t critical value (when populated). # Mirrors the OVERALL sup-t crit print at line ~1019. if self.path_sup_t_bands is not None and path in self.path_sup_t_bands: @@ -1482,8 +1531,10 @@ def to_dataframe(self, level: str = "overall") -> pd.DataFrame: ``effect``, ``se``, ``t_stat``, ``p_value``, ``conf_int_lower``, ``conf_int_upper``, ``n_obs``, ``cband_lower``, ``cband_upper``, ``cumulated_effect``, - ``cumulated_se``. The ``horizon`` column takes negative - ints for placebo rows when ``placebo=True``. The + ``cumulated_se``, ``het_beta``, ``het_se``, + ``het_t_stat``, ``het_p_value``, ``het_conf_int_lower``, + ``het_conf_int_upper``. The ``horizon`` column takes + negative ints for placebo rows when ``placebo=True``. The ``cband_*`` columns mirror the OVERALL ``level="event_study"`` schema (joint sup-t simultaneous bands); they are populated for positive-horizon rows of @@ -1495,7 +1546,13 @@ def to_dataframe(self, level: str = "overall") -> pd.DataFrame: positive-horizon rows when ``trends_linear=True`` is also set, NaN for placebo rows or non-trends_linear fits (always-present, NaN-when-None — same convention as - ``cband_*``). + ``cband_*``). The ``het_*`` columns surface the per-path + heterogeneity coefficient (Web Appendix Section 1.5, + Lemma 7) when ``heterogeneity=""`` is also set; + populated for positive-horizon rows and NaN for placebo + rows / non-heterogeneity fits / the requested-but-empty + fallback DataFrame (always-present, NaN-when-None — same + convention as ``cband_*`` and ``cumulated_*``). Returns ------- @@ -1759,6 +1816,12 @@ def to_dataframe(self, level: str = "overall") -> pd.DataFrame: "cband_upper", "cumulated_effect", "cumulated_se", + "het_beta", + "het_se", + "het_t_stat", + "het_p_value", + "het_conf_int_lower", + "het_conf_int_upper", ] ) rows = [] @@ -1788,6 +1851,14 @@ def to_dataframe(self, level: str = "overall") -> pd.DataFrame: if self.path_cumulated_event_study is not None else {} ) + # Per-path heterogeneity entries (under heterogeneity=col). + # Always-present het_* columns, NaN when not requested or + # when the path's per-horizon entry is missing. + path_het = ( + self.path_heterogeneity_effects.get(path, {}) + if self.path_heterogeneity_effects is not None + else {} + ) for lag_key in sorted(placebo_horizons.keys()): ph_entry = placebo_horizons[lag_key] # Placebos do not get joint sup-t bands in this @@ -1816,6 +1887,15 @@ def to_dataframe(self, level: str = "overall") -> pd.DataFrame: "cband_upper": ph_cband[1] if ph_cband else np.nan, "cumulated_effect": np.nan, "cumulated_se": np.nan, + # Heterogeneity is forward-only (R doesn't ship + # per-path predict_het on placebos); placebo + # rows always emit NaN here. + "het_beta": np.nan, + "het_se": np.nan, + "het_t_stat": np.nan, + "het_p_value": np.nan, + "het_conf_int_lower": np.nan, + "het_conf_int_upper": np.nan, } ) for l_h in sorted(horizons.keys()): @@ -1826,6 +1906,8 @@ def to_dataframe(self, level: str = "overall") -> pd.DataFrame: # `TestByPathSupTBands::test_path_sup_t_to_dataframe_emits_cband_columns`. h_cband = h_entry.get("cband_conf_int", (np.nan, np.nan)) cum_entry = path_cumulated.get(l_h, {}) + het_entry = path_het.get(l_h, {}) if path_het else {} + het_ci = het_entry.get("conf_int", (np.nan, np.nan)) rows.append( { "path": path, @@ -1843,6 +1925,16 @@ def to_dataframe(self, level: str = "overall") -> pd.DataFrame: "cband_upper": h_cband[1] if h_cband else np.nan, "cumulated_effect": cum_entry.get("effect", np.nan), "cumulated_se": cum_entry.get("se", np.nan), + # Per-path heterogeneity (Wave 5 #11). Always- + # present, NaN when not requested or when the + # entry is missing (mirrors cband_*/cumulated_* + # convention). + "het_beta": het_entry.get("beta", np.nan), + "het_se": het_entry.get("se", np.nan), + "het_t_stat": het_entry.get("t_stat", np.nan), + "het_p_value": het_entry.get("p_value", np.nan), + "het_conf_int_lower": het_ci[0] if het_ci else np.nan, + "het_conf_int_upper": het_ci[1] if het_ci else np.nan, } ) return pd.DataFrame(rows) diff --git a/diff_diff/guides/llms-full.txt b/diff_diff/guides/llms-full.txt index 6245f62b..e3eaa251 100644 --- a/diff_diff/guides/llms-full.txt +++ b/diff_diff/guides/llms-full.txt @@ -242,8 +242,8 @@ ChaisemartinDHaultfoeuille( placebo: bool = True, # Auto-compute single-lag placebo twfe_diagnostic: bool = True, # Auto-compute Theorem 1 TWFE decomposition drop_larger_lower: bool = True, # Drop multi-switch groups (matches R DIDmultiplegtDYN) - by_path: int | None = None, # Top-k per-path event study; requires drop_larger_lower=False, L_max>=1; supports binary or integer-coded discrete D (D in Z); composes with survey_design (analytical TSL + replicate-weight; multiplier bootstrap n_bootstrap>0 still gated under survey); mutex with paths_of_interest - paths_of_interest: list[tuple[int, ...]] | None = None, # User-specified path subset, alternative to by_path=k (Python-only API; mutex with by_path; composes with survey_design same as by_path=k) + by_path: int | None = None, # Top-k per-path event study; requires drop_larger_lower=False, L_max>=1; supports binary or integer-coded discrete D (D in Z); composes with survey_design (analytical TSL + replicate-weight; multiplier bootstrap n_bootstrap>0 still gated under survey) and heterogeneity (per-path predict_het, mirrors R did_multiplegt_dyn(..., by_path, predict_het)); mutex with paths_of_interest + paths_of_interest: list[tuple[int, ...]] | None = None, # User-specified path subset, alternative to by_path=k (Python-only API; mutex with by_path; composes with survey_design and heterogeneity same as by_path=k) rank_deficient_action: str = "warn", # Used by TWFE diagnostic OLS ) ``` diff --git a/docs/api/chaisemartin_dhaultfoeuille.rst b/docs/api/chaisemartin_dhaultfoeuille.rst index cb5b3895..10c3533d 100644 --- a/docs/api/chaisemartin_dhaultfoeuille.rst +++ b/docs/api/chaisemartin_dhaultfoeuille.rst @@ -26,7 +26,11 @@ supports binary or integer-coded discrete (D in Z) treatment, and composes with ``survey_design`` for analytical Binder TSL SE and replicate-weight bootstrap variance (multiplier bootstrap under survey + by_path remains gated; no R parity since R -``did_multiplegt_dyn`` does not support survey weighting). +``did_multiplegt_dyn`` does not support survey weighting). ``by_path`` +and ``paths_of_interest`` also compose with ``heterogeneity=""``: +per-path heterogeneity coefficient surfaces on +``results.path_heterogeneity_effects`` (mirrors R +``did_multiplegt_dyn(..., by_path, predict_het)`` per-by_level). The estimator: diff --git a/docs/methodology/REGISTRY.md b/docs/methodology/REGISTRY.md index 076632d0..596c97df 100644 --- a/docs/methodology/REGISTRY.md +++ b/docs/methodology/REGISTRY.md @@ -640,7 +640,9 @@ The guard is fired by `_survey_se_from_group_if` (analytical and replicate) and - **Note (Phase 3 Design-2 switch-in/switch-out):** Convenience wrapper for Web Appendix Section 1.6 (Assumption 16). Identifies groups with exactly 2 treatment changes (join then leave), reports switch-in and switch-out mean effects. This is a descriptive summary, not a full re-estimation with specialized control pools as described in the paper. **Always uses raw (unadjusted) outcomes** regardless of active `controls`, `trends_linear`, or `trends_nonparam` options - those adjustments apply to the main estimator surface but not to the Design-2 descriptive block. For full adjusted Design-2 estimation with proper control pools, the paper recommends "running the command on a restricted subsample and using `trends_nonparam` for the entry-timing grouping." Activated via `design2=True` in `fit()`, requires `drop_larger_lower=False` to retain 2-switch groups. -- **Note (Phase 3 `by_path` per-path event-study disaggregation):** Per-path disaggregation of the multi-horizon event study, mirroring R `did_multiplegt_dyn(..., by_path=k)`. Activated via `ChaisemartinDHaultfoeuille(by_path=k, drop_larger_lower=False)` where `k` is a positive integer (top-k most common observed paths by switcher-group frequency). **Window convention:** the path tuple for a switcher group `g` is `(D_{g, F_g-1}, D_{g, F_g}, ..., D_{g, F_g-1+L_max})` — length `L_max + 1`, matching R's window `[F_{g-1}, F_{g-1+l}]`. **Ranking:** paths are ranked by descending frequency; ties are broken lexicographically on the path tuple for deterministic ordering, so every selected path has a unique `frequency_rank`. If `by_path` exceeds the number of observed paths, all observed paths are returned with a `UserWarning`. **Per-path SE convention (joiners/leavers precedent):** the per-path influence function follows the joiners-only / leavers-only IF construction at `chaisemartin_dhaultfoeuille.py:5495-5504`: the switcher-side contribution `+S_g * (Y_{g,out} - Y_{g,ref})` is zeroed for groups whose observed trajectory is NOT the selected path; control contributions and the full cohort structure `(D_{g,1}, F_g, S_g)` are unchanged. After applying the singleton-baseline eligible mask and cohort-recentering with the original cohort IDs, the plug-in SE uses the path-specific divisor `N_l_path` (count of path switchers eligible at horizon `l`) — same pattern as `joiners_se` using `joiner_total`. This gives the **within-path mean** estimand `DID_{path,l}` as the within-path average of `DID_{g,l}`. **Degenerate-cohort behavior per path:** when a path's centered IF at some horizon is identically zero (every variance-eligible path switcher forms its own `(D_{g,1}, F_g, S_g)` cohort, or the path has a single contributing group), SE / t_stat / p_value / conf_int are NaN-consistent and a `UserWarning` is emitted scoped to `(path, horizon)`. This mirrors the overall-path degenerate-cohort surface and is common for rare paths with few contributing groups. **Empty-state contract:** `results.path_effects` distinguishes "not requested" (`None`) from "requested but empty" (`{}` — all switchers have windows outside the panel or unobserved cells). The empty-dict case emits a `UserWarning` at fit-time and renders as an explicit "no observed paths" notice in `summary()`; `to_dataframe(level="by_path")` returns an empty DataFrame with the canonical column set (mirrors the `linear_trends` pattern when `trends_linear=True` but no horizons survive). **Requirements:** `drop_larger_lower=False` (multi-switch groups are the object of interest; default `True` filters them out) and `L_max >= 1` (path window depends on the horizon). **Scope:** combinations with `heterogeneity`, `design2`, and `honest_did` remain gated behind explicit `NotImplementedError` (deferred to follow-up wave PRs). `n_bootstrap > 0` is now supported — see the **Bootstrap SE** paragraph below. `survey_design` is supported under analytical Binder TSL and replicate-weight bootstrap — see the **Per-path survey-design SE** paragraph below; multiplier bootstrap (`n_bootstrap > 0`) under `survey_design + by_path/paths_of_interest` remains gated. `placebo=True` is now supported per-path — see the **Per-path placebos** paragraph below. **TWFE diagnostic** remains a sample-level summary (not computed per path) in this release. Results are exposed on `results.path_effects` as `Dict[Tuple[int, ...], Dict[str, Any]]` with nested `horizons` dicts per horizon `l`, and on `results.to_dataframe(level="by_path")` as a long-format table with columns `[path, frequency_rank, n_groups, horizon, effect, se, t_stat, p_value, conf_int_lower, conf_int_upper, n_obs, cband_lower, cband_upper, cumulated_effect, cumulated_se]` (the `cband_*` columns are added by the joint sup-t Note below, populated for positive-horizon rows of paths with a finite sup-t crit and NaN otherwise; the `cumulated_*` columns are added by the per-path linear-trends Note below, populated for positive-horizon rows when `trends_linear=True` is set and NaN otherwise). Gated tests live in `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathGates` / `::TestByPathBehavior` / `::TestByPathEdgeCases`. **R-parity** against `DIDmultiplegtDYN 2.3.3` is confirmed at `tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityByPath` via two scenarios: `mixed_single_switch_by_path` (2 paths, `by_path=2`) and `multi_path_reversible_by_path` (4 paths, `by_path=3`; path-assignment deterministic on `F_g` so each `(D_{g,1}, F_g, S_g)` cohort contains switchers from a single path). Per-path point estimates and per-path switcher counts match R exactly; per-path SE matches within the Phase 2 multi-horizon SE envelope (observed rtol ≤ 10.2% on the 2-path mixed scenario, ≤ 4.2% on the 4-path cohort-clean scenario). **Deviation from R (cross-path cohort-sharing SE):** our analytical SE is the marginal variance of the path-contribution estimator cohort-centered on the *full-panel* cohort structure (joiners/leavers precedent — non-path switchers contribute to cohort means via their zeroed switcher row). R's `did_multiplegt_dyn(..., by_path=k)` re-runs the estimator per path, so cohort means are computed over the path's own switchers only. When a cohort `(D_{g,1}, F_g, S_g)` spans multiple observed paths, Python and R SE diverge materially (our empirical probes with random post-window toggling saw rtol > 100%); when every cohort is single-path (scenario 13 by design, scenario 14 by construction), the two approaches coincide up to the documented Phase 2 envelope. Practitioners with cohort structures that mix paths should interpret the per-path SE as a within-full-panel marginal variance, not a per-path conditional variance. **Bootstrap SE:** when `n_bootstrap > 0` is set, the top-k paths are enumerated once on the observed data (R-faithful: matches `did_multiplegt_dyn(..., by_path=k, bootstrap=B)`'s path-stability convention — verified empirically against DIDmultiplegtDYN 2.3.3) and the multiplier bootstrap (`bootstrap_weights ∈ {"rademacher", "mammen", "webb"}`) runs per `(path, horizon)` target via the shared `_bootstrap_one_target` / `compute_effect_bootstrap_stats` helpers. Point estimates are unchanged from the analytical path. Bootstrap SE replaces the analytical SE in `path_effects[path]["horizons"][l]["se"]`, and `p_value` / `conf_int` are taken as the **bootstrap percentile** statistics, matching the Round-10 library convention for overall / joiners / leavers / multi-horizon bootstrap (see the `Note (bootstrap inference surface)` elsewhere in this file and the pinned regression `test_bootstrap_p_value_and_ci_propagated_to_top_level`). `t_stat` is SE-derived via `safe_inference` per the anti-pattern rule. Interpretation: inference is *conditional on the observed path set*. **SE inherits the analytical cross-path cohort-sharing deviation:** the bootstrap input is the exact same full-panel cohort-centered path IF that the analytical path computes (`_collect_path_bootstrap_inputs` reuses the same enumeration / cohort IDs / IF construction), so the bootstrap SE is a Monte Carlo analog of the analytical SE — it inherits the same cross-path cohort-sharing deviation from R's per-path re-run convention documented above. On single-path-cohort panels (scenarios 13 and 14 of the R-parity fixture, and any DGP where `(D_{g,1}, F_g, S_g)` cohorts never span multiple observed paths), bootstrap SE tracks analytical SE up to Monte Carlo noise and both coincide with R up to the Phase 2 envelope. On cross-path cohort panels, bootstrap SE inherits the >100% rtol divergence from R that analytical already has. **Deviation from R (CI method):** R's per-path CI is normal-theory around the bootstrap SE (half-width ≈ `1.96·se`); ours is the bootstrap percentile CI, intentionally diverging from R to keep the dCDH inference surface internally consistent across all bootstrap targets. Practitioners who want *unconditional* inference capturing path-selection uncertainty need a pairs-bootstrap (deferred — no R precedent). Positive regressions live in `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathBootstrap` (gated `@pytest.mark.slow`): point-estimate invariance, finite positive SE on non-degenerate panels, SE-within-30%-rtol of analytical on cohort-clean fixtures, degenerate-cohort NaN propagation, Rademacher/Mammen/Webb parity, seed reproducibility, and percentile-vs-normal-theory CI pinning. **Per-path placebos:** when `placebo=True` (and `L_max >= 1`) is combined with `by_path=k`, per-path backward-horizon placebos `DID^{pl}_{path, l}` for `l = 1..L_max` are computed using the same joiners/leavers IF precedent applied to `_compute_per_group_if_placebo_horizon` (with the new `switcher_subset_mask` parameter): switcher contributions are zeroed for groups not in the path; the control pool and the variance-eligible cohort structure `(D_{g,1}, F_g, S_g)` are unchanged. Plug-in SE uses the path-specific divisor `N^{pl}_{l, path}` (count of path switchers eligible at backward lag `l`). Surfaced on `results.path_placebo_event_study[path][-l]` with the same `{effect, se, t_stat, p_value, conf_int, n_obs}` shape as `placebo_event_study` (negative-int inner keys parallel the existing per-path event-study positive-int keys, so a unified forward+backward view is well-formed). **Inherits the cross-path cohort-sharing SE deviation from R** documented above for `path_effects` (same convention applied backward); tracks R within numerical tolerance on single-path-cohort panels and diverges on cohort-mixed panels. Multiplier bootstrap (when `n_bootstrap > 0`) runs per `(path, lag)` target via the same `_bootstrap_one_target` dispatch used for the per-path event-study, with the canonical NaN-on-invalid contract. The bootstrap SE is a Monte Carlo analog of the analytical placebo SE — same per-path centered IF input — and inherits the same deviation. Surfaced through `summary()` (negative-keyed rows rendered alongside positive-keyed event-study rows under each path block) and `to_dataframe(level="by_path")` (`horizon` column takes negative ints for placebo rows). **Empty-state contract:** `results.path_placebo_event_study` mirrors `path_effects` — `None` when `by_path + placebo` was not requested, `{}` when requested but no observed path has a complete window within the panel (same regime that returns `{}` for `path_effects`, with the same fit-time `UserWarning`). R-parity is confirmed at `tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityByPathPlacebo` on the `multi_path_reversible_by_path_placebo` scenario; positive analytical + bootstrap invariants live in `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathPlacebo` (with the gated `::TestByPathPlacebo::TestBootstrap` subclass). **Per-path covariate residualization (DID^X):** when `controls=[...]` is set with `by_path=k`, the per-baseline OLS residualization (Web Appendix Section 1.2) runs once on the first-differenced outcome BEFORE path enumeration. All four downstream surfaces — analytical per-path SE, bootstrap SE, per-path placebos, and per-path joint sup-t bands — consume the residualized `Y_mat` automatically (Frisch-Waugh-Lovell). Per-period effects remain unadjusted, consistent with the existing `controls` + per-period DID contract (per-period DID does not support residualization). Failed-stratum baselines (rank-deficient X) zero out `N_mat` for affected groups, which the path enumeration treats as ineligible per its existing convention. **Deviation from R on multi-baseline switcher panels (point estimates):** R `did_multiplegt_dyn(..., by_path, controls)` re-runs the per-baseline residualization on each path's restricted subsample (`R/R/did_multiplegt_dyn.R` lines 401-405: rows of the path's switchers OR rows where `yet_to_switch=1 AND baseline matches the path's baseline`). The first-stage residualization sample R uses for path B equals: pre-switch rows of all switchers with matching baseline + all rows of never-switchers with matching baseline — bit-identical to our global first-stage sample under single-baseline switcher panels (every switcher shares the same `D_{g,1}`, regardless of how `F_g` or path identity varies across switchers). Per-path point estimates therefore coincide with R on those panels up to the existing **DID^X first-stage cell-weighting deviation** documented above in `Note (Phase 3 DID^X covariate adjustment)` (Python's first-stage OLS uses equal cell weights — one observation per `(g, t)` cell, consistent with the library's cell-aggregated input convention; R weights by `N_gt`). On panels with one observation per `(g, t)` cell (the common case after the cell-aggregation step in `fit()`), Python matches R bit-exactly: the `multi_path_reversible_by_path_controls` parity fixture has 4 paths with switcher `F_g` values spanning [0..6] under `D_{g,1}=0` and Python matches R to rtol ~1e-11. On multi-baseline switcher panels (some switchers have `D_{g,1}=0`, others have `D_{g,1}=1`) R's per-path subset drops switchers whose baseline differs from the path's baseline, so the per-baseline regression coefficients diverge per path under R and point estimates can diverge between Python and R — a `UserWarning` is emitted at fit-time when this configuration is detected so practitioners do not silently consume estimates that disagree with R. The warning filters to switcher groups only; never-switchers (never-treated + always-treated controls) at multiple baseline values do NOT trigger the warning because they don't affect R's per-path subset construction. **Inherits the cross-path cohort-sharing SE deviation from R** documented above for `path_effects` — bootstrap SE, placebo SE, and sup-t crit are Monte Carlo / joint-distribution analogs of the same residualized analytical IF and carry the same deviation. R-parity is confirmed against `did_multiplegt_dyn(..., by_path=3, controls="X1")` at `tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityByPathControls` on the `multi_path_reversible_by_path_controls` scenario (single-baseline DGP, exact point-estimate match measured rtol ~1e-11); cross-surface inheritance and the multi-baseline warning are regression-tested at `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathControls` (analytical + bootstrap + placebo + sup-t + `to_dataframe(level="by_path")` cband columns + multi-baseline `UserWarning`). **Per-path linear-trends DID^{fd}:** when `trends_linear=True` is set with `by_path=k`, the first-differencing transform at `chaisemartin_dhaultfoeuille.py:1599-1630` runs once globally BEFORE path enumeration (replaces `Y_mat` with `Z_mat = Y_t - Y_{t-1}` and shrinks the time axis by one), so per-path raw second-differences `DID^{fd}_{path, l}` surface on `path_effects[path]["horizons"][l]` automatically. Per-path cumulated level effects `delta_{path, l} = sum_{l'=1..l} DID^{fd}_{path, l'}` (the quantity R returns under `did_multiplegt_dyn(..., by_path, trends_lin)` per the existing parity test pivot at `tests/test_chaisemartin_dhaultfoeuille_parity.py:403-409`) surface on the new `results.path_cumulated_event_study[path][l]` field — a per-group running sum of `DID^{fd}_{g, l'}` averaged over the path's switchers eligible at horizon `l`, mirroring the global `linear_trends_effects` cumulation logic at `chaisemartin_dhaultfoeuille.py:3340-3398`. SE on the cumulated layer is the conservative upper bound (sum of per-horizon component SEs from `path_effects[path]["horizons"][l]["se"]`, NaN-consistent: any non-finite component yields a NaN cumulated SE). **Post-bootstrap recomputation:** the cumulated layer is built AFTER the bootstrap propagation block at `chaisemartin_dhaultfoeuille.py:3034-3081` so it reads the FINAL post-bootstrap per-horizon SEs (mirrors the global `linear_trends_effects` placement). When `n_bootstrap > 0`, cumulated SE / t / p / CI are derived from bootstrap per-horizon SEs; when bootstrap produces non-finite SE (e.g., `n_bootstrap=1` degenerate distribution), the cumulated layer's full inference tuple is NaN per the library-wide NaN-on-invalid bootstrap contract. `to_dataframe(level="by_path")` exposes `cumulated_effect` and `cumulated_se` columns (always present, NaN-when-None — mirrors the `cband_*` always-present convention from PR #374). `summary()` renders a `Cumulated Level Effects (DID^{fd}, trends_linear)` sub-section under each per-path block. **Path enumeration uses the post-first-differenced `N_mat_fd`**: switchers with `F_g==2` fail the window-eligibility check and are dropped from path enumeration entirely (the existing global `F_g >= 3` warning at line 1620 surfaces the issue), so a path whose switchers all have `F_g < 3` is silently absent from `path_effects` rather than present-with-NaN. **F_g=3 boundary-case divergence (`by_path + trends_linear`):** `F_g=3` switchers have exactly 2 pre-switch periods, which after first-differencing and the `time==1` filter leaves only 1 valid pre-window Z value. R's per-path full-pipeline call handles this single-pre-period regime differently from Python's global-then-disaggregate architecture, producing 30%+ relative divergence on point estimates for paths whose switchers include `F_g=3` (empirically observed on the parity fixture's earlier `F_g=3` variant). A separate `UserWarning` fires at fit-time when the panel includes any `F_g=3` switcher AND `by_path + trends_linear` is set, mirroring the `F_g < 3` exclusion warning. The shipped parity fixture (`single_baseline_multi_path_by_path_trends_lin`) restricts to `F_g >= 4` exclusively to avoid this regime; per-path R parity is asserted only there. **Placebo under `trends_linear` returns RAW per-horizon values** (no per-path placebo cumulation surface) — verified empirically against the existing `joiners_only_trends_lin` parity fixture: R's per-path Placebo_l matches Python's `path_placebo_event_study[path][-l]` (raw) bit-exactly under non-`by_path` trends_lin. **Deviation from R on multi-baseline switcher panels (point estimates):** R `did_multiplegt_dyn(..., by_path, trends_lin)` re-runs the full pipeline (including first-differencing) on each path's restricted subsample, so it operates on different switcher samples per path when switchers have different baseline values `D_{g,1}`. Python first-differences once globally before path enumeration. On single-baseline switcher panels the two architectures coincide; on multi-baseline switcher panels per-path point estimates can diverge — a `UserWarning` is emitted at fit-time when this configuration is detected so practitioners do not silently consume estimates that disagree with R (mirroring the analogous `by_path + controls` warning). Per-path R parity is confirmed against `did_multiplegt_dyn(..., by_path=3, trends_lin=TRUE, placebo=1)` at `tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityByPathTrendsLinear` on the `single_baseline_multi_path_by_path_trends_lin` scenario (single-baseline + cohort-single-path + `F_g >= 4` DGP designed to eliminate the multi-baseline divergence, the cross-path cohort-sharing deviation, and the F_g=3 boundary case under R's per-path full-pipeline call). Per-path cumulated point estimates match R bit-exactly (rtol ~1e-9) on event horizons under those conditions; cumulated SE_RTOL is widened to `0.20` (vs `0.12` used for non-cumulated by_path parity) because the conservative upper-bound SE compounds the cross-path cohort-sharing deviation under summation. **Placebo parity is intentionally skipped for `trends_linear`**: R's per-path placebo computation re-runs on the path-restricted subsample with different control eligibility than Python's global-then-disaggregate architecture surfaces, producing a sign-and-magnitude divergence on paths whose switchers have minimal pre-window depth (e.g., `F_g=4` switchers). Placebo under `by_path + trends_linear` is exercised via internal regression in `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathTrendsLinear` (finite values, bootstrap inheritance) but not pinned to R bit-by-bit. Cross-surface invariants (analytical + bootstrap + placebo + sup-t + `path_cumulated_event_study` + `to_dataframe` columns + `summary()` rendering) are regression-tested at `TestByPathTrendsLinear`. **Per-path state-set trends:** when `trends_nonparam="state_col"` is set with `by_path=k`, the set membership column is validated and stored once globally as `set_ids_arr` (time-invariance, NaN rejection, partition-coarseness checks unchanged from the non-by_path path). The `set_ids` parameter is threaded through the four per-path IF helpers (`_compute_path_effects`, `_compute_path_placebos`, `_collect_path_bootstrap_inputs`, `_collect_path_placebo_bootstrap_inputs`) so per-path analytical SE, bootstrap, placebos, and sup-t bands all consume the set-restricted control pool automatically. R does NOT first-difference and does NOT cumulate under `trends_nonparam` (unlike `trends_lin`); per-horizon `Effect_l` is a normal DID with set-restricted controls. Per-path R parity is confirmed against `did_multiplegt_dyn(..., by_path=3, trends_nonparam="state", placebo=1)` at `tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityByPathTrendsNonparam` on the `multi_path_reversible_by_path_trends_nonparam` scenario; per-path point estimates AND placebos match R bit-exactly (rtol ~1e-9), per-path SE matches within the Phase 2 envelope (~13% rtol observed). Cross-surface invariants are regression-tested at `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathTrendsNonparam`. **Per-path non-binary treatment:** integer-coded discrete treatment (D in Z, e.g. ordinal {0, 1, 2}) is supported under `by_path=k` and `paths_of_interest`. Path tuples become integer-state tuples (`(0, 2, 2, 2)`) keyed bit-for-bit against R's comma-separated path strings (`"0,2,2,2"`) for D in {0..9}. Continuous D (e.g. `1.5`) raises `ValueError` at fit-time per the no-silent-failures contract — the existing `int(round(float(v)))` cast in `_enumerate_treatment_paths` is now defensive (no-op for integer-coded D). **Deviation from R for D >= 10:** R's `did_multiplegt_by_path` derives the per-path baseline via `path_index$baseline_XX <- substr(path_index$path, 1, 1)` (extracted 2026-05-03 via `Rscript -e 'cat(paste(deparse(DIDmultiplegtDYN:::did_multiplegt_by_path), collapse="\n"))'`), capturing only the first character of the comma-separated path string. For D >= 10 this captures `"1"` instead of `"12"` for `path = "12,12,..."`, mis-allocating R's per-path control-pool subset. Python's tuple-key matching is correct in this regime; the per-path point estimates we compute are correct, R's per-path subset for the same path is buggy. The shipped parity scenario stays in `D in {0, 1, 2}` to avoid the R bug; R-parity for D in {0..9} is asserted at `tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityByPathNonBinary` on the `multi_path_reversible_by_path_non_binary` scenario (78 switchers, 3 paths, single-baseline custom DGP, F_g >= 4) — per-path point estimates match R bit-exactly (rtol ~1e-9 events; rtol+atol envelope for placebo near-zero values), SE inherits the documented cross-path cohort-sharing deviation (~5% rtol observed; SE_RTOL=0.15 envelope). Cross-surface invariants regression-tested at `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathNonBinary`. **Per-path survey-design SE** (analytical Binder TSL + replicate-weight bootstrap): under `by_path` / `paths_of_interest` + `survey_design`, the per-path per-horizon SE routes through `_survey_se_from_group_if` using the cell-period allocator. The per-path influence function `U_pp_l_path` is the per-period IF with non-path switcher-side contributions skipped — control contributions remain unchanged, matching the joiners/leavers IF convention from the **Per-path SE convention** paragraph above (the `switcher_subset_mask` zeroes the switcher row of the per-group IF, which trivially zeroes the corresponding row of the per-cell IF, preserving the row-sum identity `U_pp.sum(axis=1) == U`). The IF is cohort-recentered via `_cohort_recenter_per_period` and expanded to observations as `psi_i = U_pp[g_i, t_i] · (w_i / W_{g_i, t_i})`. Replicate-weight designs unconditionally route through the cell allocator (Class A contract, PR #323). Multiplier bootstrap (`n_bootstrap > 0`) under `survey_design + by_path/paths_of_interest` raises `NotImplementedError` at fit-time — the survey-aware perturbation pivot for path-restricted IFs is methodologically underived and deferred to a future wave; the global non-by_path TSL multiplier bootstrap is unaffected and continues to ship. **Path-enumeration ranking is unweighted** under `survey_design`: top-k selection uses group cardinality (`path_to_count[p]` = number of groups), not population-weight mass — survey weights do not affect which paths are selected as "top-k". A weighted-ranking variant (sum of survey weights per path) is deferred until concrete demand. **`df_survey` propagation:** under replicate weights, every per-path per-horizon fit contributes an `n_valid` count to the shared `_replicate_n_valid_list` accumulator and the final `_effective_df_survey = min(...) - 1` reflects all per-path replicate fits. A post-call `_refresh_path_inference` helper re-runs `safe_inference` on every populated entry so `multi_horizon_inference`, `placebo_horizon_inference`, `path_effects`, and `path_placebos` all use the same final df after per-path appends complete. **Lonely-PSU policy is sample-wide, not per-path** — the `lonely_psu` policy (`remove`/`certainty`/`adjust`) operates on the full design-level PSU/strata structure, not on path-restricted subsamples. **Telescope invariant:** on a single-path panel where every switcher follows the same trajectory and `eligible_groups` matches between by_path and non-by_path, per-path SE equals the global non-by_path survey SE bit-exactly — pinned at `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathSurveyDesignTelescope::test_telescope_analytical_TSL`. **Deviation from R:** none — R `did_multiplegt_dyn` does not support survey weighting, so this is a Python-only methodology extension (no R parity available; no R parity test class). Regression test anchor: `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathSurveyDesignAnalytical` covering analytical SE, replicate-weight SE, the `n_bootstrap` gate, the global anti-regression, per-path placebos, `trends_linear` composition, and unobserved-path warnings under survey. **Per-path user-specified path selection (`paths_of_interest`):** Python-only API extension — R's `did_multiplegt_dyn(..., by_path=k)` only accepts a positive int (top-k automatic ranking) or `-1` (all observed paths) and provides no list-based selection. Activated via `ChaisemartinDHaultfoeuille(paths_of_interest=[(0, 1, 1, 1), (0, 1, 0, 0)], drop_larger_lower=False)` as an alternative to `by_path=k`; the two are **mutually exclusive** (setting both raises `ValueError` at `__init__` and `set_params` time). Each path tuple must have length `L_max + 1`; the type / element / non-empty / length-uniformity checks fire at `__init__`, the length-vs-L_max check fires at fit-time. `bool` and `np.bool_` are explicitly rejected; `np.integer` is accepted and canonicalized to Python `int` for tuple-key consistency. Duplicates emit a `UserWarning` and are deduplicated; paths not observed in the panel emit a `UserWarning` and are omitted from `path_effects`. Paths appear in `results.path_effects` in the user-specified order, modulo deduplication and unobserved-path filtering. Composes with non-binary D and all downstream `by_path` surfaces (bootstrap, per-path placebos, per-path joint sup-t bands, `controls`, `trends_linear`, `trends_nonparam`) — mechanical filter on observed paths, no methodology change. Behavior + cross-feature regressions live at `tests/test_chaisemartin_dhaultfoeuille.py::TestPathsOfInterest`. +- **Note (Phase 3 `by_path` per-path event-study disaggregation):** Per-path disaggregation of the multi-horizon event study, mirroring R `did_multiplegt_dyn(..., by_path=k)`. Activated via `ChaisemartinDHaultfoeuille(by_path=k, drop_larger_lower=False)` where `k` is a positive integer (top-k most common observed paths by switcher-group frequency). **Window convention:** the path tuple for a switcher group `g` is `(D_{g, F_g-1}, D_{g, F_g}, ..., D_{g, F_g-1+L_max})` — length `L_max + 1`, matching R's window `[F_{g-1}, F_{g-1+l}]`. **Ranking:** paths are ranked by descending frequency; ties are broken lexicographically on the path tuple for deterministic ordering, so every selected path has a unique `frequency_rank`. If `by_path` exceeds the number of observed paths, all observed paths are returned with a `UserWarning`. **Per-path SE convention (joiners/leavers precedent):** the per-path influence function follows the joiners-only / leavers-only IF construction at `chaisemartin_dhaultfoeuille.py:5495-5504`: the switcher-side contribution `+S_g * (Y_{g,out} - Y_{g,ref})` is zeroed for groups whose observed trajectory is NOT the selected path; control contributions and the full cohort structure `(D_{g,1}, F_g, S_g)` are unchanged. After applying the singleton-baseline eligible mask and cohort-recentering with the original cohort IDs, the plug-in SE uses the path-specific divisor `N_l_path` (count of path switchers eligible at horizon `l`) — same pattern as `joiners_se` using `joiner_total`. This gives the **within-path mean** estimand `DID_{path,l}` as the within-path average of `DID_{g,l}`. **Degenerate-cohort behavior per path:** when a path's centered IF at some horizon is identically zero (every variance-eligible path switcher forms its own `(D_{g,1}, F_g, S_g)` cohort, or the path has a single contributing group), SE / t_stat / p_value / conf_int are NaN-consistent and a `UserWarning` is emitted scoped to `(path, horizon)`. This mirrors the overall-path degenerate-cohort surface and is common for rare paths with few contributing groups. **Empty-state contract:** `results.path_effects` distinguishes "not requested" (`None`) from "requested but empty" (`{}` — all switchers have windows outside the panel or unobserved cells). The empty-dict case emits a `UserWarning` at fit-time and renders as an explicit "no observed paths" notice in `summary()`; `to_dataframe(level="by_path")` returns an empty DataFrame with the canonical column set (mirrors the `linear_trends` pattern when `trends_linear=True` but no horizons survive). **Requirements:** `drop_larger_lower=False` (multi-switch groups are the object of interest; default `True` filters them out) and `L_max >= 1` (path window depends on the horizon). **Scope:** combinations with `design2` and `honest_did` remain gated behind explicit `NotImplementedError` (deferred to follow-up wave PRs); `heterogeneity` is supported per-path — see the **Per-path heterogeneity testing** paragraph below. `n_bootstrap > 0` is now supported — see the **Bootstrap SE** paragraph below. `survey_design` is supported under analytical Binder TSL and replicate-weight bootstrap — see the **Per-path survey-design SE** paragraph below; multiplier bootstrap (`n_bootstrap > 0`) under `survey_design + by_path/paths_of_interest` remains gated. `placebo=True` is now supported per-path — see the **Per-path placebos** paragraph below. **TWFE diagnostic** remains a sample-level summary (not computed per path) in this release. Results are exposed on `results.path_effects` as `Dict[Tuple[int, ...], Dict[str, Any]]` with nested `horizons` dicts per horizon `l`, and on `results.to_dataframe(level="by_path")` as a long-format table with columns `[path, frequency_rank, n_groups, horizon, effect, se, t_stat, p_value, conf_int_lower, conf_int_upper, n_obs, cband_lower, cband_upper, cumulated_effect, cumulated_se]` (the `cband_*` columns are added by the joint sup-t Note below, populated for positive-horizon rows of paths with a finite sup-t crit and NaN otherwise; the `cumulated_*` columns are added by the per-path linear-trends Note below, populated for positive-horizon rows when `trends_linear=True` is set and NaN otherwise). Gated tests live in `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathGates` / `::TestByPathBehavior` / `::TestByPathEdgeCases`. **R-parity** against `DIDmultiplegtDYN 2.3.3` is confirmed at `tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityByPath` via two scenarios: `mixed_single_switch_by_path` (2 paths, `by_path=2`) and `multi_path_reversible_by_path` (4 paths, `by_path=3`; path-assignment deterministic on `F_g` so each `(D_{g,1}, F_g, S_g)` cohort contains switchers from a single path). Per-path point estimates and per-path switcher counts match R exactly; per-path SE matches within the Phase 2 multi-horizon SE envelope (observed rtol ≤ 10.2% on the 2-path mixed scenario, ≤ 4.2% on the 4-path cohort-clean scenario). **Deviation from R (cross-path cohort-sharing SE):** our analytical SE is the marginal variance of the path-contribution estimator cohort-centered on the *full-panel* cohort structure (joiners/leavers precedent — non-path switchers contribute to cohort means via their zeroed switcher row). R's `did_multiplegt_dyn(..., by_path=k)` re-runs the estimator per path, so cohort means are computed over the path's own switchers only. When a cohort `(D_{g,1}, F_g, S_g)` spans multiple observed paths, Python and R SE diverge materially (our empirical probes with random post-window toggling saw rtol > 100%); when every cohort is single-path (scenario 13 by design, scenario 14 by construction), the two approaches coincide up to the documented Phase 2 envelope. Practitioners with cohort structures that mix paths should interpret the per-path SE as a within-full-panel marginal variance, not a per-path conditional variance. **Bootstrap SE:** when `n_bootstrap > 0` is set, the top-k paths are enumerated once on the observed data (R-faithful: matches `did_multiplegt_dyn(..., by_path=k, bootstrap=B)`'s path-stability convention — verified empirically against DIDmultiplegtDYN 2.3.3) and the multiplier bootstrap (`bootstrap_weights ∈ {"rademacher", "mammen", "webb"}`) runs per `(path, horizon)` target via the shared `_bootstrap_one_target` / `compute_effect_bootstrap_stats` helpers. Point estimates are unchanged from the analytical path. Bootstrap SE replaces the analytical SE in `path_effects[path]["horizons"][l]["se"]`, and `p_value` / `conf_int` are taken as the **bootstrap percentile** statistics, matching the Round-10 library convention for overall / joiners / leavers / multi-horizon bootstrap (see the `Note (bootstrap inference surface)` elsewhere in this file and the pinned regression `test_bootstrap_p_value_and_ci_propagated_to_top_level`). `t_stat` is SE-derived via `safe_inference` per the anti-pattern rule. Interpretation: inference is *conditional on the observed path set*. **SE inherits the analytical cross-path cohort-sharing deviation:** the bootstrap input is the exact same full-panel cohort-centered path IF that the analytical path computes (`_collect_path_bootstrap_inputs` reuses the same enumeration / cohort IDs / IF construction), so the bootstrap SE is a Monte Carlo analog of the analytical SE — it inherits the same cross-path cohort-sharing deviation from R's per-path re-run convention documented above. On single-path-cohort panels (scenarios 13 and 14 of the R-parity fixture, and any DGP where `(D_{g,1}, F_g, S_g)` cohorts never span multiple observed paths), bootstrap SE tracks analytical SE up to Monte Carlo noise and both coincide with R up to the Phase 2 envelope. On cross-path cohort panels, bootstrap SE inherits the >100% rtol divergence from R that analytical already has. **Deviation from R (CI method):** R's per-path CI is normal-theory around the bootstrap SE (half-width ≈ `1.96·se`); ours is the bootstrap percentile CI, intentionally diverging from R to keep the dCDH inference surface internally consistent across all bootstrap targets. Practitioners who want *unconditional* inference capturing path-selection uncertainty need a pairs-bootstrap (deferred — no R precedent). Positive regressions live in `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathBootstrap` (gated `@pytest.mark.slow`): point-estimate invariance, finite positive SE on non-degenerate panels, SE-within-30%-rtol of analytical on cohort-clean fixtures, degenerate-cohort NaN propagation, Rademacher/Mammen/Webb parity, seed reproducibility, and percentile-vs-normal-theory CI pinning. **Per-path placebos:** when `placebo=True` (and `L_max >= 1`) is combined with `by_path=k`, per-path backward-horizon placebos `DID^{pl}_{path, l}` for `l = 1..L_max` are computed using the same joiners/leavers IF precedent applied to `_compute_per_group_if_placebo_horizon` (with the new `switcher_subset_mask` parameter): switcher contributions are zeroed for groups not in the path; the control pool and the variance-eligible cohort structure `(D_{g,1}, F_g, S_g)` are unchanged. Plug-in SE uses the path-specific divisor `N^{pl}_{l, path}` (count of path switchers eligible at backward lag `l`). Surfaced on `results.path_placebo_event_study[path][-l]` with the same `{effect, se, t_stat, p_value, conf_int, n_obs}` shape as `placebo_event_study` (negative-int inner keys parallel the existing per-path event-study positive-int keys, so a unified forward+backward view is well-formed). **Inherits the cross-path cohort-sharing SE deviation from R** documented above for `path_effects` (same convention applied backward); tracks R within numerical tolerance on single-path-cohort panels and diverges on cohort-mixed panels. Multiplier bootstrap (when `n_bootstrap > 0`) runs per `(path, lag)` target via the same `_bootstrap_one_target` dispatch used for the per-path event-study, with the canonical NaN-on-invalid contract. The bootstrap SE is a Monte Carlo analog of the analytical placebo SE — same per-path centered IF input — and inherits the same deviation. Surfaced through `summary()` (negative-keyed rows rendered alongside positive-keyed event-study rows under each path block) and `to_dataframe(level="by_path")` (`horizon` column takes negative ints for placebo rows). **Empty-state contract:** `results.path_placebo_event_study` mirrors `path_effects` — `None` when `by_path + placebo` was not requested, `{}` when requested but no observed path has a complete window within the panel (same regime that returns `{}` for `path_effects`, with the same fit-time `UserWarning`). R-parity is confirmed at `tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityByPathPlacebo` on the `multi_path_reversible_by_path_placebo` scenario; positive analytical + bootstrap invariants live in `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathPlacebo` (with the gated `::TestByPathPlacebo::TestBootstrap` subclass). **Per-path covariate residualization (DID^X):** when `controls=[...]` is set with `by_path=k`, the per-baseline OLS residualization (Web Appendix Section 1.2) runs once on the first-differenced outcome BEFORE path enumeration. All four downstream surfaces — analytical per-path SE, bootstrap SE, per-path placebos, and per-path joint sup-t bands — consume the residualized `Y_mat` automatically (Frisch-Waugh-Lovell). Per-period effects remain unadjusted, consistent with the existing `controls` + per-period DID contract (per-period DID does not support residualization). Failed-stratum baselines (rank-deficient X) zero out `N_mat` for affected groups, which the path enumeration treats as ineligible per its existing convention. **Deviation from R on multi-baseline switcher panels (point estimates):** R `did_multiplegt_dyn(..., by_path, controls)` re-runs the per-baseline residualization on each path's restricted subsample (`R/R/did_multiplegt_dyn.R` lines 401-405: rows of the path's switchers OR rows where `yet_to_switch=1 AND baseline matches the path's baseline`). The first-stage residualization sample R uses for path B equals: pre-switch rows of all switchers with matching baseline + all rows of never-switchers with matching baseline — bit-identical to our global first-stage sample under single-baseline switcher panels (every switcher shares the same `D_{g,1}`, regardless of how `F_g` or path identity varies across switchers). Per-path point estimates therefore coincide with R on those panels up to the existing **DID^X first-stage cell-weighting deviation** documented above in `Note (Phase 3 DID^X covariate adjustment)` (Python's first-stage OLS uses equal cell weights — one observation per `(g, t)` cell, consistent with the library's cell-aggregated input convention; R weights by `N_gt`). On panels with one observation per `(g, t)` cell (the common case after the cell-aggregation step in `fit()`), Python matches R bit-exactly: the `multi_path_reversible_by_path_controls` parity fixture has 4 paths with switcher `F_g` values spanning [0..6] under `D_{g,1}=0` and Python matches R to rtol ~1e-11. On multi-baseline switcher panels (some switchers have `D_{g,1}=0`, others have `D_{g,1}=1`) R's per-path subset drops switchers whose baseline differs from the path's baseline, so the per-baseline regression coefficients diverge per path under R and point estimates can diverge between Python and R — a `UserWarning` is emitted at fit-time when this configuration is detected so practitioners do not silently consume estimates that disagree with R. The warning filters to switcher groups only; never-switchers (never-treated + always-treated controls) at multiple baseline values do NOT trigger the warning because they don't affect R's per-path subset construction. **Inherits the cross-path cohort-sharing SE deviation from R** documented above for `path_effects` — bootstrap SE, placebo SE, and sup-t crit are Monte Carlo / joint-distribution analogs of the same residualized analytical IF and carry the same deviation. R-parity is confirmed against `did_multiplegt_dyn(..., by_path=3, controls="X1")` at `tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityByPathControls` on the `multi_path_reversible_by_path_controls` scenario (single-baseline DGP, exact point-estimate match measured rtol ~1e-11); cross-surface inheritance and the multi-baseline warning are regression-tested at `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathControls` (analytical + bootstrap + placebo + sup-t + `to_dataframe(level="by_path")` cband columns + multi-baseline `UserWarning`). **Per-path linear-trends DID^{fd}:** when `trends_linear=True` is set with `by_path=k`, the first-differencing transform at `chaisemartin_dhaultfoeuille.py:1599-1630` runs once globally BEFORE path enumeration (replaces `Y_mat` with `Z_mat = Y_t - Y_{t-1}` and shrinks the time axis by one), so per-path raw second-differences `DID^{fd}_{path, l}` surface on `path_effects[path]["horizons"][l]` automatically. Per-path cumulated level effects `delta_{path, l} = sum_{l'=1..l} DID^{fd}_{path, l'}` (the quantity R returns under `did_multiplegt_dyn(..., by_path, trends_lin)` per the existing parity test pivot at `tests/test_chaisemartin_dhaultfoeuille_parity.py:403-409`) surface on the new `results.path_cumulated_event_study[path][l]` field — a per-group running sum of `DID^{fd}_{g, l'}` averaged over the path's switchers eligible at horizon `l`, mirroring the global `linear_trends_effects` cumulation logic at `chaisemartin_dhaultfoeuille.py:3340-3398`. SE on the cumulated layer is the conservative upper bound (sum of per-horizon component SEs from `path_effects[path]["horizons"][l]["se"]`, NaN-consistent: any non-finite component yields a NaN cumulated SE). **Post-bootstrap recomputation:** the cumulated layer is built AFTER the bootstrap propagation block at `chaisemartin_dhaultfoeuille.py:3034-3081` so it reads the FINAL post-bootstrap per-horizon SEs (mirrors the global `linear_trends_effects` placement). When `n_bootstrap > 0`, cumulated SE / t / p / CI are derived from bootstrap per-horizon SEs; when bootstrap produces non-finite SE (e.g., `n_bootstrap=1` degenerate distribution), the cumulated layer's full inference tuple is NaN per the library-wide NaN-on-invalid bootstrap contract. `to_dataframe(level="by_path")` exposes `cumulated_effect` and `cumulated_se` columns (always present, NaN-when-None — mirrors the `cband_*` always-present convention from PR #374). `summary()` renders a `Cumulated Level Effects (DID^{fd}, trends_linear)` sub-section under each per-path block. **Path enumeration uses the post-first-differenced `N_mat_fd`**: switchers with `F_g==2` fail the window-eligibility check and are dropped from path enumeration entirely (the existing global `F_g >= 3` warning at line 1620 surfaces the issue), so a path whose switchers all have `F_g < 3` is silently absent from `path_effects` rather than present-with-NaN. **F_g=3 boundary-case divergence (`by_path + trends_linear`):** `F_g=3` switchers have exactly 2 pre-switch periods, which after first-differencing and the `time==1` filter leaves only 1 valid pre-window Z value. R's per-path full-pipeline call handles this single-pre-period regime differently from Python's global-then-disaggregate architecture, producing 30%+ relative divergence on point estimates for paths whose switchers include `F_g=3` (empirically observed on the parity fixture's earlier `F_g=3` variant). A separate `UserWarning` fires at fit-time when the panel includes any `F_g=3` switcher AND `by_path + trends_linear` is set, mirroring the `F_g < 3` exclusion warning. The shipped parity fixture (`single_baseline_multi_path_by_path_trends_lin`) restricts to `F_g >= 4` exclusively to avoid this regime; per-path R parity is asserted only there. **Placebo under `trends_linear` returns RAW per-horizon values** (no per-path placebo cumulation surface) — verified empirically against the existing `joiners_only_trends_lin` parity fixture: R's per-path Placebo_l matches Python's `path_placebo_event_study[path][-l]` (raw) bit-exactly under non-`by_path` trends_lin. **Deviation from R on multi-baseline switcher panels (point estimates):** R `did_multiplegt_dyn(..., by_path, trends_lin)` re-runs the full pipeline (including first-differencing) on each path's restricted subsample, so it operates on different switcher samples per path when switchers have different baseline values `D_{g,1}`. Python first-differences once globally before path enumeration. On single-baseline switcher panels the two architectures coincide; on multi-baseline switcher panels per-path point estimates can diverge — a `UserWarning` is emitted at fit-time when this configuration is detected so practitioners do not silently consume estimates that disagree with R (mirroring the analogous `by_path + controls` warning). Per-path R parity is confirmed against `did_multiplegt_dyn(..., by_path=3, trends_lin=TRUE, placebo=1)` at `tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityByPathTrendsLinear` on the `single_baseline_multi_path_by_path_trends_lin` scenario (single-baseline + cohort-single-path + `F_g >= 4` DGP designed to eliminate the multi-baseline divergence, the cross-path cohort-sharing deviation, and the F_g=3 boundary case under R's per-path full-pipeline call). Per-path cumulated point estimates match R bit-exactly (rtol ~1e-9) on event horizons under those conditions; cumulated SE_RTOL is widened to `0.20` (vs `0.12` used for non-cumulated by_path parity) because the conservative upper-bound SE compounds the cross-path cohort-sharing deviation under summation. **Placebo parity is intentionally skipped for `trends_linear`**: R's per-path placebo computation re-runs on the path-restricted subsample with different control eligibility than Python's global-then-disaggregate architecture surfaces, producing a sign-and-magnitude divergence on paths whose switchers have minimal pre-window depth (e.g., `F_g=4` switchers). Placebo under `by_path + trends_linear` is exercised via internal regression in `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathTrendsLinear` (finite values, bootstrap inheritance) but not pinned to R bit-by-bit. Cross-surface invariants (analytical + bootstrap + placebo + sup-t + `path_cumulated_event_study` + `to_dataframe` columns + `summary()` rendering) are regression-tested at `TestByPathTrendsLinear`. **Per-path state-set trends:** when `trends_nonparam="state_col"` is set with `by_path=k`, the set membership column is validated and stored once globally as `set_ids_arr` (time-invariance, NaN rejection, partition-coarseness checks unchanged from the non-by_path path). The `set_ids` parameter is threaded through the four per-path IF helpers (`_compute_path_effects`, `_compute_path_placebos`, `_collect_path_bootstrap_inputs`, `_collect_path_placebo_bootstrap_inputs`) so per-path analytical SE, bootstrap, placebos, and sup-t bands all consume the set-restricted control pool automatically. R does NOT first-difference and does NOT cumulate under `trends_nonparam` (unlike `trends_lin`); per-horizon `Effect_l` is a normal DID with set-restricted controls. Per-path R parity is confirmed against `did_multiplegt_dyn(..., by_path=3, trends_nonparam="state", placebo=1)` at `tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityByPathTrendsNonparam` on the `multi_path_reversible_by_path_trends_nonparam` scenario; per-path point estimates AND placebos match R bit-exactly (rtol ~1e-9), per-path SE matches within the Phase 2 envelope (~13% rtol observed). Cross-surface invariants are regression-tested at `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathTrendsNonparam`. **Per-path non-binary treatment:** integer-coded discrete treatment (D in Z, e.g. ordinal {0, 1, 2}) is supported under `by_path=k` and `paths_of_interest`. Path tuples become integer-state tuples (`(0, 2, 2, 2)`) keyed bit-for-bit against R's comma-separated path strings (`"0,2,2,2"`) for D in {0..9}. Continuous D (e.g. `1.5`) raises `ValueError` at fit-time per the no-silent-failures contract — the existing `int(round(float(v)))` cast in `_enumerate_treatment_paths` is now defensive (no-op for integer-coded D). **Deviation from R for D >= 10:** R's `did_multiplegt_by_path` derives the per-path baseline via `path_index$baseline_XX <- substr(path_index$path, 1, 1)` (extracted 2026-05-03 via `Rscript -e 'cat(paste(deparse(DIDmultiplegtDYN:::did_multiplegt_by_path), collapse="\n"))'`), capturing only the first character of the comma-separated path string. For D >= 10 this captures `"1"` instead of `"12"` for `path = "12,12,..."`, mis-allocating R's per-path control-pool subset. Python's tuple-key matching is correct in this regime; the per-path point estimates we compute are correct, R's per-path subset for the same path is buggy. The shipped parity scenario stays in `D in {0, 1, 2}` to avoid the R bug; R-parity for D in {0..9} is asserted at `tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityByPathNonBinary` on the `multi_path_reversible_by_path_non_binary` scenario (78 switchers, 3 paths, single-baseline custom DGP, F_g >= 4) — per-path point estimates match R bit-exactly (rtol ~1e-9 events; rtol+atol envelope for placebo near-zero values), SE inherits the documented cross-path cohort-sharing deviation (~5% rtol observed; SE_RTOL=0.15 envelope). Cross-surface invariants regression-tested at `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathNonBinary`. **Per-path survey-design SE** (analytical Binder TSL + replicate-weight bootstrap): under `by_path` / `paths_of_interest` + `survey_design`, the per-path per-horizon SE routes through `_survey_se_from_group_if` using the cell-period allocator. The per-path influence function `U_pp_l_path` is the per-period IF with non-path switcher-side contributions skipped — control contributions remain unchanged, matching the joiners/leavers IF convention from the **Per-path SE convention** paragraph above (the `switcher_subset_mask` zeroes the switcher row of the per-group IF, which trivially zeroes the corresponding row of the per-cell IF, preserving the row-sum identity `U_pp.sum(axis=1) == U`). The IF is cohort-recentered via `_cohort_recenter_per_period` and expanded to observations as `psi_i = U_pp[g_i, t_i] · (w_i / W_{g_i, t_i})`. Replicate-weight designs unconditionally route through the cell allocator (Class A contract, PR #323). Multiplier bootstrap (`n_bootstrap > 0`) under `survey_design + by_path/paths_of_interest` raises `NotImplementedError` at fit-time — the survey-aware perturbation pivot for path-restricted IFs is methodologically underived and deferred to a future wave; the global non-by_path TSL multiplier bootstrap is unaffected and continues to ship. **Path-enumeration ranking is unweighted** under `survey_design`: top-k selection uses group cardinality (`path_to_count[p]` = number of groups), not population-weight mass — survey weights do not affect which paths are selected as "top-k". A weighted-ranking variant (sum of survey weights per path) is deferred until concrete demand. **`df_survey` propagation:** under replicate weights, every per-path per-horizon fit contributes an `n_valid` count to the shared `_replicate_n_valid_list` accumulator and the final `_effective_df_survey = min(...) - 1` reflects all per-path replicate fits. A post-call `_refresh_path_inference` helper re-runs `safe_inference` on every populated entry so `multi_horizon_inference`, `placebo_horizon_inference`, `path_effects`, and `path_placebos` all use the same final df after per-path appends complete. **Lonely-PSU policy is sample-wide, not per-path** — the `lonely_psu` policy (`remove`/`certainty`/`adjust`) operates on the full design-level PSU/strata structure, not on path-restricted subsamples. **Telescope invariant:** on a single-path panel where every switcher follows the same trajectory and `eligible_groups` matches between by_path and non-by_path, per-path SE equals the global non-by_path survey SE bit-exactly — pinned at `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathSurveyDesignTelescope::test_telescope_analytical_TSL`. **Deviation from R:** none — R `did_multiplegt_dyn` does not support survey weighting, so this is a Python-only methodology extension (no R parity available; no R parity test class). Regression test anchor: `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathSurveyDesignAnalytical` covering analytical SE, replicate-weight SE, the `n_bootstrap` gate, the global anti-regression, per-path placebos, `trends_linear` composition, and unobserved-path warnings under survey. **Per-path heterogeneity testing** (analytical OLS / WLS + survey-aware Binder TSL + replicate-weight): under `by_path` / `paths_of_interest` + `heterogeneity=""`, the per-path per-horizon coefficient `beta_X^path_l` is computed by re-running `_compute_heterogeneity_test` on the path-restricted switcher subsample. The path filter (`path_groups: Optional[Set[int]]`) restricts eligibility to switchers ON path `p` inside the inner regression; the variance machinery (standard WLS vcov for non-survey, cell-period IF allocator for Binder TSL, group-level allocator for Rao-Wu replicate) is unchanged from the global heterogeneity path. **Cohort dummies absorb baseline by construction** — the cohort key `(D_{g,1}, F_g, S_g)` includes baseline, so multi-baseline switcher panels do not produce R-divergence (unlike `controls` / `trends_linear`); no parallel `UserWarning` is emitted. **R parity:** matches `did_multiplegt_dyn(..., by_path, predict_het)` per-by_level on the `multi_path_reversible_by_path_predict_het` scenario (rtol ~1e-6 on point estimates AND SE), inheriting the SAME tolerance as the new global `multi_path_reversible_predict_het` scenario (`TestDCDHDynRParityHeterogeneity`) since the per-path R call is `did_multiplegt_main(..., predict_het=...)` per path-restricted subsample with no additional numerical loss. R's `dont_drop_larger_lower=TRUE` is set in both fixture scenarios to match the Python `drop_larger_lower=False` requirement. **Survey composition:** inherits from the **Per-path survey-design SE** paragraph above — analytical Binder TSL routes through `_survey_se_from_group_if`'s cell-period allocator on the post-period of the transition; replicate-weights route through the group-level allocator. Multiplier bootstrap (`n_bootstrap > 0`) under `by_path + heterogeneity + survey_design` inherits the existing per-path multiplier-bootstrap-survey gate. **`df_survey` propagation:** every per-(path, horizon) replicate-weight fit appends `n_valid` to the shared `_replicate_n_valid_list` accumulator; per-path heterogeneity inference is refreshed with the FINAL `_effective_df_survey(...)` in the R2 P1b refresh block (separate dedicated loop because the schema shape is `{path: {l: {...}}}` rather than `{path: {"horizons": {l: {...}}}}`). **Result schema:** `results.path_heterogeneity_effects: Dict[Tuple[int, ...], Dict[int, Dict[str, Any]]]` keyed `{path: {l: {beta, se, t_stat, p_value, conf_int, n_obs}}}`. Empty-state contract mirrors `path_effects`: `None` when not requested, `{}` when requested but no path has eligible switchers. **DataFrame integration:** `to_dataframe(level="by_path")` adds always-present `het_*` columns (`het_beta`, `het_se`, `het_t_stat`, `het_p_value`, `het_conf_int_lower`, `het_conf_int_upper`), populated for positive-horizon rows when `heterogeneity` is set and NaN otherwise (mirrors the `cband_*` and `cumulated_*` always-present convention). Placebo rows (negative `horizon`) have NaN in `het_*` columns: per-path placebo heterogeneity is not exposed in this release (R does not ship per-path predict_het on placebos either, so parity is preserved by deferral). Regression test anchors: `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathHeterogeneity` (gate dispatch, behavior, telescope-to-global on single-path panel, zero-signal anti-regression, multi-baseline UserWarning anti-regression, DataFrame integration, edge cases) + `tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityHeterogeneity` (global anchor, FIRST `predict_het` parity baseline) + `::TestDCDHDynRParityByPathHeterogeneity` (per-path). + +**Per-path user-specified path selection (`paths_of_interest`):** Python-only API extension — R's `did_multiplegt_dyn(..., by_path=k)` only accepts a positive int (top-k automatic ranking) or `-1` (all observed paths) and provides no list-based selection. Activated via `ChaisemartinDHaultfoeuille(paths_of_interest=[(0, 1, 1, 1), (0, 1, 0, 0)], drop_larger_lower=False)` as an alternative to `by_path=k`; the two are **mutually exclusive** (setting both raises `ValueError` at `__init__` and `set_params` time). Each path tuple must have length `L_max + 1`; the type / element / non-empty / length-uniformity checks fire at `__init__`, the length-vs-L_max check fires at fit-time. `bool` and `np.bool_` are explicitly rejected; `np.integer` is accepted and canonicalized to Python `int` for tuple-key consistency. Duplicates emit a `UserWarning` and are deduplicated; paths not observed in the panel emit a `UserWarning` and are omitted from `path_effects`. Paths appear in `results.path_effects` in the user-specified order, modulo deduplication and unobserved-path filtering. Composes with non-binary D and all downstream `by_path` surfaces (bootstrap, per-path placebos, per-path joint sup-t bands, `controls`, `trends_linear`, `trends_nonparam`) — mechanical filter on observed paths, no methodology change. Behavior + cross-feature regressions live at `tests/test_chaisemartin_dhaultfoeuille.py::TestPathsOfInterest`. - **Note (Phase 3 `by_path` per-path joint sup-t bands):** When `n_bootstrap > 0` is set with `by_path=k`, per-path joint sup-t simultaneous confidence bands are computed across horizons `1..L_max` within each path. **Methodology:** a single `(n_bootstrap, n_eligible)` multiplier weight matrix (using the estimator's configured `bootstrap_weights` — Rademacher / Mammen / Webb) is drawn per path and broadcast across all horizons of that path, producing correlated bootstrap distributions across horizons within the path. The path-specific critical value `c_p = quantile(max_l |t_l|, 1 - α)` is then used to construct symmetric joint bands `effect_l ± c_p · se_l` per horizon, surfaced in `path_effects[path]["horizons"][l]["cband_conf_int"]` and at top-level `results.path_sup_t_bands[path] = {"crit_value", "alpha", "n_bootstrap", "method", "n_valid_horizons"}`. **Gates:** a path must have `>= 2` valid horizons (finite bootstrap SE > 0) AND a strict majority (more than 50%) of finite sup-t draws to receive a band; otherwise the path is absent from `path_sup_t_bands`. Both gates mirror the OVERALL `event_study_sup_t_bands` semantics at `chaisemartin_dhaultfoeuille_bootstrap.py:605,612`: `len(valid_horizons) >= 2` AND `finite_mask.sum() > 0.5 * n_bootstrap`. Exactly half-finite draws are NOT enough — the gate is strictly greater than half. **Empty-state contract:** `path_sup_t_bands is None` when not requested (no bootstrap, or both `by_path` and `paths_of_interest` are `None`); `{}` when requested but no path passes both gates. **`to_dataframe(level="by_path")` integration:** the table now includes `cband_lower` / `cband_upper` columns for parity with OVERALL `level="event_study"`; populated for positive-horizon rows of paths with a finite sup-t crit, NaN for placebo rows / unbanded paths / the requested-but-empty fallback DataFrame. **Methodology asymmetry vs OVERALL:** OVERALL sup-t reuses the same multi-horizon shared-draw distribution for both the SE in the t-stat denominator and the bootstrap distribution in the numerator. The per-path sup-t draws a fresh shared weight matrix per path AFTER the per-path SE bootstrap block has already populated `results.path_ses` via independent per-(path, horizon) draws — numerator: fresh shared draws, denominator: bootstrap SEs from the earlier independent draws. Asymptotically equivalent to OVERALL's self-consistent reuse, but NOT bit-identical. The fresh draw is intentional: it preserves RNG-state isolation and keeps every existing per-path SE seed-reproducibility test bit-stable post-implementation. **Inherited deviation from R:** the bootstrap SE used as the t-stat denominator carries the cross-path cohort-sharing SE deviation from R documented for `path_effects` above; the per-path sup-t crit therefore inherits the same deviation. **Interpretation:** the band covers joint inference *within a single path across horizons*; it does NOT provide simultaneous coverage *across paths* (a different inference target requiring a `path × horizon` re-derivation, deferred to a future wave). **Deviation from R:** `did_multiplegt_dyn` provides no joint / sup-t / simultaneous bands at any surface — this is a Python-only methodology extension, consistent with the existing OVERALL `event_study_sup_t_bands` (also Python-only). Regression test anchor: `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathSupTBands`. diff --git a/tests/test_chaisemartin_dhaultfoeuille.py b/tests/test_chaisemartin_dhaultfoeuille.py index ccaba2e9..f7d496f6 100644 --- a/tests/test_chaisemartin_dhaultfoeuille.py +++ b/tests/test_chaisemartin_dhaultfoeuille.py @@ -3828,19 +3828,17 @@ def test_requires_lmax(self): @pytest.mark.parametrize( "fit_kwargs, msg", [ - # NB: prior `controls` (Wave 3 #5), `trends_linear`, and - # `trends_nonparam` (Wave 3 #6+#7) entries were removed - # when their gates were lifted. After gate removal, those - # combinations either fit successfully (controls passes - # column-validation; trends_linear/trends_nonparam route - # to their respective code paths) or raise a non- - # NotImplementedError specific to the parameter (e.g., - # trends_nonparam=group raises a partition-coarseness - # ValueError because the set partition equals the group - # partition). Coverage for those combinations now lives - # in `TestByPathControls`, `TestByPathTrendsLinear`, and - # `TestByPathTrendsNonparam`. - ({"heterogeneity": "group"}, "heterogeneity"), + # NB: prior `controls` (Wave 3 #5), `trends_linear` / + # `trends_nonparam` (Wave 3 #6+#7), and `heterogeneity` + # (Wave 5 #11) entries were removed when their gates were + # lifted. After gate removal, those combinations either fit + # successfully (heterogeneity routes to + # path_heterogeneity_effects; controls passes column- + # validation; trends_linear/trends_nonparam route to their + # respective code paths) or raise a non-NotImplementedError + # specific to the parameter. Coverage for those combinations + # now lives in `TestByPathControls`, `TestByPathTrendsLinear`, + # `TestByPathTrendsNonparam`, and `TestByPathHeterogeneity`. ({"design2": True}, "design2"), ({"honest_did": True}, "honest_did"), ], @@ -9924,3 +9922,681 @@ def test_telescope_analytical_TSL(self): res_g.event_study_effects[l_h]["se"], atol=1e-12, ) + + +# =========================================================================== +# Wave 5 #11: by_path / paths_of_interest + heterogeneity testing +# =========================================================================== + + +def _by_path_het_data(seed=44, n_switchers=90, n_controls=30, n_periods=10): + """Multi-path panel with binary `het_x` covariate. + + Layered on `TestHeterogeneityTesting._make_panel_with_het` shape + (binary het_x, half each) plus multi-path structure (3 paths, F_g + independent of path so each path has multiple cohorts). Includes + never-treated controls so the heterogeneity regression has cohort + variation at every horizon under the reversal-path eligibility + filter (cf. PR #408 R parity preflight: without controls, R drops + reversal paths past horizon 1 leaving a single cohort and triggering + empty-cohort-dummy errors). Outcome: 0.5*t + (5 + 3*het_x) * D + N(0, 0.5). + """ + rng = np.random.RandomState(seed) + rows = [] + paths = [(0, 1, 1, 1), (0, 1, 0, 0), (0, 1, 1, 0)] + for g in range(n_switchers): + F_g = 3 + ((g // 3) % 3) # F_g in {3,4,5} + path = paths[g % 3] + het_x = 1 if g < n_switchers // 2 else 0 + effect = 5.0 + 3.0 * het_x + for t in range(n_periods): + if F_g - 1 <= t < F_g - 1 + len(path): + d = path[t - (F_g - 1)] + elif t >= F_g - 1 + len(path): + d = path[-1] + else: + d = 0 + y = 0.5 * t + effect * d + rng.normal(0, 0.5) + rows.append({ + "group": g, "period": t, "treatment": d, + "outcome": y, "het_x": het_x, + }) + # Never-treated controls (D=0 throughout), het_x balanced + for k in range(n_controls): + het_x = 1 if k < n_controls // 2 else 0 + g = n_switchers + k + for t in range(n_periods): + y = 0.5 * t + rng.normal(0, 0.5) + rows.append({ + "group": g, "period": t, "treatment": 0, + "outcome": y, "het_x": het_x, + }) + return pd.DataFrame(rows) + + +class TestByPathHeterogeneity: + """Per-path heterogeneity (Wave 5 #11) — composes ``by_path`` / + ``paths_of_interest`` with ``heterogeneity=""``. + + R parity coverage in + ``tests/test_chaisemartin_dhaultfoeuille_parity.py:: + TestDCDHDynRParityByPathHeterogeneity``. + """ + + # Gate dispatch: lifts no longer raise + + def test_no_longer_raises_on_heterogeneity(self): + """``by_path=k`` + ``heterogeneity`` no longer raises.""" + df = _by_path_het_data() + est = ChaisemartinDHaultfoeuille(drop_larger_lower=False, by_path=2) + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + res = est.fit( + df, outcome="outcome", group="group", time="period", + treatment="treatment", L_max=3, heterogeneity="het_x", + ) + assert res.path_heterogeneity_effects is not None + + def test_paths_of_interest_with_heterogeneity_no_longer_raises(self): + """``paths_of_interest`` + ``heterogeneity`` no longer raises.""" + df = _by_path_het_data() + est = ChaisemartinDHaultfoeuille( + drop_larger_lower=False, + paths_of_interest=[(0, 1, 1, 1), (0, 1, 0, 0)], + ) + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + res = est.fit( + df, outcome="outcome", group="group", time="period", + treatment="treatment", L_max=3, heterogeneity="het_x", + ) + assert res.path_heterogeneity_effects is not None + + def test_heterogeneity_still_rejects_controls_under_by_path(self): + """``heterogeneity + controls`` mutex still fires under by_path.""" + df = _by_path_het_data() + df["X1"] = np.random.RandomState(42).normal(0, 1, len(df)) + with pytest.raises(ValueError, match="cannot be combined with controls"): + ChaisemartinDHaultfoeuille( + drop_larger_lower=False, by_path=2 + ).fit( + df, outcome="outcome", group="group", time="period", + treatment="treatment", L_max=3, + heterogeneity="het_x", controls=["X1"], + ) + + def test_heterogeneity_still_rejects_trends_linear_under_by_path(self): + """``heterogeneity + trends_linear`` mutex still fires under by_path.""" + df = _by_path_het_data() + with pytest.raises( + ValueError, match="cannot be combined with trends_linear" + ): + ChaisemartinDHaultfoeuille( + drop_larger_lower=False, by_path=2 + ).fit( + df, outcome="outcome", group="group", time="period", + treatment="treatment", L_max=3, + heterogeneity="het_x", trends_linear=True, + ) + + def test_heterogeneity_still_rejects_trends_nonparam_under_by_path(self): + """``heterogeneity + trends_nonparam`` mutex still fires under by_path.""" + df = _by_path_het_data() + df["state"] = df["group"] % 3 + with pytest.raises( + ValueError, match="cannot be combined with trends_nonparam" + ): + ChaisemartinDHaultfoeuille( + drop_larger_lower=False, by_path=2 + ).fit( + df, outcome="outcome", group="group", time="period", + treatment="treatment", L_max=3, + heterogeneity="het_x", trends_nonparam="state", + ) + + # Behavior + + def test_per_path_heterogeneity_finite_under_known_signal(self): + """Detects positive heterogeneity on the path that contains the + effect-varying switchers.""" + df = _by_path_het_data() + est = ChaisemartinDHaultfoeuille(drop_larger_lower=False, by_path=3) + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + res = est.fit( + df, outcome="outcome", group="group", time="period", + treatment="treatment", L_max=3, heterogeneity="het_x", + ) + assert res.path_heterogeneity_effects + # At horizon 1 every path has switchers; heterogeneity beta should + # be positive (DGP: effect = 5 + 3*het_x). + for path, horizons in res.path_heterogeneity_effects.items(): + assert 1 in horizons + assert np.isfinite(horizons[1]["beta"]) + assert np.isfinite(horizons[1]["se"]) + assert horizons[1]["beta"] > 0, ( + f"path={path} l=1: expected positive het beta " + f"(DGP: 5 + 3*het_x), got {horizons[1]['beta']}" + ) + + def test_per_path_heterogeneity_telescope_to_global_on_single_path(self): + """On a single-path panel, per-path == global heterogeneity. + Plain OLS path: bit-exact via path_groups identity.""" + # Single-path DGP: all switchers follow (0,1,1,1) + rng = np.random.RandomState(44) + rows = [] + n_switchers = 60 + n_controls = 20 + for g in range(n_switchers): + F_g = 3 + ((g // 3) % 3) + path = (0, 1, 1, 1) + het_x = 1 if g < n_switchers // 2 else 0 + effect = 5.0 + 3.0 * het_x + for t in range(10): + if F_g - 1 <= t < F_g - 1 + len(path): + d = path[t - (F_g - 1)] + elif t >= F_g - 1 + len(path): + d = path[-1] + else: + d = 0 + rows.append({ + "group": g, "period": t, "treatment": d, + "outcome": 0.5 * t + effect * d + rng.normal(0, 0.5), + "het_x": het_x, + }) + for k in range(n_controls): + het_x = 1 if k < n_controls // 2 else 0 + for t in range(10): + rows.append({ + "group": n_switchers + k, "period": t, "treatment": 0, + "outcome": 0.5 * t + rng.normal(0, 0.5), + "het_x": het_x, + }) + df = pd.DataFrame(rows) + # Run with by_path=1 (path is observed) + est_p = ChaisemartinDHaultfoeuille(drop_larger_lower=False, by_path=1) + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + res_p = est_p.fit( + df, outcome="outcome", group="group", time="period", + treatment="treatment", L_max=3, heterogeneity="het_x", + ) + # Run global (no by_path) + est_g = ChaisemartinDHaultfoeuille(drop_larger_lower=False) + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + res_g = est_g.fit( + df, outcome="outcome", group="group", time="period", + treatment="treatment", L_max=3, heterogeneity="het_x", + ) + assert res_p.path_heterogeneity_effects + path_key = (0, 1, 1, 1) + assert path_key in res_p.path_heterogeneity_effects + for l_h in range(1, 4): + py_path = res_p.path_heterogeneity_effects[path_key][l_h] + py_global = res_g.heterogeneity_effects[l_h] + if not np.isfinite(py_path["beta"]): + assert not np.isfinite(py_global["beta"]) + continue + np.testing.assert_allclose( + py_path["beta"], py_global["beta"], atol=1e-14, rtol=1e-14, + err_msg=f"l={l_h}: per-path beta != global beta (telescope failed)", + ) + np.testing.assert_allclose( + py_path["se"], py_global["se"], atol=1e-14, rtol=1e-14, + err_msg=f"l={l_h}: per-path se != global se (telescope failed)", + ) + + def test_per_path_heterogeneity_zero_signal_yields_small_beta(self): + """Uncorrelated covariate yields beta near zero per (path, l).""" + rng = np.random.RandomState(123) + rows = [] + n_switchers = 90 + n_controls = 30 + paths = [(0, 1, 1, 1), (0, 1, 0, 0), (0, 1, 1, 0)] + for g in range(n_switchers): + F_g = 3 + ((g // 3) % 3) + path = paths[g % 3] + # het_x is random and uncorrelated with anything + het_x = rng.normal(0, 1) + for t in range(10): + if F_g - 1 <= t < F_g - 1 + len(path): + d = path[t - (F_g - 1)] + elif t >= F_g - 1 + len(path): + d = path[-1] + else: + d = 0 + # Effect is constant 5.0 — no heterogeneity by het_x + rows.append({ + "group": g, "period": t, "treatment": d, + "outcome": 0.5 * t + 5.0 * d + rng.normal(0, 0.5), + "het_x": het_x, + }) + for k in range(n_controls): + # Draw het_x ONCE per group (must be time-invariant) + het_x = rng.normal(0, 1) + for t in range(10): + rows.append({ + "group": n_switchers + k, "period": t, "treatment": 0, + "outcome": 0.5 * t + rng.normal(0, 0.5), + "het_x": het_x, + }) + df = pd.DataFrame(rows) + est = ChaisemartinDHaultfoeuille(drop_larger_lower=False, by_path=3) + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + res = est.fit( + df, outcome="outcome", group="group", time="period", + treatment="treatment", L_max=3, heterogeneity="het_x", + ) + assert res.path_heterogeneity_effects + for path, horizons in res.path_heterogeneity_effects.items(): + for l_h, vals in horizons.items(): + if np.isfinite(vals["beta"]): + # |beta| should be small (well within 3 standard + # errors of zero) under the null + assert abs(vals["beta"]) < 5.0, ( + f"path={path} l={l_h}: |beta|={abs(vals['beta']):.3f} " + f"too large for zero-signal DGP" + ) + + def test_path_with_too_few_eligible_yields_nan(self): + """A path with <3 eligible switchers per horizon emits NaN.""" + # Construct a panel where one path has only 2 switchers — the + # n_obs >= 3 guard should fire. Use paths_of_interest to ensure + # the rare path is selected. + rng = np.random.RandomState(45) + rows = [] + # 30 switchers on path (0,1,1,1), 2 switchers on (0,1,0,0) + for g in range(30): + F_g = 3 + (g % 3) + path = (0, 1, 1, 1) + het_x = 1 if g < 15 else 0 + for t in range(10): + if F_g - 1 <= t < F_g - 1 + len(path): + d = path[t - (F_g - 1)] + elif t >= F_g - 1 + len(path): + d = path[-1] + else: + d = 0 + rows.append({ + "group": g, "period": t, "treatment": d, + "outcome": 0.5 * t + 5.0 * d + rng.normal(0, 0.5), + "het_x": het_x, + }) + # 2 switchers on the rare path — under-eligible + for g in range(30, 32): + F_g = 3 + path = (0, 1, 0, 0) + het_x = 1 + for t in range(10): + if F_g - 1 <= t < F_g - 1 + len(path): + d = path[t - (F_g - 1)] + elif t >= F_g - 1 + len(path): + d = path[-1] + else: + d = 0 + rows.append({ + "group": g, "period": t, "treatment": d, + "outcome": 0.5 * t + 5.0 * d + rng.normal(0, 0.5), + "het_x": het_x, + }) + # Controls + for k in range(15): + for t in range(10): + rows.append({ + "group": 32 + k, "period": t, "treatment": 0, + "outcome": 0.5 * t + rng.normal(0, 0.5), "het_x": 0, + }) + df = pd.DataFrame(rows) + est = ChaisemartinDHaultfoeuille( + drop_larger_lower=False, + paths_of_interest=[(0, 1, 1, 1), (0, 1, 0, 0)], + ) + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + res = est.fit( + df, outcome="outcome", group="group", time="period", + treatment="treatment", L_max=3, heterogeneity="het_x", + ) + assert res.path_heterogeneity_effects + rare = res.path_heterogeneity_effects[(0, 1, 0, 0)] + # All horizons for the rare path should have NaN inference (n_obs < 3) + for l_h, vals in rare.items(): + assert vals["n_obs"] < 3, ( + f"rare path l={l_h}: expected n_obs < 3, got {vals['n_obs']}" + ) + assert not np.isfinite(vals["beta"]) + assert not np.isfinite(vals["se"]) + assert not np.isfinite(vals["t_stat"]) + assert not np.isfinite(vals["p_value"]) + assert not np.isfinite(vals["conf_int"][0]) + assert not np.isfinite(vals["conf_int"][1]) + + def test_per_path_heterogeneity_no_multi_baseline_warning(self): + """Anti-regression: heterogeneity + by_path does NOT emit the + multi-baseline UserWarning that controls/trends_linear emit. + Cohort dummies absorb baseline by construction (REGISTRY).""" + df = _by_path_het_data() + with warnings.catch_warnings(record=True) as caught: + warnings.simplefilter("always") + res = ChaisemartinDHaultfoeuille( + drop_larger_lower=False, by_path=3 + ).fit( + df, outcome="outcome", group="group", time="period", + treatment="treatment", L_max=3, heterogeneity="het_x", + ) + assert res.path_heterogeneity_effects + # Filter for any multi-baseline-style warning + multi_baseline = [ + w for w in caught + if "baseline" in str(w.message).lower() + and "multi" in str(w.message).lower() + ] + assert not multi_baseline, ( + f"Unexpected multi-baseline warning(s): " + f"{[str(w.message) for w in multi_baseline]}" + ) + + # Survey composition (slow) + + @staticmethod + def _by_path_het_data_with_survey(seed=44, n_replicates=0): + """Extends `_by_path_het_data` with survey columns (weights / + strata / PSU). When ``n_replicates > 0``, also attaches BRR + replicate-weight columns ``rep_0..rep_{n_replicates-1}``. + + Strata are coarser than groups (3 strata) and PSU=group for the + analytical Binder TSL path. Replicate weights are mutually + exclusive with strata/PSU/FPC at the SurveyDesign level (see + survey.py validation), so the caller picks one mode by passing + the appropriate kwargs to SurveyDesign. + """ + rng = np.random.RandomState(seed) + n_switchers, n_controls, n_periods = 90, 30, 10 + n_groups_total = n_switchers + n_controls + H = ( + rng.choice([-1, 1], size=(n_groups_total, n_replicates)) + if n_replicates > 0 + else None + ) + rows = [] + paths = [(0, 1, 1, 1), (0, 1, 0, 0), (0, 1, 1, 0)] + for g in range(n_switchers): + F_g = 3 + ((g // 3) % 3) + path = paths[g % 3] + het_x = 1 if g < n_switchers // 2 else 0 + effect = 5.0 + 3.0 * het_x + stratum = g // 30 + psu = g // 3 + weight = 1.0 + 0.1 * (g % 5) + for t in range(n_periods): + if F_g - 1 <= t < F_g - 1 + len(path): + d = path[t - (F_g - 1)] + elif t >= F_g - 1 + len(path): + d = path[-1] + else: + d = 0 + y = 0.5 * t + effect * d + rng.normal(0, 0.5) + row = { + "group": g, + "period": t, + "treatment": d, + "outcome": y, + "het_x": het_x, + "survey_weights": weight, + "strata": stratum, + "psu": psu, + } + if H is not None: + for r in range(n_replicates): + row[f"rep_{r}"] = float(weight) * (1 + 0.5 * H[g, r]) + rows.append(row) + for k in range(n_controls): + het_x = 1 if k < n_controls // 2 else 0 + g = n_switchers + k + stratum = g // 30 + psu = g // 3 + weight = 1.0 + 0.1 * (k % 5) + for t in range(n_periods): + row = { + "group": g, + "period": t, + "treatment": 0, + "outcome": 0.5 * t + rng.normal(0, 0.5), + "het_x": het_x, + "survey_weights": weight, + "strata": stratum, + "psu": psu, + } + if H is not None: + for r in range(n_replicates): + row[f"rep_{r}"] = float(weight) * (1 + 0.5 * H[g, r]) + rows.append(row) + return pd.DataFrame(rows) + + @pytest.mark.slow + def test_per_path_heterogeneity_under_survey_finite(self): + """Analytical Binder TSL SE finite per (path, l) under + ``by_path + heterogeneity + survey_design``. Wave 5 #11 plan + regression coverage for the documented survey composition + (REGISTRY: "Per-path heterogeneity testing" → "Survey + composition").""" + from diff_diff.survey import SurveyDesign + + df = self._by_path_het_data_with_survey() + sd = SurveyDesign(weights="survey_weights", strata="strata", psu="psu") + est = ChaisemartinDHaultfoeuille(drop_larger_lower=False, by_path=2) + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + res = est.fit( + df, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + L_max=3, + heterogeneity="het_x", + survey_design=sd, + ) + assert res.path_heterogeneity_effects + finite_count = 0 + for path, horizons in res.path_heterogeneity_effects.items(): + for l_h, vals in horizons.items(): + if vals["n_obs"] >= 3: + assert np.isfinite(vals["beta"]), ( + f"path={path} l={l_h}: beta is NaN under survey TSL" + ) + assert np.isfinite(vals["se"]) and vals["se"] > 0, ( + f"path={path} l={l_h}: se non-positive under survey TSL" + ) + finite_count += 1 + assert finite_count >= 4, ( + f"Expected ≥4 finite (path, l) entries, got {finite_count}" + ) + + @pytest.mark.slow + def test_per_path_heterogeneity_replicate_weights_propagates_n_valid(self): + """Under replicate weights, every per-(path, l) replicate fit + appends ``n_valid`` to the shared accumulator and the final + ``survey_metadata.df_survey`` reflects ``min(n_valid) - 1``. + + For BRR with ``n_replicates=8`` and well-formed data, the + expected df_survey is ``n_replicates - 1 = 7`` (every replicate + produces a finite SE on this DGP). Anti-regression: drives the + end-to-end `_replicate_n_valid_list` accumulator through per- + (path, l) heterogeneity calls. + """ + from diff_diff.survey import SurveyDesign + + n_replicates = 8 + df = self._by_path_het_data_with_survey(n_replicates=n_replicates) + sd = SurveyDesign( + weights="survey_weights", + replicate_weights=[f"rep_{r}" for r in range(n_replicates)], + replicate_method="BRR", + ) + est = ChaisemartinDHaultfoeuille(drop_larger_lower=False, by_path=2) + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + res = est.fit( + df, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + L_max=3, + heterogeneity="het_x", + survey_design=sd, + ) + assert res.path_heterogeneity_effects + assert res.survey_metadata is not None + # df_survey ≤ n_replicates - 1 per Rao-Wu replicate convention. + # With well-formed BRR weights and n_obs >= 3 per (path, l), we + # expect every replicate fit to produce finite SE → df = 7. + assert res.survey_metadata.df_survey is not None, ( + "df_survey must be populated under replicate-weight survey" + ) + assert res.survey_metadata.df_survey == n_replicates - 1, ( + f"df_survey={res.survey_metadata.df_survey}, " + f"expected {n_replicates - 1}" + ) + # Every populated (path, l) should have finite inference under + # replicate weights too. + for path, horizons in res.path_heterogeneity_effects.items(): + for l_h, vals in horizons.items(): + if vals["n_obs"] >= 3: + assert np.isfinite(vals["se"]), ( + f"path={path} l={l_h}: replicate SE non-finite" + ) + + @pytest.mark.slow + def test_survey_design_plus_n_bootstrap_with_heterogeneity_still_raises( + self, + ): + """The existing ``by_path + survey_design + n_bootstrap > 0`` + gate (PR #408) must still fire when ``heterogeneity`` is also + set. Anti-regression: confirms heterogeneity composition does + not accidentally re-route around the multiplier-bootstrap + gate. + """ + from diff_diff.survey import SurveyDesign + + df = self._by_path_het_data_with_survey() + sd = SurveyDesign(weights="survey_weights", strata="strata", psu="psu") + est = ChaisemartinDHaultfoeuille( + drop_larger_lower=False, by_path=2, n_bootstrap=10, seed=1 + ) + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + with pytest.raises(NotImplementedError, match="multiplier"): + est.fit( + df, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + L_max=3, + heterogeneity="het_x", + survey_design=sd, + ) + + # DataFrame integration + + def test_to_dataframe_by_path_includes_heterogeneity_columns(self): + """``to_dataframe(level='by_path')`` includes het_* columns; + populated for positive horizons and NaN for placebo rows.""" + df = _by_path_het_data() + est = ChaisemartinDHaultfoeuille( + drop_larger_lower=False, by_path=2, placebo=True + ) + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + res = est.fit( + df, outcome="outcome", group="group", time="period", + treatment="treatment", L_max=3, heterogeneity="het_x", + ) + out = res.to_dataframe(level="by_path") + assert "het_beta" in out.columns + assert "het_se" in out.columns + assert "het_t_stat" in out.columns + assert "het_p_value" in out.columns + assert "het_conf_int_lower" in out.columns + assert "het_conf_int_upper" in out.columns + # Placebo rows: het_* must be NaN + if (out.horizon < 0).any(): + placebo_rows = out[out.horizon < 0] + assert placebo_rows["het_beta"].isna().all() + # Positive horizons: at least some entries are populated + positive_rows = out[out.horizon > 0] + assert positive_rows["het_beta"].notna().any() + + def test_per_path_heterogeneity_renders_in_summary(self): + """``summary()`` includes per-path heterogeneity sub-block. + + Sibling-surface mirror of `_render_heterogeneity_section` + (global) and `path_cumulated_event_study` rendering. Anti- + regression: ensures `path_heterogeneity_effects` is not + silently omitted from the user-facing report. + """ + df = _by_path_het_data() + est = ChaisemartinDHaultfoeuille(drop_larger_lower=False, by_path=2) + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + res = est.fit( + df, outcome="outcome", group="group", time="period", + treatment="treatment", L_max=3, heterogeneity="het_x", + ) + report = res.summary() + assert "Heterogeneity Test (Section 1.5, partial)" in report, ( + "summary() must render the per-path heterogeneity sub-block" + ) + # The header appears in BOTH the global and per-path blocks; check + # that at least one populated path's beta value is rendered. We + # use a small float comparison rather than a full string match + # because `_format_inference_row` formats with 4 decimal places. + assert res.path_heterogeneity_effects + rendered_any = False + for path, horizons in res.path_heterogeneity_effects.items(): + for l_h, vals in horizons.items(): + if not np.isfinite(vals["beta"]): + continue + fragment = f"{vals['beta']:.4f}" + if fragment in report: + rendered_any = True + break + if rendered_any: + break + assert rendered_any, ( + "summary() must contain at least one per-path heterogeneity " + "beta value rounded to 4 decimal places" + ) + + # Edge cases + + def test_path_unobserved_under_heterogeneity_warns_omits(self): + """POI with unobserved path emits unobserved-path warning and + omits the path from path_heterogeneity_effects.""" + df = _by_path_het_data() + # (1, 1, 1, 0) is not in the DGP (all paths start with 0) + est = ChaisemartinDHaultfoeuille( + drop_larger_lower=False, + paths_of_interest=[(0, 1, 1, 1), (1, 1, 1, 0)], + ) + with warnings.catch_warnings(record=True) as caught: + warnings.simplefilter("always") + res = est.fit( + df, outcome="outcome", group="group", time="period", + treatment="treatment", L_max=3, heterogeneity="het_x", + ) + # Unobserved path warning should have fired (at least once; + # may fire from path_effects + path_heterogeneity_effects) + unobs = [ + w for w in caught + if "(1, 1, 1, 0)" in str(w.message) + and "zero observed groups" in str(w.message) + ] + assert unobs, "expected unobserved-path UserWarning" + assert res.path_heterogeneity_effects is not None + assert (1, 1, 1, 0) not in res.path_heterogeneity_effects + assert (0, 1, 1, 1) in res.path_heterogeneity_effects diff --git a/tests/test_chaisemartin_dhaultfoeuille_parity.py b/tests/test_chaisemartin_dhaultfoeuille_parity.py index ef1fbe4b..dd0bae4e 100644 --- a/tests/test_chaisemartin_dhaultfoeuille_parity.py +++ b/tests/test_chaisemartin_dhaultfoeuille_parity.py @@ -1319,3 +1319,154 @@ def test_parity_multi_path_reversible_by_path_non_binary( f"path={path_key} h={h} SE: " f"py={py_se:.4f} vs r={r_se:.4f}" ) + + +class TestDCDHDynRParityHeterogeneity: + """Parity tests for global ``predict_het`` against R DIDmultiplegtDYN. + + Wave 5 #11 introduces this as the FIRST ``predict_het`` parity + baseline in the repo. Anchors the per-path tolerance for + ``TestDCDHDynRParityByPathHeterogeneity`` since the per-path R call + is ``did_multiplegt_main(..., predict_het=...)`` per path-restricted + subsample with no additional numerical loss. + + Fixture: scenario 20 (``multi_path_reversible_predict_het``) — 90 + switchers across 3 reversal paths + 30 never-treated controls, + binary ``het_x`` with effect = 5.0 + 3.0 * het_x. R is invoked with + ``dont_drop_larger_lower=TRUE`` so the reversal-path eligibility + matches Python's ``drop_larger_lower=False`` requirement. + """ + + BETA_RTOL = 1e-6 + BETA_ATOL = 1e-9 + SE_RTOL = 1e-5 + + def test_parity_multi_path_reversible_predict_het(self, golden_values): + """Global heterogeneity coefficient parity per horizon.""" + import warnings + + scenario = golden_values.get("multi_path_reversible_predict_het") + if scenario is None: + pytest.skip( + "scenario 'multi_path_reversible_predict_het' not in " + "golden values" + ) + + df = _golden_to_df_with_extra(scenario["data"], extra_cols=["het_x"]) + est = ChaisemartinDHaultfoeuille(drop_larger_lower=False) + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + results = est.fit( + df, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + L_max=3, + heterogeneity="het_x", + ) + + assert results.heterogeneity_effects is not None + r_predict_het = scenario["results"]["predict_het"] + + for h_str, r_h in r_predict_het.items(): + h = int(h_str) + assert h in results.heterogeneity_effects, ( + f"horizon {h} missing from Python heterogeneity_effects" + ) + py_h = results.heterogeneity_effects[h] + assert py_h["beta"] == pytest.approx( + r_h["beta"], rel=self.BETA_RTOL, abs=self.BETA_ATOL + ), f"h={h} beta: py={py_h['beta']:.6f} vs r={r_h['beta']:.6f}" + assert py_h["se"] == pytest.approx(r_h["se"], rel=self.SE_RTOL), ( + f"h={h} se: py={py_h['se']:.6f} vs r={r_h['se']:.6f}" + ) + + +class TestDCDHDynRParityByPathHeterogeneity: + """Parity tests for ``by_path`` + ``predict_het`` against R DIDmultiplegtDYN. + + Wave 5 #11 lift. R's per-path dispatcher + (``did_multiplegt_dyn.R:226-257``) re-runs + ``did_multiplegt_main(..., predict_het=...)`` on each path-restricted + subsample. Python mirrors via ``_compute_path_heterogeneity_test``, + which loops ``_compute_heterogeneity_test`` over selected paths + with a ``path_groups`` filter restricting eligibility to the path's + switchers. Cohort dummies absorb baseline by construction, so + multi-baseline switcher panels do not produce R-divergence. + + Fixture: scenario 21 (``multi_path_reversible_by_path_predict_het``) + — same DGP as scenario 20 so per-path tolerances inherit from + ``TestDCDHDynRParityHeterogeneity``. + """ + + BETA_RTOL = 1e-6 + BETA_ATOL = 1e-9 + SE_RTOL = 1e-5 + + def _path_key_from_r_label(self, r_label: str): + return tuple(int(x) for x in r_label.split(",")) + + def test_parity_multi_path_reversible_by_path_predict_het( + self, golden_values + ): + """Per-path heterogeneity coefficient parity per (path, horizon).""" + import warnings + + scenario = golden_values.get( + "multi_path_reversible_by_path_predict_het" + ) + if scenario is None: + pytest.skip( + "scenario 'multi_path_reversible_by_path_predict_het' " + "not in golden values" + ) + + df = _golden_to_df_with_extra(scenario["data"], extra_cols=["het_x"]) + est = ChaisemartinDHaultfoeuille(drop_larger_lower=False, by_path=3) + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + results = est.fit( + df, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + L_max=3, + heterogeneity="het_x", + ) + + assert results.path_heterogeneity_effects is not None + r_by_path = scenario["results"]["by_path_predict_het"] + + py_keys = set(results.path_heterogeneity_effects.keys()) + r_keys = {self._path_key_from_r_label(e["path"]) for e in r_by_path} + assert py_keys == r_keys, ( + f"Path-set mismatch.\n" + f" Python only: {py_keys - r_keys}\n" + f" R only: {r_keys - py_keys}" + ) + + for r_path_entry in r_by_path: + path_key = self._path_key_from_r_label(r_path_entry["path"]) + py_horizons = results.path_heterogeneity_effects[path_key] + + for h_str, r_h in r_path_entry["horizons"].items(): + h = int(h_str) + assert h in py_horizons, ( + f"path={path_key}: horizon {h} missing from Python " + f"path_heterogeneity_effects" + ) + py_h = py_horizons[h] + assert py_h["beta"] == pytest.approx( + r_h["beta"], rel=self.BETA_RTOL, abs=self.BETA_ATOL + ), ( + f"path={path_key} h={h} beta: " + f"py={py_h['beta']:.6f} vs r={r_h['beta']:.6f}" + ) + assert py_h["se"] == pytest.approx( + r_h["se"], rel=self.SE_RTOL + ), ( + f"path={path_key} h={h} se: " + f"py={py_h['se']:.6f} vs r={r_h['se']:.6f}" + )