LWDiDResults.aggregate() discards composite staggered inference

[shawcharles]

Summary

For the regression-adjustment, classical-variance, never-treated staggered path, fit() computes overall inference from a composite regression. Calling result.aggregate() afterwards recomputes the same ATT with a different, narrower standard error under an explicit cohort-independence assumption and drops the fitted degrees of freedom.

This leaves two public routes to the same overall estimand with materially different inference.

Reproduction

from diff_diff import load_castle_doctrine
from diff_diff.lwdid import LWDiD

castle = load_castle_doctrine()

for rolling in ("demean", "detrend"):
    result = LWDiD(
        rolling=rolling,
        estimator="ra",
        vce="classical",
        control_group="never_treated",
    ).fit(
        castle,
        outcome="homicide_rate",
        unit="state",
        time="year",
        treatment="treated",
        cohort="first_treat",
    )
    recomputed = result.aggregate()
    print(rolling)
    print("fit", result.att, result.se, result.df_inference)
    print("aggregate", recomputed.att, recomputed.se, recomputed.df_inference)

On the current PR head:

demean
fit       0.49983509032120127 0.1031351564132827 47
aggregate 0.4998350903212014  0.08306826962383193 None

detrend
fit       0.675120844690289  0.17851152865445166 47
aggregate 0.6751208446902892 0.16734104453234525 None

The point estimate agrees, but the post-fit route reports a narrower standard error and loses the finite-sample degrees of freedom.

Cause

The fit path uses composite-regression inference:

diff-diff/diff_diff/lwdid.py

Lines 988 to 1008 in c2694ff

	# Use composite outcome regression (LW 2026 Eq 7.18/7.19) for
	# the overall ATT and SE when control_group='never_treated' and
	# estimator='ra' with classical VCE and no controls. This produces
	# the paper's OLS SE. For non-classical VCE, fall back to delta method.
	use_composite = (
	self.control_group == "never_treated"
	and self.estimator == "ra"
	and not controls
	and self.vce == "classical"
	)
	if use_composite:
	att_overall, se_overall, df_overall = self._composite_regression_aggregation(
	df, outcome, unit, time, cohort
	)
	df_overall = max(df_overall, 1)
	else:
	att_overall, se_overall = self._aggregate_cohort_effects(
	cohort_effects, total_treated
	)
	df_overall = max(sum(e["df"] for e in cohort_effects), 1)

LWDiDResults.aggregate() instead states that it assumes cohort independence and uses a delta-method aggregation without retaining df_inference:

diff-diff/diff_diff/lwdid_results.py

Lines 306 to 429 in c2694ff

	def aggregate(self, by: str = "overall") -> LWDiDResults:
	"""Aggregate staggered cohort effects to a single ATT.
	Parameters
	----------
	by : str, default 'overall'
	Aggregation method. Currently supports 'overall' (weighted average
	across cohorts using cohort sample sizes).
	Returns
	-------
	LWDiDResults
	New results object with the aggregated ATT.
	Raises
	------
	ValueError
	If called on non-staggered results or with an unsupported method.
	"""
	if not self.is_staggered:
	raise ValueError(
	"aggregate() is only available for staggered results " "with cohort_effects."
	)
	if by != "overall":
	raise ValueError(f"Unsupported aggregation method: {by!r}")
	cohorts = self.cohort_effects
	atts = []
	weights = []
	for cohort, eff in cohorts.items(): # type: ignore[union-attr]
	att_c = eff.get("att", np.nan)
	n_c = eff.get("n_treated", 1)
	if not np.isnan(att_c):
	atts.append(att_c)
	weights.append(n_c)
	if not atts:
	return LWDiDResults(
	att=np.nan,
	se=np.nan,
	t_stat=np.nan,
	p_value=np.nan,
	conf_int=(np.nan, np.nan),
	n_obs=self.n_obs,
	n_treated=self.n_treated,
	n_control=self.n_control,
	rolling=self.rolling,
	estimator=self.estimator,
	vce_type=self.vce_type,
	alpha=self.alpha,
	cluster_name=self.cluster_name,
	n_clusters=self.n_clusters,
	)
	w = np.array(weights, dtype=float)
	w = w / w.sum()
	a = np.array(atts, dtype=float)
	agg_att = float(np.dot(w, a))
	# Aggregate SEs via delta method (independence across cohorts)
	# Exclude cohorts with NaN or non-positive SE from aggregation
	valid_mask_list = []
	for i, (cohort, eff) in enumerate(
	(c, e) for c, e in cohorts.items() if not np.isnan(e.get("att", np.nan)) # type: ignore[union-attr]
	):
	se_c = eff.get("se", np.nan)
	valid_mask_list.append(np.isfinite(se_c) and se_c > 0)
	valid_mask = np.array(valid_mask_list, dtype=bool)
	if not valid_mask.any():
	agg_se = np.nan
	agg_t = np.nan
	agg_p = np.nan
	agg_ci = (np.nan, np.nan)
	else:
	# Re-normalize weights for valid SEs only
	ses = []
	for cohort, eff in cohorts.items(): # type: ignore[union-attr]
	se_c = eff.get("se", np.nan)
	if not np.isnan(eff.get("att", np.nan)):
	ses.append(se_c)
	se_arr = np.array(ses, dtype=float)
	# Only use valid (finite, positive) SEs
	w_valid = w[valid_mask]
	w_valid = w_valid / w_valid.sum()
	se_valid = se_arr[valid_mask]
	agg_se = float(np.sqrt(np.dot(w_valid**2, se_valid**2)))
	# Use safe_inference with t-distribution for proper inference
	from diff_diff.utils import safe_inference
	# Use sum of cluster counts or residual df for aggregation
	_agg_df = (
	max(int(np.sum(valid_mask)) - 1, 1)
	if self.n_clusters is None
	else max(self.n_clusters - 1, 1)
	)
	agg_t, agg_p, agg_ci = safe_inference(agg_att, agg_se, alpha=self.alpha, df=_agg_df)
	return LWDiDResults(
	att=agg_att,
	se=agg_se,
	t_stat=agg_t,
	p_value=agg_p,
	conf_int=agg_ci,
	n_obs=self.n_obs,
	n_treated=self.n_treated,
	n_control=self.n_control,
	rolling=self.rolling,
	estimator=self.estimator,
	vce_type=self.vce_type,
	alpha=self.alpha,
	cluster_name=self.cluster_name,
	n_clusters=self.n_clusters,
	cohort_effects=self.cohort_effects,
	overall_att={
	"att": agg_att,
	"se": agg_se,
	"t_stat": agg_t,
	"p_value": agg_p,
	"conf_int": agg_ci,
	},
	)

Cohort effects share controls and are not generally independent, so this recomputation is not interchangeable with the fitted composite result.

Expected behaviour

For an overall result already computed by fit(), .aggregate("overall") should preserve the canonical ATT, standard error, confidence interval, and degrees of freedom. If an alternative aggregation is required, it needs the stored joint covariance/provenance and an explicit API distinction.

Acceptance criteria

• A regression test asserts exact agreement between the fitted overall result and .aggregate("overall") for the composite-regression path.
• df_inference is retained wherever finite-sample inference is reported.
• Any independently recomputed aggregation uses appropriate joint covariance, rather than an undocumented cohort-independence assumption.

[shawcharles]

I have proposed a focused implementation in gorgeousfish#1, targeting the source branch of #588. It preserves the canonical joint inference computed by fit() and adds an end-to-end regression test requiring exact agreement between fit() and aggregate(\"overall\") for ATT, SE, CI, p-value, and degrees of freedom.