Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
b1946fb
Close BR/DR gap #6: target-parameter clarity block in schemas
igerber Apr 20, 2026
7e8e264
Address PR #347 R1: fix StackedDiD wording, drop dead TWFE branch, fi…
igerber Apr 20, 2026
9f6b4d1
Address PR #347 R2: fix dCDH dynamic-path target-parameter contract
igerber Apr 20, 2026
450c592
Address PR #347 R3: sync docs to implemented aggregation tags + real-…
igerber Apr 20, 2026
a8e1719
Address PR #347 R4: propagate no_scalar_headline through BR/DR; Woold…
igerber Apr 20, 2026
e995954
Address PR #347 R5: DR full_report no-scalar branch + stale Wooldridg…
igerber Apr 21, 2026
982b7cb
Address PR #347 R6: drop brittle line ref from StackedDiD; sync llms-…
igerber Apr 21, 2026
5c3d0ba
Apply black formatting to _reporting_helpers.py
igerber Apr 21, 2026
fdaf94d
Address PR #347 R7: bump schema versions to 2.0 + EfficientDiD librar…
igerber Apr 21, 2026
f2fc763
Address PR #347 R8: fix TROP rst wording + add Bacon-branch regressio…
igerber Apr 21, 2026
c0d0127
Address PR #347 R9: persist trends_linear on dCDH result for empty-su…
igerber Apr 21, 2026
5a82ab7
Address PR #347 R10: propagate persisted trends_linear flag to all dC…
igerber Apr 22, 2026
696d2cd
Address PR #347 R11: cite Wooldridge 2025 (not 2023) for OLS ETWFE path
igerber Apr 22, 2026
8343eeb
Address PR #347 R12: empty-surface dCDH messaging + did_or_twfe aggre…
igerber Apr 22, 2026
8b72e23
Address PR #347 R13: route no-scalar renderers through headline.reaso…
igerber Apr 22, 2026
5046a51
Apply black formatting to business_report.py
igerber Apr 22, 2026
d6ab7ec
Address PR #347 R14: control-aware empty-surface label (DID^{X,fd}_l)
igerber Apr 22, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Added
- **`target_parameter` block in BR/DR schemas (experimental; schema version bumped to 2.0)** — `BUSINESS_REPORT_SCHEMA_VERSION` and `DIAGNOSTIC_REPORT_SCHEMA_VERSION` bumped from `"1.0"` to `"2.0"` because the new `"no_scalar_by_design"` value on the `headline.status` / `headline_metric.status` enum (dCDH `trends_linear=True, L_max>=2` configuration) is a breaking change per the REPORTING.md stability policy. BusinessReport and DiagnosticReport now emit a top-level `target_parameter` block naming what the headline scalar actually represents for each of the 16 result classes. Closes BR/DR foundation gap #6 (target-parameter clarity). Fields: `name`, `definition`, `aggregation` (machine-readable dispatch tag), `headline_attribute` (raw result attribute), `reference` (citation pointer). BR's summary emits the short `name` right after the headline; DR's overall-interpretation paragraph does the same; both full reports carry a "## Target Parameter" section with the full definition. Per-estimator dispatch is sourced from REGISTRY.md and lives in the new `diff_diff/_reporting_helpers.py::describe_target_parameter`. A few branches read fit-time config (`EfficientDiDResults.pt_assumption`, `StackedDiDResults.clean_control`, `ChaisemartinDHaultfoeuilleResults.L_max` / `covariate_residuals` / `linear_trends_effects`); others emit a fixed tag (the fit-time `aggregate` kwarg on CS / Imputation / TwoStage / Wooldridge does not change the `overall_att` scalar — disambiguating horizon / group tables is tracked under gap #9). See `docs/methodology/REPORTING.md` "Target parameter" section.

## [3.2.0] - 2026-04-19

### Added
Expand Down
634 changes: 634 additions & 0 deletions diff_diff/_reporting_helpers.py

Large diffs are not rendered by default.

132 changes: 127 additions & 5 deletions diff_diff/business_report.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,9 +42,10 @@

import numpy as np

from diff_diff._reporting_helpers import describe_target_parameter
from diff_diff.diagnostic_report import DiagnosticReport, DiagnosticReportResults

BUSINESS_REPORT_SCHEMA_VERSION = "1.0"
BUSINESS_REPORT_SCHEMA_VERSION = "2.0"

__all__ = [
"BusinessReport",
Expand Down Expand Up @@ -432,7 +433,67 @@ def _build_schema(self) -> Dict[str, Any]:
diagnostics_results.schema if diagnostics_results is not None else None
)

headline = self._extract_headline(dr_schema)
# PR #347 R4 P1: compute target_parameter BEFORE extracting
# the headline so the no-scalar-by-design case
# (``aggregation == "no_scalar_headline"``, e.g., dCDH
# ``trends_linear=True`` with ``L_max >= 2``) can route the
# headline through a dedicated branch that names the intentional
# NaN rather than an estimation-failure path.
target_parameter = describe_target_parameter(self._results)
if target_parameter.get("aggregation") == "no_scalar_headline":
# PR #347 R12 P1: the no-scalar ``reason`` must distinguish
# the populated-surface case (per-horizon table exists) from
# the empty-surface subcase (``linear_trends_effects=None``
# — no horizons survived estimation). Telling a user with
# an empty surface to "see linear_trends_effects" is
# dead-end guidance.
_surface_empty = getattr(self._results, "linear_trends_effects", None) is None
# PR #347 R14 P1: the empty-surface reason must use the
# covariate-adjusted label when covariates are active.
_has_controls = getattr(self._results, "covariate_residuals", None) is not None
_empty_surface_label = "DID^{X,fd}_l" if _has_controls else "DID^{fd}_l"
if _surface_empty:
no_scalar_reason = (
"The fitted estimator intentionally does not produce a "
"scalar overall ATT on this configuration "
"(``trends_linear=True`` with ``L_max >= 2``), and on "
f"this fit no cumulated level effects ``{_empty_surface_label}`` "
"survived estimation — the per-horizon surface is "
"empty. Re-fit with a larger ``L_max`` or with "
"``trends_linear=False`` if you need a reportable "
"estimand."
)
else:
no_scalar_reason = (
"The fitted estimator intentionally does not produce a "
"scalar overall ATT on this configuration "
"(``trends_linear=True`` with ``L_max >= 2``). Per-horizon "
"cumulated level effects are on "
"``results.linear_trends_effects[l]``."
)
headline = {
"status": "no_scalar_by_design",
"effect": None,
"se": None,
"ci_lower": None,
"ci_upper": None,
"alpha_was_honored": True,
"alpha_override_caveat": None,
"ci_level": int(round((1.0 - self._context.alpha) * 100)),
"p_value": None,
"is_significant": False,
"near_significance_threshold": False,
"unit": self._context.outcome_unit,
"unit_kind": _UNIT_KINDS.get(
self._context.outcome_unit.lower() if self._context.outcome_unit else "",
"unknown",
),
"sign": "none",
"breakdown_M": None,
"reason": no_scalar_reason,
}
else:
headline = self._extract_headline(dr_schema)
sample = self._extract_sample()
heterogeneity = _lift_heterogeneity(dr_schema)
pre_trends = _lift_pre_trends(dr_schema)
Expand Down Expand Up @@ -475,6 +536,7 @@ def _build_schema(self) -> Dict[str, Any]:
"alpha": self._context.alpha,
},
"headline": headline,
"target_parameter": target_parameter,
"assumption": assumption,
"pre_trends": pre_trends,
"sensitivity": sensitivity,
Expand Down Expand Up @@ -1167,9 +1229,18 @@ def _describe_assumption(estimator_name: str, results: Any = None) -> Dict[str,
has_controls = (
results is not None and getattr(results, "covariate_residuals", None) is not None
)
has_trends = (
results is not None and getattr(results, "linear_trends_effects", None) is not None
)
# PR #347 R10 P1: read the persisted ``trends_linear`` flag
# first — empty-horizon trends-linear fits set
# ``linear_trends_effects=None`` but are still trends-linear
# per the estimator contract. Legacy fit objects predating
# the persisted field fall back to the presence inference.
_trends_persisted = getattr(results, "trends_linear", None) if results is not None else None
if isinstance(_trends_persisted, bool):
has_trends = _trends_persisted
else:
has_trends = (
results is not None and getattr(results, "linear_trends_effects", None) is not None
)
has_heterogeneity = (
results is not None and getattr(results, "heterogeneity_effects", None) is not None
)
Expand Down Expand Up @@ -1928,6 +1999,31 @@ def _render_headline_sentence(schema: Dict[str, Any]) -> str:
"""
ctx = schema.get("context", {})
h = schema.get("headline", {})
# PR #347 R4 P1: the dCDH ``trends_linear=True`` + ``L_max>=2``
# configuration does not produce a scalar headline by design —
# ``overall_att`` is intentionally NaN (per
# ``chaisemartin_dhaultfoeuille.py:2828-2834``). Render explicit
# "no scalar headline by design" prose instead of routing through
# the non-finite / estimation-failure path.
if h.get("status") == "no_scalar_by_design":
# PR #347 R13 P1: the headline-level ``reason`` field is the
# single source for the no-scalar prose and is already
# branched on populated-vs-empty surface in ``_build_schema``.
# Use it verbatim so the headline sentence never drifts from
# the schema-level message on the empty-surface subcase.
treatment = ctx.get("treatment_label", "the treatment")
outcome_label = ctx.get("outcome_label", "the outcome")
treatment_sentence = _sentence_first_upper(treatment)
reason = h.get("reason")
if isinstance(reason, str) and reason:
return (
f"{treatment_sentence} does not produce a scalar aggregate "
f"effect on {outcome_label} under this configuration. " + reason
)
return (
f"{treatment_sentence} does not produce a scalar aggregate effect "
f"on {outcome_label} under this configuration (by design)."
)
effect = h.get("effect")
outcome = ctx.get("outcome_label", "the outcome")
treatment = ctx.get("treatment_label", "the treatment")
Expand Down Expand Up @@ -1993,6 +2089,17 @@ def _render_summary(schema: Dict[str, Any]) -> str:

# Headline sentence with significance phrase.
sentences.append(_render_headline_sentence(schema))
# BR/DR gap #6 (target-parameter clarity): name what the headline
# scalar actually represents so the stakeholder can map the number
# to a specific estimand. Rendered immediately after the headline
# and before the significance phrase. The summary surfaces only
# the short ``name`` so the paragraph stays within the
# 6-10-sentence target; ``definition`` lives in the full report
# and in the structured schema for agents that want the long form.
tp = schema.get("target_parameter", {}) or {}
tp_name = tp.get("name")
if tp_name:
sentences.append(f"Target parameter: {tp_name}.")
h = schema.get("headline", {})
p = h.get("p_value")
alpha = ctx.get("alpha", 0.05)
Expand Down Expand Up @@ -2314,6 +2421,21 @@ def _render_full_report(schema: Dict[str, Any]) -> str:
lines.append(f"Statistically, {_significance_phrase(p, alpha)}.")
lines.append("")

# Target parameter (BR/DR gap #6): name what the headline scalar
# represents so the stakeholder can map the number to a specific
# estimand. Rendered between "Headline" and "Identifying Assumption"
# because the target parameter is about what the scalar IS, whereas
# identifying assumption is about what makes it valid.
tp = schema.get("target_parameter", {}) or {}
if tp.get("name") or tp.get("definition"):
lines.append("## Target Parameter")
lines.append("")
if tp.get("name"):
lines.append(f"- **{tp['name']}**")
if tp.get("definition"):
lines.append(f"- {tp['definition']}")
lines.append("")

# Identifying assumption
lines.append("## Identifying Assumption")
lines.append("")
Expand Down
Loading
Loading