Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .claude/commands/bump-version.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ Files that need updating:
| `pyproject.toml` | `version = "X.Y.Z"` | ~7 |
| `rust/Cargo.toml` | `version = "X.Y.Z"` | ~3 |
| `CHANGELOG.md` | Section header + comparison link | Top + bottom |
| `docs/llms-full.txt` | `- Version: X.Y.Z` | ~5 |
| `diff_diff/guides/llms-full.txt` | `- Version: X.Y.Z` | ~5 |

## Instructions

Expand Down Expand Up @@ -80,7 +80,7 @@ Files that need updating:
Replace `version = "OLD_VERSION"` (the first version line under [package]) with `version = "NEW_VERSION"`
Note: Rust version may differ from Python version; always sync to the new version

- `docs/llms-full.txt`:
- `diff_diff/guides/llms-full.txt`:
Replace `- Version: OLD_VERSION` with `- Version: NEW_VERSION`

6. **Update CHANGELOG comparison links**:
Expand All @@ -101,7 +101,7 @@ Files that need updating:
- diff_diff/__init__.py: __version__ = "NEW_VERSION"
- pyproject.toml: version = "NEW_VERSION"
- rust/Cargo.toml: version = "NEW_VERSION"
- docs/llms-full.txt: Version: NEW_VERSION
- diff_diff/guides/llms-full.txt: Version: NEW_VERSION
- CHANGELOG.md: Added/verified [NEW_VERSION] entry

Next steps:
Expand Down
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,7 +139,7 @@ category (`Methodology/Correctness`, `Performance`, or `Testing/Docs`):
| `CONTRIBUTING.md` | Documentation requirements, test writing guidelines |
| `.claude/commands/dev-checklists.md` | Checklists for params, methodology, warnings, reviews, bugs (run `/dev-checklists`) |
| `.claude/memory.md` | Debugging patterns, tolerances, API conventions (git-tracked) |
| `docs/llms-practitioner.txt` | Baker et al. (2025) 8-step practitioner workflow for AI agents |
| `diff_diff/guides/llms-practitioner.txt` | Baker et al. (2025) 8-step practitioner workflow for AI agents (accessible at runtime via `diff_diff.get_llm_guide("practitioner")`) |
| `docs/performance-plan.md` | Performance optimization details |
| `docs/benchmarks.rst` | Validation results vs R |

Expand Down
14 changes: 11 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,11 +69,19 @@ Signif. codes: '***' 0.001, '**' 0.01, '*' 0.05, '.' 0.1

## For AI Agents

If you are an AI agent or LLM using this library, read [`docs/llms.txt`](docs/llms.txt) for a concise API reference with an 8-step practitioner workflow (based on Baker et al. 2025). The workflow ensures rigorous DiD analysis — not just calling `fit()`, but testing assumptions, running sensitivity analysis, and checking robustness.
If you are an AI agent or LLM using this library, call `diff_diff.get_llm_guide()` for a concise API reference with an 8-step practitioner workflow (based on Baker et al. 2025). The workflow ensures rigorous DiD analysis — not just calling `fit()`, but testing assumptions, running sensitivity analysis, and checking robustness.

After estimation, call `practitioner_next_steps(results)` for context-aware guidance on remaining diagnostic steps.
```python
from diff_diff import get_llm_guide

get_llm_guide() # concise API reference
get_llm_guide("practitioner") # 8-step workflow (Baker et al. 2025)
get_llm_guide("full") # comprehensive documentation
```

The guides are bundled in the wheel, so they are accessible from a `pip install` with no network access required.

Detailed guide: [`docs/llms-practitioner.txt`](docs/llms-practitioner.txt)
After estimation, call `practitioner_next_steps(results)` for context-aware guidance on remaining diagnostic steps.

## For Data Scientists

Expand Down
15 changes: 10 additions & 5 deletions diff_diff/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,14 @@
This library provides sklearn-like estimators for causal inference
using the difference-in-differences methodology.

For rigorous analysis, follow the 8-step practitioner workflow in
docs/llms-practitioner.txt (based on Baker et al. 2025). After
estimation, call ``practitioner_next_steps(results)`` for context-aware
guidance on remaining diagnostic steps.
For rigorous analysis, follow the 8-step practitioner workflow based
on Baker et al. (2025). After estimation, call
``practitioner_next_steps(results)`` for context-aware guidance on
remaining diagnostic steps.

AI agent reference: docs/llms.txt
AI agents: call ``diff_diff.get_llm_guide()`` for a complete API reference.
Use ``get_llm_guide("practitioner")`` for the 8-step workflow or
``get_llm_guide("full")`` for comprehensive documentation.
"""

# Import backend detection from dedicated module (avoids circular imports)
Expand Down Expand Up @@ -200,6 +202,7 @@
plot_synth_weights,
)
from diff_diff.practitioner import practitioner_next_steps
from diff_diff._guides_api import get_llm_guide
from diff_diff.datasets import (
clear_cache,
list_datasets,
Expand Down Expand Up @@ -402,4 +405,6 @@
"clear_cache",
# Practitioner guidance
"practitioner_next_steps",
# LLM guide accessor
"get_llm_guide",
]
48 changes: 48 additions & 0 deletions diff_diff/_guides_api.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
"""Runtime accessor for bundled LLM guide files."""
from __future__ import annotations

from importlib.resources import files

_VARIANT_TO_FILE = {
"concise": "llms.txt",
"full": "llms-full.txt",
"practitioner": "llms-practitioner.txt",
}


def get_llm_guide(variant: str = "concise") -> str:
"""Return the contents of a bundled LLM guide.

Parameters
----------
variant : str, default "concise"
Which guide to load. Names are case-sensitive. One of:

- ``"concise"`` -- compact API reference (llms.txt)
- ``"full"`` -- complete API documentation (llms-full.txt)
- ``"practitioner"`` -- 8-step practitioner workflow (llms-practitioner.txt)

Returns
-------
str
The full text of the requested guide.

Raises
------
ValueError
If ``variant`` is not one of the known guide names.

Examples
--------
>>> from diff_diff import get_llm_guide
>>> concise = get_llm_guide()
>>> workflow = get_llm_guide("practitioner")
"""
try:
filename = _VARIANT_TO_FILE[variant]
except (KeyError, TypeError):
valid = ", ".join(repr(k) for k in _VARIANT_TO_FILE)
raise ValueError(
f"Unknown guide variant {variant!r}. Valid options: {valid}."
) from None
return files("diff_diff.guides").joinpath(filename).read_text(encoding="utf-8")
1 change: 1 addition & 0 deletions diff_diff/guides/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
"""LLM guide files bundled with diff-diff."""
8 changes: 7 additions & 1 deletion docs/llms-full.txt → diff_diff/guides/llms-full.txt
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ print(f"ATT: {results.att:.3f} (SE: {results.se:.3f})")

## Practitioner Workflow (based on Baker et al. 2025)

For rigorous DiD analysis, follow the 8-step framework in docs/llms-practitioner.txt.
For rigorous DiD analysis, follow the 8-step framework (call `diff_diff.get_llm_guide("practitioner")`).
After estimation, call:

```python
Expand Down Expand Up @@ -1029,6 +1029,12 @@ Returned by `SyntheticDiD.fit()`.

**Methods:** `summary()`, `print_summary()`, `to_dict()`, `to_dataframe()`, `get_unit_weights_df()`, `get_time_weights_df()`

**Validation diagnostics** (call after `fit()`):
- `get_weight_concentration(top_k=5)` - effective N and top-k weight share; flags fragile synthetic controls dominated by a few donor units
- `get_loo_effects_df()` - per-unit leave-one-out influence from the jackknife pass (DataFrame includes both control and treated rows). Requires `variance_method="jackknife"`; raises `ValueError` if LOO is unavailable (see the method docstring for the full set of conditions, e.g. single treated unit or only one control with nonzero effective weight)
- `in_time_placebo()` - re-estimate on shifted fake treatment dates in the pre-period; near-zero placebo ATTs indicate a credible design
- `sensitivity_to_zeta_omega()` - re-estimate across a grid of unit-weight regularization values; checks ATT robustness to the auto-selected zeta_omega

### TripleDifferenceResults

Returned by `TripleDifference.fit()`.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,21 @@
> An 8-step workflow for rigorous Difference-in-Differences analysis, based on
> Baker et al. (2025) "Difference-in-Differences Designs: A Practitioner's
> Guide" and adapted for the diff-diff library. Some steps are reorganized or
> extended relative to the paper (see docs/methodology/REGISTRY.md for details).
> extended relative to the paper:
>
> - **Numbering**: diff-diff uses 1-Define, 2-Assumptions, 3-Test PT,
> 4-Choose estimator, 5-Estimate, 6-Sensitivity, 7-Heterogeneity,
> 8-Robustness. The paper uses 1-Define, 2-Assumptions, 3-Estimation method,
> 4-Uncertainty, 5-Estimate, 6-Sensitivity, 7-Heterogeneity, 8-Keep learning.
> - **Parallel trends testing** is a separate Step 3 (the paper embeds it in
> Step 2), to ensure AI agents execute it as a distinct action.
> - **Sources of uncertainty** (paper's Step 4) are folded into Step 5
> (Estimate) with an explicit cluster-count check directive: >= 50 clusters
> for asymptotic SEs, otherwise wild bootstrap. The 50-cluster threshold is
> a diff-diff convention.
> - **Step 8** is "Robustness & Reporting" (compare estimators, report with
> and without covariates). The paper's Step 8 is "Keep learning." The
> mandatory with/without covariate comparison is a diff-diff convention.

## Instructions for AI Agents

Expand Down Expand Up @@ -282,7 +296,8 @@ to your estimator's API. Examples:
- **StackedDiD**: vary `clean_control` definition
- **EfficientDiD**: compare `control_group='never_treated'` vs `'last_cohort'`
- **ImputationDiD/TwoStageDiD**: leave-one-cohort-out, cross-estimator comparison
- **SyntheticDiD/TROP**: in-time or in-space placebo (fake treatment date, leave-one-unit-out)
- **SyntheticDiD**: built-in diagnostics on the results object - `results.in_time_placebo()`, `results.get_loo_effects_df()` (requires `variance_method="jackknife"` at fit time), `results.sensitivity_to_zeta_omega()`, and `results.get_weight_concentration()`
- **TROP**: in-time or in-space placebo (fake treatment date, leave-one-unit-out)

```python
from diff_diff import run_all_placebo_tests
Expand Down
4 changes: 2 additions & 2 deletions docs/llms.txt → diff_diff/guides/llms.txt
Original file line number Diff line number Diff line change
Expand Up @@ -27,13 +27,13 @@ diagnostic steps produces unreliable results.
After estimation, call `practitioner_next_steps(results)` for context-aware
guidance on remaining steps.

Full practitioner guide: docs/llms-practitioner.txt
Full practitioner guide: call `diff_diff.get_llm_guide("practitioner")`

## Documentation

### Getting Started

- [Practitioner Guide](docs/llms-practitioner.txt): 8-step workflow for rigorous DiD analysis (Baker et al. 2025) — **start here**
- **Practitioner Guide** (call `diff_diff.get_llm_guide("practitioner")`): 8-step workflow for rigorous DiD analysis (Baker et al. 2025) — **start here**
- [Quickstart](https://diff-diff.readthedocs.io/en/stable/quickstart.html): Installation, basic 2x2 DiD — column-name and formula interfaces, covariates, fixed effects, cluster-robust SEs
- [Choosing an Estimator](https://diff-diff.readthedocs.io/en/stable/choosing_estimator.html): Decision flowchart for selecting the right estimator for your research design
- [Troubleshooting](https://diff-diff.readthedocs.io/en/stable/troubleshooting.html): Common issues and solutions
Expand Down
8 changes: 6 additions & 2 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@
]

templates_path = ["_templates"]
exclude_patterns = ["_build", "Thumbs.db", ".DS_Store", "llms.txt", "llms-full.txt"]
exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]

# -- Options for autodoc -----------------------------------------------------
autodoc_default_options = {
Expand Down Expand Up @@ -71,7 +71,11 @@
"https://diff-diff.readthedocs.io/en/stable/",
)
html_baseurl = _canonical_url
html_extra_path = ["llms.txt", "llms-full.txt"]
html_extra_path = [
"../diff_diff/guides/llms.txt",
"../diff_diff/guides/llms-full.txt",
"../diff_diff/guides/llms-practitioner.txt",
]
sitemap_url_scheme = "{link}"

html_theme_options = {
Expand Down
Loading
Loading