Skip to content

ConleySpatialHAC methodology-review-tracker promotion + Bertanha-Imbens 2014 citation correction#496

Merged
igerber merged 1 commit into
mainfrom
feature/conley-methodology-promotion-citation-fix
May 26, 2026
Merged

ConleySpatialHAC methodology-review-tracker promotion + Bertanha-Imbens 2014 citation correction#496
igerber merged 1 commit into
mainfrom
feature/conley-methodology-promotion-citation-fix

Conversation

@igerber
Copy link
Copy Markdown
Owner

@igerber igerber commented May 26, 2026

Summary

Methodology references

  • Method name(s): ConleySpatialHAC

  • Paper / source link(s): Conley, T. G. (1999), GMM Estimation with Cross-Sectional Dependence, J. Econometrics 92(1), 1-45. Paper review on file: docs/methodology/papers/conley-1999-review.md. Secondary: Andrews (1991) HAC theory; Düsterhöft (2021) conleyreg (CRAN v0.1.9) parity target; Colella, Lalive, Sakalli & Thoenig (2019) Stata acreg parallel.

  • Any intentional deviations from the source (and why):

    All documented in docs/methodology/REGISTRY.md § ConleySpatialHAC + METHODOLOGY_REVIEW.md § ConleySpatialHAC Deviations block.

Validation

  • Tests added/updated:
    • tests/test_methodology_conley.py (CREATE, ~1600 LoC, 10 classes, 60 tests)
    • tests/test_conley_vcov.py (EDIT, methodology-anchored tests extracted; defensive surface preserved; ~1135 LoC removed)
  • Backtest / simulation / notebook evidence: R conleyreg v0.1.9 parity at atol=1e-6 on 6 fixtures (3 cross-sectional + 3 panel) + sparse-forced cross-sectional + sparse-forced panel; internal block-decomposition cross-check at atol=1e-12.
  • Local verification: pytest tests/test_methodology_conley.py + tests/test_conley_vcov.py = 176 pass + 5 slow deselected. grep -rn "Bertanha" diff_diff/ docs/ benchmarks/ = 0 hits. black + ruff clean.

Security / privacy

  • Confirm no secrets/PII in this PR: Yes

Generated with Claude Code

…ns 2014 citation correction

Three bundled changes:

1. ConleySpatialHAC methodology-review-tracker promotion (In Progress → Complete).
   - METHODOLOGY_REVIEW.md L91 + L1262-1290 flipped Complete with full
     Verified Components / Test Coverage / R Comparison Results inline
     table / Corrections Made / Deviations / Outstanding Concerns
     structure, with Last Review = 2026-05-26.
   - New tests/test_methodology_conley.py (~1600 LoC; 10 classes;
     60 tests, 5 of them @pytest.mark.slow). Paper-equation-numbered
     Verified Components walk-through covering:
       * Eq. 4.2 cross-sectional sandwich (pairwise-distance
         specialization; Eq. 3.13 is the lattice-indexed form reserved
         for grid coordinates)
       * Eq. 4.2 HC0 + rank-1 limits + K(0) = 1 diagonal contribution
       * Andrews (1991) HAC lag truncation matching
         conleyreg::time_dist.cpp
       * Haversine convention with Earth radius 6371.01 km (matches
         conleyreg::haversine_dist)
       * Phase 2 panel block-decomposed sandwich at atol=1e-12 (internal
         cross-check vs hand-coded reference + cluster time-invariance
         contract)
       * Wave A #120 sparse k-d-tree numerical correctness (sparse-vs-
         dense bit-identity at atol=1e-10 on cross-sectional, panel,
         and sparse-with-cluster paths)
       * R conleyreg v0.1.9 parity at atol=1e-6 on 6 fixtures
         (3 cross-sectional + 3 panel) + sparse-forced cross-sectional
         + sparse-forced panel + time-asymmetric kernel literal-matching
     Three dedicated deviations-area classes:
       * TestConleyLibraryExtensions (6 tests, no R correspondence):
         combined spatial+cluster product kernel Wave A #119 (two
         limit-fixture anchors), callable conley_metric validation
         Wave A #123, sparse k-d-tree activation Wave A #120,
         indefiniteness guard on Bartlett + uniform kernels.
       * TestConleyDeviationsFromR (3 tests): 1-D radial Bartlett vs
         paper's 2-D separable Eq. 3.14, time-label normalization,
         independent temporal kernel deferred.
       * TestConleyDeferrals (5 tests): fail-closed NotImplementedError
         / TypeError contracts for LinearRegression + survey_design,
         DiD/MPD/TWFE + survey_design (estimator-level via
         conley.py::_validate_conley_estimator_inputs), Conley +
         weights (rejected for any weight_type — pweight, aweight,
         fweight), SyntheticDiD + Conley (TypeError), wild_bootstrap
         + Conley.
   - tests/test_conley_vcov.py extracted 1135+ LoC out (4248 → 3113);
     defensive surface preserved (input validation, NaN/inf guards,
     dispatch-level validity, estimator-level integration smoke tests,
     set_params atomicity, sparse-path activation thresholds + density-
     gate fallback). Module docstring refreshed to describe its current
     defensive-regression role.

2. Stale priority-queue cleanup at METHODOLOGY_REVIEW.md L1386:
   - PreTrendsPower removed (already Complete since 2026-05-19).
   - ConleySpatialHAC removed (this PR).
   - Substantive-review-blocked renumbered #2-#5#1-#4.
   - Consolidation-pass-blocked renumbered #6-#8#5-#6.

3. Bertanha-Imbens 2014 citation correction across 16 sites:
   - linalg.py × 8, conley.py × 1, llms-full.txt × 2, REGISTRY.md × 4,
     spillover.rst × 1.
   - NBER w20773 is on FRD external validity, NOT weighted spatial-HAC.
   - The boundary is now framed as a tri-part contract:
     * Shipped — SpilloverDiD + Conley + survey via Wave E.1/E.2/E.3
       (PR #468/#474/#482, stratified-Conley sandwich on PSU totals
       with within-PSU serial Bartlett HAC for lag_cutoff > 0);
       TwoStageDiD + Conley + survey via Wave E.3 parity (PR #485).
     * Deferred (generic linalg surface, any weight_type) — DiD/MPD/
       TWFE/LinearRegression generic path + Conley + survey_design;
       LinearRegression / compute_robust_vcov Conley + weights rejected
       for pweight, aweight, AND fweight (weighted Conley is not
       implemented on the generic linalg surface).
     * Open methodological question (subset) — the pweight /
       survey_design portion of the deferral additionally lacks a
       canonical methodological extension of Conley (1999) for weighted
       spatial-HAC under probability sampling.
   - REGISTRY sites use canonical `**Note (open methodological
     question):**` label wrapper per CLAUDE.md "Documenting Deviations".
   - Historical CHANGELOG entries (pre-[Unreleased]) intentionally
     retain Bertanha-Imbens 2014 attribution as accurate records of
     past release claims.

Verification:
- pytest tests/test_methodology_conley.py: 60 tests (55 unit + 5 slow)
  pass.
- pytest tests/test_methodology_conley.py + tests/test_conley_vcov.py:
  176 pass + 5 slow deselected (preserves pre-edit baseline coverage
  176 vs original 175 + 1 new uniform-kernel indefiniteness guard).
- grep -rn "Bertanha" diff_diff/ docs/ benchmarks/: 0 hits.
- black + ruff clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown

Overall Assessment

✅ Looks good

Executive Summary

  • No P0/P1 issues found. The PR does not change Conley estimator math, weighting, variance computation, identification assumptions, or defaults; it is primarily a documentation/error-text/test reorganization PR.
  • The methodology promotion is consistent with the Conley paper review and the registry: Eq. 4.2 pairwise form, Eq. 3.14 separable-Bartlett caveat, hardcoded Bartlett temporal kernel, and the generic weighted-Conley deferral all match the existing implementation.
  • P3 — the citation/open-question cleanup is incomplete. The old generic-roadmap framing still remains in diff_diff/twfe.py and TODO.md, which now conflicts with the updated registry/code wording.
  • I could not run pytest here because this environment is missing pytest and numpy, so the test review is static rather than execution-backed.

Methodology

  • P3 — Impact: repository guidance on generic Conley + survey_design/weights is internally inconsistent after this PR. diff_diff/twfe.py (L102-L104) still describes survey_design + Conley as a generic “Phase 5 follow-up,” and TODO.md still cites Bertanha-Imbens 2014, while docs/methodology/REGISTRY.md (L3767-L3780), diff_diff/conley.py, and diff_diff/linalg.py now frame the generic surface as an open methodological question with estimator-specific shipped exceptions. Concrete fix: update the leftover TWFE docstring and TODO row to the same wording used in REGISTRY/conley.py/linalg.py, and keep SpilloverDiD/TwoStageDiD survey-aware Conley explicitly carved out.
  • No other methodology findings. I did not find an undocumented change to estimator math, weighting, variance/SE, defaults, or assumptions in the diff.

Code Quality

  • No findings.

Performance

  • No findings.

Maintainability

  • No findings beyond the stale documentation references noted in Methodology.

Tech Debt

  • No findings. The deferred generic weighted-Conley work is already tracked, but the tracker text itself should be updated as noted above.

Security

  • No findings.

Documentation/Tests

  • No findings on the split itself; moving paper-anchored assertions into tests/test_methodology_conley.py and leaving defensive coverage in tests/test_conley_vcov.py is a sensible reorganization.
  • Verification note: I could not execute the affected tests in this environment because pytest and numpy are unavailable.

@igerber igerber added the ready-for-ci Triggers CI test workflows label May 26, 2026
@igerber igerber merged commit 34c3744 into main May 26, 2026
33 of 34 checks passed
@igerber igerber deleted the feature/conley-methodology-promotion-citation-fix branch May 26, 2026 19:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-ci Triggers CI test workflows

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant