Skip to content

[Bug]: Result of Wild Cluster Bootstrap is inconsistent to Cluster-robust #543

Description

@mohathoillah

Description

I revised the notebook for the Basic DiD module by running the available code of notebook. I found a strange result from the Wild Cluster bootstrap method in this module. There was an inconsistency between the p-value and 95% CI from the Wild Cluster estimation; the CI is above 0, so the p-value should be significant. That's what I thought.

Image Image

Minimal reproducer

# Wild cluster bootstrap inference
did_bootstrap = DifferenceInDifferences(
    cluster="cluster",
    inference="wild_bootstrap",
    n_bootstrap=999,
    bootstrap_weights="rademacher",
    seed=42
)

results_bootstrap = did_bootstrap.fit(
    clustered_df,
    outcome="outcome",
    treatment="treated",
    time="post"
)

print(results_bootstrap.summary())

# Compare inference methods
print("Comparison of inference methods:")
print(f"{'Method':<25} {'SE':>10} {'p-value':>10} {'95% CI':>25}")
print("-" * 70)
print(f"{'Cluster-robust (analytical)':<25} {results_cluster.se:>10.4f} {results_cluster.p_value:>10.4f} [{results_cluster.conf_int[0]:>8.4f}, {results_cluster.conf_int[1]:>8.4f}]")
print(f"{'Wild cluster bootstrap':<25} {results_bootstrap.se:>10.4f} {results_bootstrap.p_value:>10.4f} [{results_bootstrap.conf_int[0]:>8.4f}, {results_bootstrap.conf_int[1]:>8.4f}]")

Expected behavior

The result of the p-value should be consistent with the Cluster-robust.

Actual behavior

The result of p-value from the Wild cluster bootstrap was inconsistent with the Cluster-robust

diff-diff version

  1. Robust Inference

Python version

3.13

Backend

Not sure

Operating system

Windows

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions