Overview

This document describes the replication materials for Leavitt, T. and V. Rivera-Burgos. Navigating the Mismeasurement of Intermediary Variables in Message-Based Experiments. Political Science Research and Methods.

---

Computational Requirements

All replication code was run on a 2023 MacBook Pro (Apple M3 Max, 36 GB RAM)
using macOS Sonoma 14.7.1.

• R version: 4.4.2
• Required R packages:
  • dplyr (v. 1.1.4)
  • xtable (v. 1.8-4)
  • randomizr (v. 1.0.0)
  • remotes (used to install the specific package versions above)

The script install_packages.R uses the remotes package to install the exact versions of dplyr, xtable, and randomizr needed to reproduce the results.
Approximate runtime: 11.93 seconds.

Data

Files:

• data/data_cleaning.R
• data/raw/hughes_et_al_2020/replication_data.csv
• data/raw/moy_2021/Moy_JEPS.RData

Description:
The script data_cleaning.R loads the raw Hughes et al. (2020) and Moy (2021) datasets, processes them according to the procedures described in the manuscript, and outputs two cleaned datasets:

• data/hughes_et_al_data.rda
• data/moy_data.rda

Approximate runtime for data_cleaning.R: 0.07 seconds.

---

Code and Output

To reproduce all results in the manuscript, run:

source("master.R")

from the project root.

The total approximate runtime of the full replication (including package installation, data cleaning, all analyses, and table generation) is 20.58 seconds.

All file paths used in the replication scripts are relative to the project root, so no manual changes to the working directory are required.

Custom Functions (code_and_output/custom_functions/)

The following scripts contain functions used across multiple replications:

• fine_strat_var_est.R
  Implements Fogarty’s (2018, 2023) variance estimator for finely stratified designs.

• post_strat_consv_var_est.R
  Implements the Miratrix, Sekhon & Yu (2013) conservative post-stratified variance estimator using simulated random assignments.

---

Replication Scripts (code_and_output/)

These scripts generate all manuscript results:

• hughes_et_al_2020_bounds.R
  Reproduces all results in Section 3.3 (Hughes et al. 2020 application).
  Approximate runtime: 0.07 seconds.

• moy_2021_bounds.R
  Reproduces all results in Section 4.3 (Moy 2021 application).
  Approximate runtime: 6.88 seconds.

• table_2.R
  Generates the LaTeX file corresponding to Table 2 in the manuscript.
  Approximate runtime: 0.01 seconds.

---

Output folder

• code_and_output/tables/
  Contains LaTeX output generated by the replication scripts, including:
  • tab_2.tex — block-level table from the Hughes et al. (2020) application.

References

Fogarty, C. B. (2018). On Mitigating the Analytical Limitations of Finely Stratified Experiments. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 80(5), 1035–1056.

Fogarty, C. B. (2023). Testing Weak Nulls in Matched Observational Studies. Biometrics, 79(3), 2196–2207.

Hughes, D. A., Gell-Redman, M., Crabtree, C., Krishnaswami, N., Rodenberger, D., & Monge, G. (2020). Persistent Bias Among Local Election Officials. Journal of Experimental Political Science, 7(3), 179–187.

Miratrix, L. W., Sekhon, J. S., & Yu, B. (2013). Adjusting Treatment Effect Estimates by Post-Stratification in Randomized Experiments. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 75(2), 369–396.

Moy, B. J. (2021). Can Social Pressure Foster Responsiveness? An Open Records Field Experiment with Mayoral Offices. Journal of Experimental Political Science, 8(2), 117–127.