This repository provides open access to data and materials from the paper:
- Wolf, B., & Klager, C. (2026). Subgroup effects, compositional effects, and treatment effect heterogeneity. Journal of Research on Educational Effectiveness (JREE).
Education researchers are increasingly interested in understanding ‘what works’ for whom and under what conditions, often through analyses of subgroup and compositional findings. Yet such findings are frequently reported without strong empirical or theoretical justification, raising concerns about their interpretability. Drawing on the What Works Clearinghouse (WWC) study database, this study uses meta-analysis to examine systematic patterns in subgroup and compositional effects across educational evaluations and investigates empirical explanations for those patterns. Within studies, we find that treatment effects are modestly more favorable for economically disadvantaged students and female students and somewhat less favorable for high-performing students. Across studies, evaluations with higher proportions of economically disadvantaged students or students with disabilities have larger average treatment effects, but these compositional effects do not persist when accounting for variation in intervention characteristics. Despite these systematic patterns, we observe considerable treatment effect heterogeneity and wide prediction intervals, highlighting the need to interpret average treatment effects and subgroup findings with caution. We discuss implications for researchers, funders of research, and policymakers working to improve the design, evaluation, and equitable impact of educational interventions.
The data can be found below:
And the codebook can be found below:
- [Codebook]
But most of the variables come directly from the What Works Clearinghouse.
This work was partly commissioned by the WWC to both inform and promote discussion about educational research. This work was created as part of the Contributors' official duties as employees of the United States Government and is therefore a work of the U.S. Government. The content of the publication does not necessarily reflect the views or policies of the U.S. Government nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government. In accordance with 17 U.S.C. 105, the report and the accompanying data are in the public domain. While permission to use these data is not necessary, the data should be cited as:
- Wolf, B., & Klager, C. (2026). Subgroup effects, compositional effects, and treatment effect heterogeneity. Journal of Research on Educational Effectiveness (JREE).
The online technical appendix can be found [here].
Betsy and Chris were Education Research Analysts at the What Works Clearinghouse before nearly all staff at the Institute of Education Sciences were subject to a reduction in force as part of DOGE efforts to eliminate research and accountability across multiple sectors. The DOGE cuts have implications for both the public sector and the private sector, as many education researchers in the private sector were also affected. The future of the What Works Clearinghouse remains uncertain.
- Pustejovsky, J. E. (2025). clubSandwich: Cluster-Robust (Sandwich) Variance Estimators with Small-Sample Corrections. R package version 0.6.0. https://CRAN.R-project.org/package=clubSandwich
- Pustejovsky, J., Joshi, M., & Citkowicz, M. (2025). metaselection: Meta-analytic selection models with cluster-robust and cluster-bootstrap standard errors for dependent effect size estimates. R package version 0.1.5, commit 223e8c29e62547d803a14f5b12746b9af9453eb2. https://github.com/jepusto/metaselection
- Viechtbauer, W. (2010). metafor: Conducting meta-analyses in R with the metafor package. Journal of Statistical Software, 36(3), 1-48. https://doi.org/10.18637/jss.v036.i03
- Wickham H (2016). ggplot2: Elegant Graphics for Data Analysis. Springer-Verlag New York. ISBN 978-3-319-24277-4, https://ggplot2.tidyverse.org.