This repository contains two folders. The implementation
folder contains implementation details and examples. The file implementation.R
contains four main functions and four helper functions, as follows.
-
test.of.tests
: Run the test of tests for a given dataset, given public hypothesis test, and fixed input parameters$m, \alpha_0$ . -
power.test.of.tests
: Compute the theoretical power of the test of tests for a given public hypothesis test and fixed input parameters$m,\alpha_0$ . -
optimal.m.alpha0
: Find the optimal input parameters for a given public hypothesis test and known effect size. -
practical.m.alpha0
: Find practical values of the input parameters for unknown effect size by searching for the minimum effect size required to achieve$\rho$ power for a given public hypothesis test. - Helper functions
rtulap
,ptulap
,dp.binom.p.val
, anddp.binom.alt.prob
: implementation of the binomial test described in Awan and Slavkovic's Differentially Private Uniformly Most Powerful Tests for Binomial Data.
The file examples.Rmd
and corresponding compiled examples.pdf
walk through an example of the use of these functions when the public test is a one-sample t-test. The file power_curves.Rmd
and corresponding compiled power_curves.pdf
provide complete code to produce the test of tests power curve for the figures in the main text of the manuscript.
The figures
folder contains code to produce the figures in the manuscript, CSV files containing the raw data, and PNG files for each plot. The file figures_arXiv.Rmd
and corresponding compiled figures_arXiv.pdf
contain the code to create all figures in the arXiv version of the paper. The file figures_ICML.Rmd
and corresponding compiled figures_ICML.pdf
contain the code to create all figures in the ICML version of the paper. The runs used to create these plots are stored in the figures
folder as CSVs and the plots are stored in arXiv_plots
and ICML_plots
.