This repository contains the simulation code used to generate the numerical results and figures for the Bernoulli article and supplement.
The code was prepared for Python 3.10. From the repository root, install the Python dependencies with either pip or conda.
Using pip:
python -m pip install -r requirements.txtUsing conda:
conda env create -f environment.yml
conda activate rande
python -m pip install -r requirements.txtTo run all experiments and regenerate the figure files, run:
./run_exp.shBy default this uses one worker process and writes intermediate results to results/ and generated figures to figures/. To use more processes or a different output root, set environment variables before running the script:
PROCESSES=8 OUTPUT_ROOT=reproduction ./run_exp.shThis writes intermediate results under reproduction/results/ and figures under reproduction/figures/.
The full reproduction run can be computationally expensive. The script reuses existing intermediate result files when present, so interrupted runs can usually be restarted with the same output directory.
The reproduction script runs the following registered experiments:
gaussian_rho: e-BH and randomized variants of e-BH.gaussian_p_rho: BY and randomized BY comparisons.gaussian_rho_stability: stability of randomized procedures.gaussian_rho_umi_prop: UMI randomization proportion study.gaussian_derandomization: derandomization comparisons.gaussian_dynamic: dynamic randomization comparisons.guo_rao: Guo-Rao sharpness example for BY.
Each experiment can also be run directly:
python src/main.py --processes 1 --exp gaussian_rho --out_dir results/gaussian_rho --result_dir figures/gaussian_rhoRun the lightweight test suite with:
python -m unittest discover -s testsIf pytest is installed, the same tests can also be run with:
python -m pytest testssrc/: experiment, data-generation, p-value, metric, and algorithm code.src/configs/: registered experiment configurations.tests/: lightweight regression tests.run_exp.sh: convenience script for reproducing all figures.requirements.txtandenvironment.yml: dependency specifications.
Generated results/, generated figures/, Python bytecode caches, and operating-system metadata files are intentionally excluded from the submission archive.