Project Overview

• All functions and modules are in this repository, and the required packages are in requirements.txt. Use the following command to install them:

  pip3 install -r requirements.txt

• The scripts for reproducing the experiments are in the scripts folder. There are five scripts: sig_berkson.py and sig_classical.py run the synthetic experiments for Berkson and classical ME, respectively.
  • sig_berkson.py and sig_classical.py run the synthetic experiments for Berkson and classical ME, respectively (Fig. 3).
  • CAS.py runs the Californian school data experiment (Fig. 4 and Table 3). The CASschools.csv file is constructed from the California Test Score Data in the AER R package
  • Engel_reliability.py runs the Engel curve analysis for a range of rho values (Fig. 5), and Engel_variance.py investigates the variance of each method through subsampling (Tables 4-5).
  • To run the scripts, use the following command format:

  python3 scripts/sig_berkson.py

  • The results will be saved in the data_from_scripts folder.
• A few notebooks are included:
  • Synthetic_data_demo.ipynb produces demonstrations for synthetic data for Berkson and classical ME under the sigmoid model (Fig. 2).
  • HMC_diagnostics_Berkson.ipynb and HMC_diagnostics_classical.ipynb produce HMC diagnostics for Berkson and classical ME, respectively (Fig. 6, Fig. 7 and Table 6).
  • sensitivity_analysis.ipynb runs the sensitivity analysis (Tables 7-8).

Testing Section

• I added a testing section to check that all NPL methods run correctly in 1 iteration. Add -t or --test after the running command to test. For example:

  python3 scripts/sig_berkson.py -t

A list of all input arguments for the scripts

These arguments are all optional.

• Enable testing mode: -t or --test. It is a store_true argument.
• Experiment name: -e or --experiment_name. Default = ''. E.g. -e reproduce_experiment
• Below are arguments Only for sig_berkson.py and sig_classical.py:
  • Main comparison goal: -m or --main_comparison_goal. Default = 'method'. E.g. -m prior_coef
  • Splitting parameters: -s or --splitting_params. Default = ['None', 'None', 'None']. E.g. -s ME cr cl
  • Include NLS in the plot: -n or --include_nls. It is a store_true argument.

Below is a list of commands to run all scripts. You need to define experiment_name (string) and testing_flag (-t or --test if testing, nothing otherwise).

python3 scripts/sig_berkson.py -e "$experiment_name" -m method -n -s None None None $testing_flag
python3 scripts/sig_classical.py -e "$experiment_name" -m method -n -s None None None $testing_flag
python3 scripts/CAS.py -e "$experiment_name" $testing_flag
python3 scripts/Engel_reliability.py -e "$experiment_name" $testing_flag
python3 scripts/Engel_variance.py -e "$experiment_name" $testing_flag