Skip to content
Variational Resampling Based Assessment of Deep Neural Networks Robustness under Distribution Shift
Python TeX R Other
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
Bayesian_CNN @ 8c05f58
Pytorch-Utils @ 962b9c2
paper/icml_udl_2019_style
paper_2019_distribution_shift_resample @ 8f09ca0
plots
refactor_Bayesian_CNN
results/VAE_fashion-mnist_64_62
tensorflow-generative-model-collections @ 3abde8a
.gitmodules
ACGAN.py
Makefile
README.md
VAE.py
VGMM.py
config.py
config_manager.py
convnet.py
copy_result.sh
data_generator.py
demo_cross_validation.py
docker-togo.sh
getdir.py
main.py
ops.py
prior_factory.py
requirements.txt
requirements_cpu.txt
run_all.sh
run_all_test.sh
scheduler.sh
statistic.py
system_requirement.sh
test_gpu.py
utils_parent.py
visualization.py
wd_rand.txt
wd_rand_matrix.txt
wd_vgmm.txt
wd_vgmm_matrix.txt

README.md

Variational Gaussian Mixture model Cross Validation resampling of Bayesian and Frequest Neural Networks

Dependencies

  • pip install Cython
  • pip install pot
  • if you have conda, install pot with: conda install -c conda-forge pot
  • pip install -r requirements.txt

Tips on generating dependencies file

pip freeze https://medium.com/python-pandemonium/better-python-dependency-and-package-management-b5d8ea29dff1

How to run or reproduce the experiment

Preparation

  • clone a new repository and go to the root directory
  • make build (30mins for the first run on fujitsu-celcius) # equivalent to python main.py --cluster True (train vae on all data and cluster), results could be stored in results/VAE_fashion-mnist_64_62/L-1 for example
  • make label (1 hours on fujitsu-celcius) # equivalent to python main.py --labeled True --cluster True (train vae according to label and cluster each label, then merge), results could be stored in results/VAE_fashion-mnist_64_62/L0 unutil results/VAE_fashion-mnist_64_62/L9 for example
  • results for the two steps are stored in results/VAE_fashion-mnist_64_62 for example, where 62 is the latent space dimension of VAE (see configuration file named config.py), while data is stored in /data/FashionMNIST for example

Evaluate Neural Network

  • change directory to refactor_Bayesian_CNN
  • make rand frand|vgmm|fvgmm_alexnet

statistic

  • before you run this command, you should previously run make build and make label
  • change directory to root folder
  • make wasser_cv_emd : compute wasserstein distance for random cross validation
  • make wasser_vgmm_emd: compute wasserstein distance for vgmm-vae cross validation
  • make t-SNE: generate t-SNE plot for all data divided by vgmm-vae (results could be stored in /results/VAE_fashion-mnist_64_62 for example)
  • make distribution_y: plot the histogram of class distribution for each cluster, result is store in distribution_y.txt

Plotting

  • go to /plots and use the R code to generate the beautiful ggplot

Guide to the code

Configuration

in root folder and refactor_Bayesian_CNN, files start with config stores global configuration parameters.

arguments for main.py in project root

python main.py --cluster <True,False (default)> --dataset <'mnist', 'fashion-mnist' (default)> --z_dim <1-inf,62(default)> --labeled <True,False (default)>

Misc Resources

Semi-supervised vae

python make

parallel job in python

to avoid no space on device problem when run parallel in pytorch

%env JOBLIB_TEMP_FOLDER=/tmp

You can’t perform that action at this time.