Skip to content
master
Go to file
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.

README.md

Decorrelated Batch Normalization

Code for reproducing the results in the following paper:

Decorrelated Batch Normalization
Lei Huang, Dawei Yang, Bo Lang, Jia Deng
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018. arXiv:1804.08450

Requirements and Dependency

  • Install MAGMA (you can find the instructions in  'Install MAGMA.md' ). Note: MAGMA is required for SVD on GPU. Without MAGMA, you can run the code on CPU only, while all the CNN experiments in the paper are run on GPU.
  • Install Torch with CUDA (for GPU). Note that cutorch should be compiled with MAGMA support if you have installed MAGMA and set the environments correctly.
  • Install cudnn v5.
  • Install the dependency optnet by:
luarocks install optnet

Experiments

1. Reproduce the results for PCA whitening:

  • Run:
bash execute_MLP_0debug_MNIST.sh

This script will download MNIST automatically and you should put the mnist.t7/ under ./dataset/. The experiment results will be saved at ./set_result/MLP/.

2. Reproduce the results for MLP architecture:

(1) FIM experiments on YaleB dataset
  • Prepare the data: download the YaleB dataset here, and put the data files under /dataset/ so that the paths look like ./dataset/YaleB/YaleB_train.dat and ./dataset/YaleB/YaleB_test.dat.
  • Run:
bash execute_MLP_1FIM_YaleB_best.sh

The experiment results will be saved at directory: 'set_result/MLP/'.

You can experiment with different hyperparameters by running these scripts -- execute_MLP_1FIM_YaleB_HyperP.sh and execute_MLP_1FIM_YaleB_HyperP_nnn.sh.

(2) Experiments on PIE dataset
  • Prepare the data: download the PIE dataset here, and put the data file under ./dataset/ such that the paths look like ./dataset/PIE/PIE_train.dat and ./dataset/PIE/PIE_test.dat.
  • To experiment with different group sizes, run:
bash execute_MLP_2PIE_DBNGroup.sh
  • To obtain different baseline performances, execute:
 bash execute_MLP_2PIE.sh
 bash execute_MLP_2PIE_nnn.sh

Note that the experiments until this point can be run on CPU, so MAGMA is not needed in above experiments.


3. Reproduce the results for VGG-A architecture on CIFAR-10:

  • Prepare the data: follow the instructions for CIFAR-10 in this project . It will generate a preprocessed dataset and save a 1400MB file. Put this file cifar_provider.t7 under ./dataset/.
  • Run:
bash execute_Conv_1vggA_2test_adam.sh
bash execute_Conv_1vggA_2test_base.sh
bash execute_Conv_1vggA_2test_ELU.sh
bash execute_Conv_1vggA_2test_var.sh

Note that if your machine has fewer than 4 GPUs, the environment variable CUDA_VISIBLE_DEVICES should be changed accordingly.

4. Analyze the properties of DBN on CIFAR-10 datset:

  • Prepare the data: same as in VGG-A experiments.
  • Run:
bash exp_Conv_4Splain_1deep.lua
bash exp_Conv_4Splain_2large.lua

5. Reproduce the ResNet experiments on CIFAR-10 datset:

  • Prepare the data: download CIFAR-10 and CIFAR-100, and put the data files under ./dataset/.
  • Run:
bash execute_Conv_2residual_old.sh
bash execute_Conv_3residual_wide_Cifar100_wr_BN_d28_h48_g16_b128_dr0.3_s1_C2.sh
bash execute_Conv_3residual_wide_Cifar100_wr_DBN_scale_L1_d28_h48_g16_b128_dr0.3_s1_C3.sh
bash execute_Conv_3residual_wide_Cifar10_wr_BN_d28_h48_g16_b128_dr0.3_s1_C2.sh
bash execute_Conv_3residual_wide_Cifar10_wr_DBN_scale_L1_d28_h48_g16_b128_dr0.3_s1_C3.sh

6. Reproduce the ImageNet experiments.

  • Clone Facebook's ResNet repo here.
  • Download ImageNet and put it in: /tmp/dataset/ImageNet/ (you can also customize the path in opts.lua)
  • Install the DBN module to Torch as a Lua package: go to the directory ./models/imagenet/cuSpatialDBN/ and run luarocks make cudbn-1.0-0.rockspec.
  • Copy the model definitions in ./models/imagenet/ (resnet_BN.lua, resnet_DBN_scale_L1.lua and init.lua) to ./models directory in the cloned repo fb.resnet.torch, for reproducing the results reported in the paper. You also can compare the pre-activation version of residual networks introduced in the paper (using the model files preresnet_BN.lua and preresnet_DBN_scale_L1.lua).
  • Use the default configuration and our models to run experiments.

Contact

Email: huanglei@nlsde.buaa.edu.cn. Any discussions and suggestions are welcome!

About

Code for Decorrelated Batch Normalization

Resources

License

Releases

No releases published

Packages

No packages published
You can’t perform that action at this time.