Skip to content
This project is the Torch implementation of our ICCV 2017 paper: Centered Weight Normalization in Accelerating Training of Deep Neural Networks
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
dataset
models
module
1_execute_MLP_svhn.sh
1_execute_MLP_svhn_adam.sh
2_execute_Conv_CIFAR10_vggA.sh
3_execute_Conv_CIFAR100_GoogLeNet.sh
4_execute_Conv_CIFAR10_resnet.sh
LICENSE
README.md
augmentation.lua
exp_GoogleNet_dataWhitening.lua
exp_MLP.lua
exp_res_dataNorm.lua
exp_vggA.lua
provider.lua

README.md

Centered Weight Normalization

This project is the Torch implementation of the paper: Centered Weight Normalization in Accelerating Training of Deep Neural Networks ( ICCV 2017).

  • bibtex:
@INPROCEEDINGS{Huang2017ICCV,
    author = {Lei Huang and Xianglong Liu and Yang Liu and  Bo Lang and Dacheng Tao},
    title = {Centered Weight Normalization  in Accelerating Training of Deep Neural Networks},
    booktitle = {ICCV},
    year = {2017}}

Requirements and Dependency

  • Install Torch with CUDA GPU
  • Install cudnn v5
  • Install dependent lua packages optnet by run: luarocks install optnet

Experiments in the paper

1. MLP architecture over SVHN dataset

  • Dataset prepraration: We can get the preprocessed SVHN dataset for MLP architecture by running:
  cd dataset
   th preProcess_div256.lua

Note that this script is based on the Torch script for SVHN

  • Execute:
 th exp_MLP.lua 
  • To reproduce the experimental results, you can run the script below, which include all the information of experimental configurations:
 bash 1_execute_MLP_svhn.sh  
 bash 1_execute_MLP_svhn_adam.sh  

2. VGG-A architecture over Cifar-10 dataset

  • Dataset preparations: the dataset is based on the preprocessed script on: https://github.com/szagoruyko/cifar.torch, and you should put the data file in the directory: './dataset/cifar_provider.t7'

  • Execute:

 th exp_vggA.lua –dataPath './dataset/cifar_provider.t7'
  • To reproduce the experimental results, you can run the script below, which include all the information of experimental configurations:
 bash   2_execute_Conv_CIFAR10_vggA.sh

3. GoogLeNet architecture over Cifar datasets

th exp_GoogleNet_dataWhitening.lua –dataPath './dataset/cifar100_whitened.t7'
  • To reproduce the experimental results, you can run the script below, which include all the information of experimental configurations:
 3_execute_Conv_CIFAR100_GoogLeNet.sh 

The GoogLeNet model is based on the project on: https://github.com/soumith/imagenet-multiGPU.torch

4. Residual network architecture over Cifar datasets

  • Dataset preparations: The dataset is based on original CIFAR datasets, and the data file should be put in the directory: ./dataset/cifar_original.t7.
  • Execute:
th exp_res_dataNorm.lua –dataPath './dataset/cifar10_original.t7'
  • To reproduce the experimental results, you can run the script below, which include all the information of experimental configurations:
4_execute_Conv_CIFAR10_resnet.sh

The normlization of Cifar dataset is in the script th exp_res_dataNorm.lua. The residual network model and respective script are based on facebook ResNet.

5. GoogLeNet over ImageNet

This experiment is based on the project at: https://github.com/soumith/imagenet-multiGPU.torch.
The proposed model are in: './models/imagenet/'

Contact

huanglei@nlsde.buaa.edu.cn, Any discussions and suggestions are welcome!

You can’t perform that action at this time.