Skip to content
Sparse Switchable Normalization with sparse activation function SparestMax
Python Shell
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
configs debug Apr 11, 2019
models update code for sparsestmax Apr 5, 2019
utils
.gitignore first commit Apr 4, 2019
README.md Update README.md Apr 26, 2019
SSN.png
eval_imagenet.py correct readme, update load_ckpt and add dataparallel for eval Apr 8, 2019
test.sh correct readme, update load_ckpt and add dataparallel for eval Apr 8, 2019
train.sh add training script Apr 11, 2019
train_imagenet.py update SSN for dataparallel Aug 12, 2019

README.md

Sparse Switchable Normalization In Image Classification

This is the PyTorch implementation of the paper SSN: Learning Sparse Switchable Normalization via SparsestMax, CVPR 2019.

By Wenqi Shao, Tianjian Meng, Jingyu Li, Ruimao Zhang, Yudian Li, Xiaogang Wang, Ping Luo.

Paper-arxiv

Sparse Switchable Normalization

Sparse Switchable Normalization is able to learn only one normalization operation for each normalization layer in a deep neural network in an end-to-end manner.

Comparisons of top-1 accuracies on the validation set of ImageNet, by using ResNet50 trained with SSN, SN, BN, and GN in different batch size settings. The bracket (·, ·) denotes (#GPUs,#samples per GPU).

Normalizer (8,32) (8,16) (8,8) (8,4) (8,2)
BN 76.4 76.3 75.5 72.7 65.3
GN 75.9 75.8 76.0 75.8 75.9
SN 76.9 76.7 76.7 75.9 75.6
SSN 77.2 77.0 76.8 76.1 75.9

Getting Started

  • Install PyTorch
  • Clone the repo:
    git clone https://github.com/switchablenorms/Sparse_SwitchNorm.git
    

Requirements

  • python packages
    • pytorch>=0.4.0
    • torchvision>=0.2.1
    • tensorboardX
    • pyyaml

Data Preparation

  • Download the ImageNet dataset and put them into the {repo_root}/data/imagenet.

Training a model from scratch

./train.sh

Number of GPUs and configuration file to use can be modified in train.sh

Evaluating performance of a model

Download the pretrained models from Model Zoo and put them into the {repo_root}/model_zoo

./test.sh

Or you can specify the checkpoint path by modifying test.sh

--checkpoint_path model_zoo/ssn_8x2_75.848.pth \

Model Zoo

We provide models pretrained with SSN on ImageNet. The configuration of SSN is denoted as (#GPUs, #images per GPU).

Model Top-1* Top-5* Download MD5
ResNet50v1+SSN (8,32) 77.25% 93.29% [Google Drive] [Baidu Pan (pin:ckd2)] 5c9fb111577b040e62461db51ffce69b
ResNet50v1+SSN (8,16) 76.98% 93.29% [Google Drive] [Baidu Pan (pin:995u)] d7085632529a9b1945a28a4d3bf4cacb
ResNet50v1+SSN (8,8) 76.75% 93.33% [Google Drive] [Baidu Pan (pin:k9ay)] af85a39920644421fc48e216aba6ff0e
ResNet50v1+SSN (8,4) 76.07% 96.71% [Google Drive] [Baidu Pan (pin:jpvu)] 949d4bd54a7c63a3371b941eb4d3ea69
ResNet50v1+SSN (8,2) 75.85% 92.73% [Google Drive] [Baidu Pan (pin:pdyx)] dadb0f1a0c49c31aedd6cb83d4996a03

*single-crop validation accuracy on ImageNet (a 224x224 center crop from resized image with shorter side=256)

In evaluation, download the above models and put them into the {repo_root}/model_zoo.

Citation

If you find this work helpful in your project or use our model zoo, please consider citing:

@article{shao2019ssn,
  title={SSN: Learning Sparse Switchable Normalization via SparsestMax},
  author={Shao, Wenqi and Meng, Tianjian and Li, Jingyu and Zhang, Ruimao and Li, Yudian and Wang, Xiaogang and Luo, Ping},
  journal={arXiv preprint arXiv:1903.03793},
  year={2019}
}

You can’t perform that action at this time.