Skip to content
Switch branches/tags

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time

Sparse Switchable Normalization In Image Classification

This is the PyTorch implementation of the paper SSN: Learning Sparse Switchable Normalization via SparsestMax, CVPR 2019.

By Wenqi Shao, Tianjian Meng, Jingyu Li, Ruimao Zhang, Yudian Li, Xiaogang Wang, Ping Luo.


Sparse Switchable Normalization

Sparse Switchable Normalization is able to learn only one normalization operation for each normalization layer in a deep neural network in an end-to-end manner.

Comparisons of top-1 accuracies on the validation set of ImageNet, by using ResNet50 trained with SSN, SN, BN, and GN in different batch size settings. The bracket (·, ·) denotes (#GPUs,#samples per GPU).

Normalizer (8,32) (8,16) (8,8) (8,4) (8,2)
BN 76.4 76.3 75.5 72.7 65.3
GN 75.9 75.8 76.0 75.8 75.9
SN 76.9 76.7 76.7 75.9 75.6
SSN 77.2 77.0 76.8 76.1 75.9

Getting Started

  • Install PyTorch
  • Clone the repo:
    git clone


  • python packages
    • pytorch>=0.4.0
    • torchvision>=0.2.1
    • tensorboardX
    • pyyaml

Data Preparation

  • Download the ImageNet dataset and put them into the {repo_root}/data/imagenet.

Training a model from scratch


Number of GPUs and configuration file to use can be modified in

Evaluating performance of a model

Download the pretrained models from Model Zoo and put them into the {repo_root}/model_zoo


Or you can specify the checkpoint path by modifying

--checkpoint_path model_zoo/ssn_8x2_75.848.pth \

Model Zoo

We provide models pretrained with SSN on ImageNet. The configuration of SSN is denoted as (#GPUs, #images per GPU).

Model Top-1* Top-5* Download MD5
ResNet50v1+SSN (8,32) 77.25% 93.29% [Google Drive] [Baidu Pan (pin:ckd2)] 5c9fb111577b040e62461db51ffce69b
ResNet50v1+SSN (8,16) 76.98% 93.29% [Google Drive] [Baidu Pan (pin:995u)] d7085632529a9b1945a28a4d3bf4cacb
ResNet50v1+SSN (8,8) 76.75% 93.33% [Google Drive] [Baidu Pan (pin:k9ay)] af85a39920644421fc48e216aba6ff0e
ResNet50v1+SSN (8,4) 76.07% 96.71% [Google Drive] [Baidu Pan (pin:jpvu)] 949d4bd54a7c63a3371b941eb4d3ea69
ResNet50v1+SSN (8,2) 75.85% 92.73% [Google Drive] [Baidu Pan (pin:pdyx)] dadb0f1a0c49c31aedd6cb83d4996a03

*single-crop validation accuracy on ImageNet (a 224x224 center crop from resized image with shorter side=256)

In evaluation, download the above models and put them into the {repo_root}/model_zoo.


If you find this work helpful in your project or use our model zoo, please consider citing:

  title={SSN: Learning Sparse Switchable Normalization via SparsestMax},
  author={Shao, Wenqi and Meng, Tianjian and Li, Jingyu and Zhang, Ruimao and Li, Yudian and Wang, Xiaogang and Luo, Ping},
  journal={arXiv preprint arXiv:1903.03793},


Sparse Switchable Normalization with sparse activation function SparestMax




No releases published


No packages published