Skip to content

welleast/InterleavedGroupConvolutions

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

27 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Interleaved Group Convolutions

This project contains the code implementation used for the experiments in the paper:

Interleaved Group Convolutions. Ting Zhang, Guo-Jun Qi, Bin Xiao, and Jingdong Wang. In International Conference on Computer Vision (ICCV), 2017. arXiv preprint arXIV:1707.02725 (2017)

Introduction

In this work, we present a simple and modularized neural network architecture, named interleaved group convolutional neural networks (IGCNets). The main point lies in a novel building block, a pair of two successive interleaved group convolutions: primary group convolution and secondary group convolution. The two group convolutions are complementary.

IGC

Illustrating the interleaved group convolution, with L = 2 primary partitions and M = 3 secondary partitions. The convolution for each primary partition in primary group convolution is spatial. The convolution for each secondary partition in secondary group convolution is point-wise (1 × 1).

Our motivation comes from the four branch presentation of regular convolution illustrated in the following picture.

RC

(a) Regular convolution. (b) Four-branch representation of the regular convolution. The shaded part in (b), we call crosssummation, is equivalent to a three-step transformation: permutation, secondary group convolution, and permutation back.

Results

#Params (M) FLOPs (10^9) training error test error
top-1 top-5 top-1 top-5
ResNet(C=64) 11.151 1.8 22.41 6.53 31.06 11.38
ResNet(C=69) 11.333 2.1 21.43 5.96 30.58 10.77
IGC-L4M32+Ident. 11.205 1.9 21.71 6.21 30.77 10.99
IGC-L16M16+Ident. 11.329 2.2 19.97 5.44 29.40 10.32
IGC-L100M2+Ident. 8.61 1.3 13.93 2.75 26.95 8.92

Imagenet classification results of a ResNet of depth 18 and our approach. Both ResNets and our networks contain four stages, and when down-sampling is performed, the channel number is doubled. For ResNets, C is the channel number at the first stage. For our networks except IGC-L100M2+Ident., we double the channel number by doubling M and keeping L unchanged. For IGCL100M2+Ident., we double the channel number by doubling L and keeping M unchanged.

More results can be found in the paper.

Requirements

  • Install MXNet on a machine (Windows, Linux, and Mac OS) with CUDA GPU and optional cuDNN.

  • Add the code in src/ to MXNet src/operator/

  • Build MXNet

  • If you fail to apply the above steps, you can simply use my MXNet repository

How to Train

Current code supports training IGCNets on Cifar-10, Cifar-100 and SVHN, such as plain, resnet, plain_igc,resnet_igc. All the networks are contained in the network folder.

For example, running the following command can train the plain_igc network on Cifar-10.

python train_model.py --network=plain_igc --depth=38 --gpus=0,1 --primary-partition=24 --secondary-partition=2 --batch-size=64 --data-dir=<dataset location> --dataset=cifar10

For example, running the following command can train the resnet_igc_imgnet_d18 network on ImageNet.

python train_imagenet.py --network=resnet_igc_imgnet_d18 --depth=18 --gpus=0,1,2,3,4,5,6,7 --primary-partition=100 --batch-size=256 --data-dir=<dataset location>

Citation

Please cite our papers in your publications if it helps your research:

@article{WangWZZ16,
  author    = {Jingdong Wang and
               Zhen Wei and
               Ting Zhang and
               Wenjun Zeng},
  title     = {Deeply-Fused Nets},
  journal   = {CoRR},
  volume    = {abs/1605.07716},
  year      = {2016},
  url       = {http://arxiv.org/abs/1605.07716}
}
@article{ZhaoWLTZ16,
  author    = {Liming Zhao and
               Jingdong Wang and
               Xi Li and
               Zhuowen Tu and
               Wenjun Zeng},
  title     = {On the Connection of Deep Fusion to Ensembling},
  journal   = {CoRR},
  volume    = {abs/1611.07718},
  year      = {2016},
  url       = {http://arxiv.org/abs/1611.07718}
}
@article{DBLP:journals/corr/ZhangQ0W17,
  author    = {Ting Zhang and
               Guo{-}Jun Qi and
               Bin Xiao and
               Jingdong Wang},
  title     = {Interleaved Group Convolutions for Deep Neural Networks},
  journal   = {CoRR},
  volume    = {abs/1707.02725},
  year      = {2017},
  url       = {http://arxiv.org/abs/1707.02725}
}

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 82.6%
  • C++ 16.9%
  • Cuda 0.5%