Skip to content
LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks
Branch: master
Clone or download
Latest commit ca764ae Aug 9, 2018
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
LICENSE Initial commit Jul 19, 2018
README.md
alexnet_model.py
cifar10-vgg-small.py
densenet_model.py
googlenet_model.py
imagenet.py Initial commit Jul 25, 2018
imagenet_utils.py Initial commit Jul 25, 2018
learned_quantization.py
resnet_model.py Initial commit Jul 25, 2018
vgg_model.py

README.md

LQ-Nets

By Dongqing Zhang, Jiaolong Yang, Dongqiangzi Ye, Gang Hua.

Microsoft Research Asia (MSRA).

Introduction

This repository contains the training code of LQ-Nets introduced in our ECCV 2018 paper:

D. Zhang*, J. Yang*, D. Ye* and G. Hua. LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks. ECCV 2018 (*: Equal contribution) PDF

Dependencies

  • Python 2.7 or 3.3+
  • Python bindings for OpenCV
  • TensorFlow >= 1.3.0
  • TensorPack

Usage

Download the ImageNet dataset and decompress into the structure like

dir/
  train/
    n01440764/
      n01440764_10026.JPEG
      ...
    ...
  val/
    ILSVRC2012_val_00000001.JPEG
    ...

To train a quantized "pre-activation" ResNet-18, simply run

python imagenet.py --gpu 0,1,2,3 --data /PATH/TO/IMAGENET --mode preact --depth 18 --qw 1 --qa 2 --logdir_id w1a2 

After the training, the result model will be stored in ./train_log/w1a2.

For more options, please refer to python imagenet.py -h.

Results

ImageNet Experiments

Quantizing both weight and activation

Model Bit-width(W/A) Top-1(%) Top-5(%)
ResNet-18 1/2 62.6 84.3
ResNet-18 2/2 64.9 85.9
ResNet-18 3/3 68.2 87.9
ResNet-18 4/4 69.3 88.8
ResNet-34 1/2 66.6 86.9
ResNet-34 2/2 69.8 89.1
ResNet-34 3/3 71.9 90.2
ResNet-50 1/2 68.7 88.4
ResNet-50 2/2 71.5 90.3
ResNet-50 3/3 74.2 91.6
ResNet-50 4/4 75.1 92.4
AlexNet 1/2 55.7 78.8
AlexNet 2/2 57.4 80.1
DenseNet-121 2/2 69.6 89.1
VGG-Variant 1/2 67.1 87.6
VGG-Variant 2/2 68.8 88.6
GoogLeNet-Variant 1/2 65.6 86.4
GoogLeNet-Variant 2/2 68.2 88.1

Quantizing weight only

Model Bit-width(W/A) Top-1(%) Top-5(%)
ResNet-18 2/32 68.0 88.0
ResNet-18 3/32 69.3 88.8
ResNet-18 4/32 70.0 89.1
ResNet-50 2/32 75.1 92.3
ResNet-50 4/32 76.4 93.1
AlexNet 2/32 60.5 82.7

More results can be found in the paper.

Citation

If you use our code or models in your research, please cite our paper with

@inproceedings{ZhangYangYeECCV2018,
    author = {Zhang, Dongqing and Yang, Jiaolong and Ye, Dongqiangzi and Hua, Gang},
    title = {LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks},
    booktitle = {European Conference on Computer Vision (ECCV)},
    year = {2018}
}
You can’t perform that action at this time.