Implementation for Trained Ternary Network.
Switch branches/tags
Nothing to show
Clone or download
Latest commit 1a67251 Jan 13, 2017
Permalink
Failed to load latest commit information.
docs Init first commit. Jan 13, 2017
examples Add train logs. Jan 13, 2017
tensorpack Init first commit. Jan 13, 2017
.gitignore Init first commit. Jan 13, 2017
LICENSE Init first commit. Jan 13, 2017
README.md Fix typo. Jan 13, 2017
opt-requirements.txt Init first commit. Jan 13, 2017
readthedocs.yml Init first commit. Jan 13, 2017
requirements.txt Init first commit. Jan 13, 2017

README.md

Trained Ternary Quantization (TTQ)

TensorFlow implementation of paper:

Trained Ternary Quantization, by Zhu et al.

This implementation is based on tensorpack. Thanks to this framework which made this implementation extremely easy.

Experimental Results:

Error Rate of Finetuned TTQ ResNet models on CIFAR-10:

Network Full Precision TTQ
ResNet-20 8.23 8.87
ResNet-32 7.67 7.63
ResNet-44 7.18 7.02
ResNet-56 6.80 6.44

Error Rate of TTQ AlexNet model on ImageNet from scratch:

Network Full Precision TTQ
Top1-error 42.8 42.5
Top5-error 19.7 20.3

Dependencies:

  • Python 2 or 3
  • TensorFlow >= 0.8
  • Python bindings for OpenCV
  • other requirements:
pip install --user -r requirements.txt
pip install --user -r opt-requirements.txt (some optional dependencies, you can install later if needed)
  • Use tcmalloc whenever possible
  • Enable import tensorpack:
export PYTHONPATH=$PYTHONPATH:`readlink -f path/to/tensorpack`

Usage

  • To train ResNet on CIFAR10 with fixed threshold:
cd examples/Ternary-Net/
python ./tw-cifar10-resnet.py --gpu 0,1 [--load MODEL_PATH] [--t threshold] [--n NSIZE]

Note: We used 2 GPUs for training and pretrained model can be obtained using /examples/ResNet/

  • To train ResNet on CIFAR10 with fixed sparsity:
cd examples/Ternary-Net/
python ./p-cifar10-resnet.py --gpu 0,1 [--load MODEL_PATH] [—p sparsity] [--n NSIZE]
  • To train AlexNet on ImageNet with fiexed threshold:
cd examples/Ternary-Net/
python ./tw-imagenet-alexnet.py --gpu 0,1,2,3 --data IMAGENET_PATH [--t threshold]

Note: We used 4 GPUs for training

Logs

Some training logs can be found here.

Support

Please use github issues for any issues related to the code. Send email to the authors for general questions related to the paper.

Citation

If you use our code or models in your research, please cite:

@article{zhu2016trained,
  title={Trained Ternary Quantization},
  author={Zhu, Chenzhuo and Han, Song and Mao, Huizi and Dally, William J},
  journal={arXiv preprint arXiv:1612.01064},
  year={2016}
}