Efficient-Deep-Learning

Related Paper of Efficient Deep Neural Networks

Reviews, Tutorials, Blog

1. Efficient Processing of Deep Neural Networks: A Tutorial and Survey
2. High-Performance Hardware for Machine Learning
3. series of deep learning on iphone blogs

Sparse, Quantization and Compression

Sparse

1. Learning both Weights and Connections for Efficient Neural Networks [NIPS 2015]
2. Dynamic Network Surgery for Efficient DNNs [NIPS2016] Code
3. Learning Structured Sparsity in Deep Neural Networks [NIPS 2016] Code
4. Sparse Convolutional Neural Networks [CVPR 2015]
5. Pruning Filters for Efficient ConvNets [ICLR 2017]

Quantization

1. Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights [ICLR 2017]
2. [https://arxiv.org/pdf/1706.02393.pdf]
3. Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding [ICLR 2016]
4. XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks [ECCV 2016] Code
5. Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
6. Trained Tenary Quantization [ICLR2017] Code
7. DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients Code
8. Binaryconnect: Training deep neural networks with binary weights during propagations [NIPS 2015]
9. Binarize neural networks: Training deep neural networks with weights and activations constrained to +1 or -1 [NIPS 2016]
10. Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
11. 8-Bit Approximations For Parallelism In Deep Learning [ICLR 2016]
12. [Quantized Convolutional Neural Networks for Mobile Devices]

Light Network Structure

1. SqueezeNet: AlexNet-Level Accuracy with 50x Fewer Parameters and < 0.5MB Model Size
2. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications Code
3. PVANet: Lightweight Deep Neural Networks for Real-time Object Detection Code

Distillation

1. Distilling the Knowledge in a Neural Network [NIPS 2014]

Speed Up

1. Fast Training of Convolutional Networks through FFTs [ICLR 2013] Code
2. Fast algorithms for convolutional neural networks [CVPR 2016]

Hardware Optimation

Related company and product

1.Movidius

2.DeePhi Tech

3.Google TPU

4.Nvidia Tesla V100