Awesome Deep learning papers and other resources

A list of recent papers regarding deep learning and deep reinforcement learning. They are sorted by time to see the recent papers first. I will renew the recent papers and add notes to these papers.

You should find the papers and software with star flag are more important or popular.

Table of Contents

• Papers
• Model Zoo
• Pretrained Model
• Courses
• Books
• Software
• Applications
• Awesome Projects

Papers

• 2017 year
  • deep learning
  • deep reinforcement learning
  • natural language process
• 2016 year
  • deep learning
  • deep reinforcement learning
  • natural language process
• 2015 year
• 2014 year
• 2013 year
• 2012 year
• 2011 year
• 2010 year
• before 2010 year

Model Zoo

• 2012 | AlexNet | pdf | https://github.com/kratzert/finetune_alexnet_with_tensorflow |
• 2013 | RCNN | arxiv | https://github.com/rbgirshick/rcnn |
• 2014 | CGNA | arxiv | https://github.com/zhangqianhui/Conditional-Gans |
• 2014 | DeepFaceVariant | pdf | https://github.com/joyhuang9473/deepid-implementation |
• 2014 | GAN | arxiv | https://github.com/goodfeli/adversarial |
• 2014 | GoogLeNet | pdf | https://github.com/google/inception |
• 2014 | OverFeat | arxiv | https://github.com/sermanet/OverFeat |
• 2014 | SPPNet | arxiv | https://github.com/yhenon/keras-spp |
• 2014 | VAE | arxiv | https://github.com/dpkingma/nips14-ssl |
• 2014 | VGGNet | arxiv | https://gist.github.com/ksimonyan/211839e770f7b538e2d8 |
• 2015 | DCGAN | arxiv | https://github.com/carpedm20/DCGAN-tensorflow |
• 2015 | DRAW | arxiv | https://github.com/ericjang/draw |
• 2015 | Global And Local Attention | arxiv | https://github.com/giancds/tsf_nmt |
• 2015 | FaceNet | arxiv | https://github.com/davidsandberg/facenet |
• 2015 | Fast RCNN | arxiv | https://github.com/rbgirshick/fast-rcnn |
• 2015 | Faster RCNN | arxiv | https://github.com/rbgirshick/py-faster-rcnn |
• 2015 | FCNT | pdf | https://github.com/scott89/FCNT |
• 2015 | Inception | arxiv | https://github.com/tensorflow/models/tree/master/inception |
• 2015 | LAPGAN | arxiv | https://github.com/facebook/eyescream |
• 2015 | NeuralGPU | arxiv | https://github.com/tensorflow/models/tree/master/neural_gpu |
• 2015 | Pointer Net | arxiv | https://github.com/devsisters/pointer-network-tensorflow |
• 2015 | ResNet | arxiv1 , arxiv2, arxiv3 | https://github.com/tensorflow/models/tree/master/resnet |
• 2015 | Skip-Thought Vectors | pdf | https://github.com/tensorflow/models/tree/master/skip_thoughts |
• 2015 | Transformer | arxiv | https://github.com/tensorflow/models/tree/master/transformer |
• 2016 | Dp_sgd | arxiv | https://github.com/tensorflow/models/tree/master/differential_privacy |
• 2016 | EnergyGAN | arxiv | https://github.com/buriburisuri/ebgan |
• 2016 | Grad-CAM | arxiv | https://github.com/Ankush96/grad-cam.tensorflow |
• 2016 | Im2txt | arxiv | https://github.com/tensorflow/models/tree/master/im2txt |
• 2016 | InfoGAN | arxiv | https://github.com/buriburisuri/supervised_infogan |
• 2016 | Multiple_teachers | arxiv | https://github.com/tensorflow/models/tree/master/differential_privacy |
• 2016 | Neural Programmer | pdf | https://github.com/tensorflow/models/tree/master/neural_programmer |
• 2016 | PCNN | arxiv | https://github.com/kundan2510/pixelCNN |
• 2016 | Pix2Pix | arxiv | https://github.com/yenchenlin/pix2pix-tensorflow |
• 2016 | PVANet | arxiv | https://github.com/sanghoon/pva-faster-rcnn |
• 2016 | R-FCN | arxiv | https://github.com/Orpine/py-R-FCN |
• 2016 | SeqGAN | pdf | https://github.com/LantaoYu/SeqGAN |
• 2016 | SqueezeNet | arxiv | https://github.com/songhan/SqueezeNet-Deep-Compression |
• 2016 | Swivel | arxiv | https://github.com/tensorflow/models/tree/master/swivel |
• 2016 | SyntaxNet | arxiv | https://github.com/tensorflow/models/tree/master/syntaxnet |
• 2016 | Textsum | | https://github.com/tensorflow/models/tree/master/textsum |
• 2016 | VGNA | arxiv | https://github.com/Shuangfei/vgan |
• 2017 | Learning to Remember Rare Events | pdf | https://github.com/tensorflow/models/tree/master/learning_to_remember_rare_events |
• 2017 | SalGAN | arxiv | https://github.com/imatge-upc/saliency-salgan-2017 |
• 2017 | WGAN | arxiv | https://github.com/Zardinality/WGAN-tensorflow |

Pretrained Model

• Available pretrained word embeddings
• Inception-v3 of imagenet
• Dependency-Based Word Embeddings.
• GloVe: Global Vectors for Word Representation
• Model of the deep residual network used for cifar10
• Pre-Trained Doc2Vec Models
• Pre-trained word vectors ⭐
• ResNet in TensorFlow Pretrain Model
• TensorFlow VGG-16 pre-trained model
• VGGNets for Scene Recognition

Courses

• [Utah] Applied Computational Genomics Course at UU
• [Stanford] CS231n: Convolutional Neural Networks for Visual Recognition
• [CUHK] ELEG 5040: Advanced Topics in Signal Processing(Introduction to Deep Learning)
• [Stanford] CS224n: Natural Language Processing with Deep Learning
• [Oxford] Deep Learning by Prof. Nando de Freitas
• [NYU] Deep Learning by Prof. Yann LeCun
• [Berkeley] CS294: Deep Reinforcement Learning
• [Berkeley] Stat212b：Topics Course on Deep Learning
• [MIT] S094: Deep Learning for Self-Driving Cars
• [CUHK] ELEG 5040: Advanced Topics in Signal Processing (Introduction to Deep Learning)
• [Stanford] CS20SI: Tensorflow for Deep Learning Research
• [Stanford] CS224n: Natural Language Processing with Deep Learning
• [MIT] S191: Introduction to Deep Learning
• [吴立德] 《深度学习课程》
• [Oxford] Deep Learning Course
• [David Silver] RL Course
• [MIT] Practical Deep Learning For Coders
• [Google] Udacity Deep Learning Online Course

Books

• Deep Learning by Ian Goodfellow, Yoshua Bengio, and Aaron Courville, [zh]
• Neural Networks and Deep Learning by Michael Nielsen
• Deep Learning Tutorial by LISA lab, University of Montreal
• 神经网络与深度学习.邱锡鹏
• UFLDL Tutorial
• Rules of Machine Learning: Best Practices for ML Engineering
• Reinforcement Learning: An Introduction [code]
• Reinforcement LearningState-of-the-Art
• A Course in Machine Learning
• 深度学习入门 by PaddlePaddle
• TensorFlow For Machine Intelligence
• First Contact With TensorFlow
• Learning scikit-learn: Machine Learning in Python

Software

• python
  • [Chainer] Chainer bridge the gap between algorithms and implementations of deep learning.
  • [ChainerRL] ChainerRL is a deep reinforcement learning library built on top of Chainer.
  • [DeepPy] DeepPy is a Pythonic deep learning framework built on top of NumPy.
  • [Deepnet] Deepnet: a GPU-based python implementation of deep learning algorithms.
  • [Deepgaze] Deepgaze: A computer vision library for human-computer interaction based on CNNs
  • [DeepQA] Tensorflow implementation of "A neural conversational model", a Deep learning based chatbot.
  • [DeepVideoAnalytics] Analyze videos & images, perform detections, index frames & detected objects, search by examples.
  • [Edward] Edward: A library for probabilistic modeling, inference, and criticism.
  • [Elephas] Distributed Deep learning with Keras & Spark.
  • [Gensim] Gensim: Deep learning toolkit implemented in python programming language intended for handling large text collections, using efficient algorithms.
  • [Hebel] Hebel: A library for deep learning with neural networks in Python using GPU acceleration with CUDA through PyCUDA.
  • [Keras] Keras: Deep Learning library for Theano and TensorFlow. ⭐
  • [Kur] Kur: Descriptive Deep Learning. ⭐
  • [Neon] Neon is Nervana's Python based Deep Learning framework.
  • [PyTorch] Tensors and Dynamic neural networks in Python with strong GPU acceleration. ⭐
  • [Scikit-Learn] scikit-learn: machine learning in Python. ⭐
  • [Skll] SciKit-Learn Laboratory (SKLL) makes it easy to run machine learning experiments.
  • [TensorFX] TensorFX is an end to end application framework to simplifies machine learning with TensorFlow
  • [Theano] Theano: Mathematical library in Python, maintained by LISA lab
    • [Pylearn2] Theano-based deep learning libraries
    • [Blocks] Blocks is a framework that helps you build neural network models on top of Theano ⭐
    • [Lasagne] Lightweight library to build and train neural networks in Theano.
• C++
  • [Caffe] Caffe: Deep learning framework by the BVLC ⭐
  • [CNTK] CNTK:The Microsoft Cognitive Toolkit.
  • [DeepDetect] DeepDetect : Open Source Deep Learning Server & API
  • [DIGITS] NVIDIA DIGITS is a new system for developing, training and visualizing deep neural networks.
  • [DSSTNE] DSSTNE is an Amazon developed library for building Deep Learning (DL) machine learning (ML) models.
  • [PaddlePaddle] PaddlePaddle (PArallel Distributed Deep LEarning) is an easy-to-use, efficient, flexible and scalable deep learning platform.
  • [MXNet] MXNet: A flexible and efficient deep learning library for heterogeneous distributed systems with multi-language support ⭐
    • [MinPy] MinPy: Providing a high performing and flexible deep learning platform, by prototyping a pure NumPy interface above MXNet backend.
  • [Singa] Singa: Singa is an Apache Incubating project for developing an open source deep learning library.
  • [Tensorflow] Tensorflow: An open source software library for numerical computation using data flow graph by Google ⭐
  • [Tiny-dnn] Tiny-dnn is a C++11 implementation of deep learning.
• Java
  • [CoreNLP] Stanford CoreNLP: A Java suite of core NLP tools.
  • [Deeplearning4J] Deeplearning4J: Neural Net Platform.
  • [Librec] LibRec: A Java Library for Recommender Systems.
  • [NeuralNetworks] This is a Java implementation of some of the algorithms for training deep neural networks.
  • [NewralNet] A lightweight, easy to use and open source Java library for experimenting with feed-forward neural nets and deep learning.
• Scala
  • [BigDL] BigDL: Distributed Deep learning on Apache Spark.
• Julia
  • [Knet] Knet: Knet (pronounced "kay-net") is the Koç University deep learning framework implemented in Julia.
  • [Mocha] Mocha is a Deep Learning framework for Julia, inspired by the C++ framework Caffe.
• Js
  • [Keras-js] Run Keras models (tensorflow backend) in the browser, with GPU support.
  • [Neurojs] A javascript deep learning and reinforcement learning library.
• Matlab
  • [MatConvNet] MatConvNet: CNNs for MATLAB
• Lua
  • [OpenNMT] OpenNMT: Open-Source Neural Machine Translation
  • [Torch7] Torch7: Deep learning library in Lua, used by Facebook and Google Deepmind ⭐
• Php
  • [PHP-ML] PHP-ML - Machine Learning library for PHP

Applications

• pytorch
  • A fast and differentiable QP solver for PyTorch.
  • A PyTorch Implementation of Single Shot MultiBox Detector.
  • A simple PyTorch Implementation of Generative Adversarial Networks, focusing on anime face drawing
  • Comprehensive Data Augmentation and Sampling for Pytorch
  • CNNs for Sentence Classification in PyTorch
  • Deep Q-Learning Network in pytorch
  • Fast Neural Style for Image Style Transform by Pytorch
  • Highway networks implemented in PyTorch.
  • OpenNMT: Open-Source Neural Machine Translation in PyTorch ⭐
  • PyTorch implementation of Fully Convolutional Networks
  • PyTorch implementation of Global Vectors for Word Representation
  • PyTorch implementation of the Value Iteration Networks
  • Pytorch Negative Sampling Loss
  • Pytorch Poetry Generation
  • Sequence to Sequence Models with PyTorch
  • SSD: Single Shot MultiBox Object Detector, in PyTorch
  • t-SNE experiments in pytorch
  • YOLOv2 in PyTorch
• theano
  • CNN-yelp-challenge-2016-sentiment-classification
  • Deep Neural Network for Sentiment Analysis on Twitter
  • Implementations of many popular deep learning models in Theano+Lasagne
• tensorflow
  • A framework for developing and evaluating reinforcement learning algorithms
  • A general-purpose encoder-decoder framework for Tensorflow that can be used for Machine Translation, Text Summarization, Conversational Modeling, Image Captioning, and more.
  • An implementation of Pix2Pix in Tensorflow for use with frames from films
  • A Practical Guide for Debugging TensorFlow Codes
  • A set of Deep Reinforcement Learning Agents implemented in Tensorflow. ⭐
  • Aspect Based Sentiment Analysis using End-to-End Memory Networks
  • A tensorflow implementation of "Deep Convolutional Generative Adversarial Networks" ⭐
  • A TensorFlow implementation of Baidu's DeepSpeech architecture
  • DeepColor: Automatic coloring and shading of manga-style lineart
  • Deep Learning based Python Library for Stock Market Prediction and Modelling
  • Deepnlp:Deep Learning NLP Pipeline implemented on Tensorflow
  • Embedding Watermarks into Deep Neural Networks
  • Fast Multi(Interpolative) Style Transfer
  • Fast PixelCNN++: speedy image generation
  • Fully differentiable deep-neural decision forest in tensorflow
  • GA3C: Reinforcement Learning through Asynchronous Advantage Actor-Critic on a GPU
  • Implementation of Generative Adversarial Networks, for Audio.
  • Interactive, node-by-node debugging and visualization for TensorFlow ⭐
  • Machine Learning on Sequential Data Using a Recurrent Weighted Average
  • Metasploit for machine learning.
  • Multilabel time series classification with LSTM
  • Neural Relation Extraction implemented with LSTM in TensorFlow
  • PixelVAE with or without regularization
  • PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation ⭐
  • Realistic Handwriting with Tensorflow
  • Real-time Joint Semantic Reasoning for Autonomous Driving
  • Self-Driving Car Engineer Nanodegree
  • Sequence-to-Sequence Grapheme-to-Phoneme toolkit
  • Simple Recommender System using Denoising AutoEncoder, implemented using TensorFlow
  • SSD in TensorFlow: Traffic Sign Detection and Classification
  • Tensorflow implementation of fast neural style transfer
  • TensorFlow on iOS demo
  • TensorFlowOnSpark
  • Tensorflow port of Image-to-Image Translation with Conditional Adversarial Nets ⭐
  • Tensorflow Tutorial files and Implementations of various Deep NLP and CV Models
  • Tutorials for deep learning
  • 使用TensorFlow实现的Sequence to Sequence的聊天机器人模型
  • Udacity SDC: Vehicle Detection ⭐
• Keras
  • Embedding Watermarks into Deep Neural Networks
  • Experimental implementation of novel neural network structures
  • keras-emoji-embeddings
  • Wasserstein DCGAN in Tensorflow/Keras

Awesome Projects

• 15 AI and Machine Learning Events
• Awesome Adversarial Machine Learning
• Awesome-Caffe
• Awesome Deep Learning
• Awesome Deep learning papers and other resources
• Awesome Deep Vision
• Awesome - Most Cited Deep Learning Papers
• Awesome Public Datasets
• Awesome PyTorch ⭐
• Awesome Reinforcement Learning
• Awesome Robotics
• Awesome Sentiment Analysis
• Awesome TensorFlow ⭐
• Chainer Info
• Collection of generative models, e.g. GAN, VAE in Tensorflow, Keras, and Pytorch ⭐
• Collection of reinforcement learners implemented in python.
• Datasets, Transforms and Models specific to Computer Vision
• Deep Learning Papers Reading Roadmap
• Machine Learning Videos
• Machine Learning From Scratch
• Paper list of multi-agent reinforcement learning (MARL)
• SCODE Word Embeddings using Substitute Words
• Summaries and notes on Deep Learning research papers
• The Incredible PyTorch: a curated list of tutorials, papers, projects, communities and more relating to PyTorch ⭐
• Various math-related things in Python code
• 用于对话系统的中英文语料