Research Paper Reading List

A list of papers and resources I have read, am reading, or want to read. The majority are deep learning research papers, specifically in representation learning, reinforcement learning, and robotics.

Last updated: 11/15/2018

General Resources

Paper Lists

• Awesome Deep Learning
• Deep Learning Papers Reading Roadmap
• Awesome Deep Vision
• Deep Reinforcement Learning Papers
• ML@B Summer Reading List

Courses

• CS 189/289A: Introduction to Machine Learning, UC Berkeley, Fall 2015 [UC Berkeley login required]
• IEOR 265: Learning and Optimization, UC Berkeley, Spring 2016
• CS 294-129: Designing, Visualizing, and Understanding Deep Neural Networks, UC Berkeley, Fall 2016 [UC Berkeley login required]
• EE 221A: Nonlinear Systems, UC Berkeley, Fall 2016
• CS 294-112: Deep Reinforcement Learning, UC Berkeley, Spring 2017
• CS 294-131: Special Topics in Deep Learning, UC Berkeley, Spring 2017
• CS 231n: Convolutional Neural Networks for Visual Recognition, Stanford

Lab Blogs

• BAIR blog
• DeepMind blog
• OpenAI blog
• Google Research blog
• gBrain blog
• FAIR blog
• NVIDIA blog
• MSR blog

Personal Blogs

• Lilian Weng, OpenAI
• Eric Jang, Robotics at Google
• Alex Irpan, Robotics at Google

Reinforcement Learning (RL)

Papers

• [UNREAL] Reinforcement Learning with Unsupervised Auxiliary Tasks (ICLR 2017), M. Jaderberg et al. [pdf]
• Deep Learning for Real-Time Atari Game Play Using Offline Monte-Carlo Tree Search Planning (NIPS 2014), X. Guo et al. [pdf]
• A Unified Game-Theoretic Approach to Multiagent Reinforcement Learning (NIPS 2017), M. Lanctot et al. [pdf]
• Curiosity-driven Exploration by Self-supervised Prediction (ICML 2017), D. Pathak et al. [pdf]
• Learning to Poke by Poking: Experiential Learning of Intuitive Physics (NIPS 2016, oral), P. Agrawal et al. [pdf], [site]
• Time Limits in Reinforcement Learning, F. Pardo et al. [pdf]
• [DQN] Human-level control through deep reinforcement learning (Nature 2015), V. Mnih et al. [pdf]
• [DAgger] A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning (AISTATS 2011), S. Ross et al. [pdf]
• [Double DQN] Deep Reinforcement Learning with Double Q-Learning (AAAI 2016), H. Hasselt et al. [pdf]
• [Prioritized Replay] Prioritized Experience Replay (ICLR 2016), T. Schaul et al. [pdf]
• [Dueling DQN] Dueling Network Architectures for Deep Reinforcement Learning (ICML 2016 Best Paper), Z. Wang et al. [pdf]
• [Policy Gradients] Policy Gradient Methods for Reinforcement Learning with Function Approximation (NIPS 1999), R. Sutton et al.
• [DDPG] Continuous control with deep reinforcement learning (ICLR 2016), T. Lillicrap et al. [pdf
• [TRPO] Trust Region Policy Optimization (ICML 2015), J. Schulman et al. [pdf]
• [A3C] Asynchronous Methods for Deep Reinforcement Learning (ICML 2016), V. Mnih et al. [pdf] ross11a/ross11a.pdf)
• Continuous Deep Q-Learning with Model-based Acceleration (ICML 2016), S. Gu et al. [pdf]
• Real-Time Grasp Detection Using Convolutional Neural Networks (ICRA 2015), J. Redmon and A. Angelova [pdf]
• Deep Visual Foresight for Planning Robot Motion (ICRA 2017), C. Finn and S. Levine [pdf]
• Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection (IJRR 2017), S. Levine et al. [pdf]
• End-to-End Training of Deep Visuomotor Policies (JMLR 2016), S. Levine et al. [pdf]
• Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning (2017), A. Nagabandi et al. [pdf]
• Learning with Opponent-Learning Awareness (2017), J. Foerster et al. [pdf]
• An Empirical Study of AI Population Dynamics with Million-agent Reinforcement Learning (2017), Y. Yang et al. [pdf]
• Exploration by Random Network Distillation (2018), Y. Burda et al. [pdf]
• Time Reversal as Self-Supervision (NIPS 2018), S. Nair et al. [pdf]
• At Human Speed: Deep Reinforcement Learning with Action Delay (2018), V. Firouiu et al. [pdf]
• Visual Reinforcement Learning with Imagined Goals (2018), A. Nair et al. [pdf]
• Zero-Shot Visual Imitation (ICLR 2018), D. Pathak et al. [pdf]
• One-Shot Visual Imitation Learning via Meta-Learning (CoRL 2017), C. Finn et al. [pdf]
• One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learning (RSS 2018), T. Yu et al. [pdf]
• Large-Scale Study of Curiosity-Driven Learning (ICML 2018), Y. Burda et al. [pdf]
• Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review (2018), S. Levine. [pdf]
• Meta-Reinforcement Learning of Structured Exploration Strategies (2018), A. Gupta et al. [pdf]
• Psychlab: A Psychology Laboratory for Deep Reinforcement Learning Agents (2018), J. Leibo et al. [pdf]
• Deep Reinforcement Learning for Vision-Based Robotic Grasping: A Simulated Comparitive Evaluation of Off-Policy Methods (ICRA 2018), D. Quillen et al. [pdf]
• QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation (CoRL 2018), D. Kalashnikov et al. [pdf]
• Deep Reinforcement Learning that Matters (AAAI 2017), P. Henderson et al. [pdf]
• [R2D2] Recurrent Experience Replay in Distributed Reinforcement Learning (Under Review, ICLR 2019). [pdf]
• Modular Meta-learning (CoRL 2018), F. Alet et al. [pdf]

Other

• [Overview, DQN, Policy Gradients] Deep Reinforcement Learning Overview (2016), D. Silver [presentation]
• [DQN] Simple Reinforcement Learning with Tensorflow Part 4: Deep Q-Networks and Beyond (2016), A. Juliani [article]
• [Policy Gradients] Deep Reinforcement Learning: Pong from Pixels (2016), A. Karpathy [article]
• [A3C] Let's Make an A3C: Theory, J. Janisch [article]
• [Policy Gradients] Going Deeper Into Reinforcement Learning: Fundamentals of Policy Gradients (2017), D. Takeshi [article]
• Deep Reinforcement Learning Doesn't Work Yet (2018), A. Irpan. [article]

Representation Learning

Papers

• [TCN] Time-Contrastive Networks: Self-Supervised Learning from Video (CVPR 2017), P. Sermanet et al. [pdf]
• Learning an Embedding Space for Transferable Robot Skills (ICLR 2018), K. Hausman et al. [pdf]
• [VAE] Auto-Encoding Variational Bayes (ICLR 2013), D. Kingma et al. [pdf]
• [Transformer] Attention is All You Need (NIPS 2017), A. Vaswani et al. [pdf]

Other

• From Autoencoder to Beta-VAE, L. Weng [article]

Computer Vision / CNNs

Papers

• [ResNets] Deep Residual Learning for Image Recognition (CVPR 2016 Best Paper, ILSVRC & COCO 2015 winner), K. He et al. [pdf]
• ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices (2017), X. Zhang et al. [pdf]
• Fully Convolutional Networks for Semantic Segmentation (CVPR 2015), E. Shelhamer et al. [pdf]
• Learning to Extract Semantic Structure from Documents Using Multimodal Fully Convolutional Neural Networks (CVPR 2017 Spotlight), X. Yang et al. [pdf]
• Identity Mappings in Deep Residual Networks (ECCV 2016), K. He et al.

Other

• Understanding CNNs Part 3, A. Deshpande [article]

Natural Language Processing / RNNs

Papers

• Tuning Recurrent Neural Networks With Reinforcement Learning (ICLR 2017), N. Jacques et al. [pdf]

Other

• The Unreasonable Effectiveness of Recurrent Neural Networks (2015), A. Karpathy [article]

Model Compression

Papers

• [Iterative Pruning] Learning both Weights and Connections for Efficient Neural Networks (NIPS 2015), S. Han et al. [pdf]
• Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization, and Huffman Coding (ICLR 2016), S. Han et al. [pdf]
• Exploring the Regularity of Sparse Structure in Convolutional Neural Networks (NIPS 2017), H. Mao et al. [pdf]
• DSD: Dense-Sparse-Dense Training for Deep Neural Networks (ICLR 2017), S. Han et al. [pdf]
• Trained Ternary Quantization (ICLR 2017), K. Guo et al.
• Pruning Convolutional Neural Networks for Resource Efficient Inference (ICLR 2017), P. Molchanov et al. [pdf]
• Pruning Filters for Efficient ConvNets (ICLR 2017), H. Li et al. [pdf]
• Structured Pruning of Deep Convolutional Neural Networks (2015), S. Anwar et al. [pdf]

Other

• Pruning deep neural networks to make them fast and small (2017), J. Gildenblat [article]

Adversarial Learning / GANs

Papers

• The Limitations of Deep Learning in Adversarial Settings (EuroS&P 2016), N. Papernot et al. [pdf]
• Practical Black-Box Attacks against Machine Learning (ACM Asia 2017), N. Papernot et al. [pdf]
• Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples (2016), N. Papernot et al. [pdf]
• Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks (SP 2016), N. Papernot et al. [pdf]
• Adversarial Example Defenses: Ensembles of Weak Defenses are not Strong (2017), W. He et al. [pdf]
• MagNet: a Two-Pronged Defense against Adversarial Examples (2017), D. Meng and H. Chen [pdf]
• Explaining and Harnessing Adversarial Examples (ICLR 2015), I. Goodfellow et al. [pdf]
• [CAN] CAN: Creative Adversarial Networks, Generating "Art" by Learning About Styles and Deviating from Style Norms (ICCC 2017), A. Elgammal et al. [pdf]
• [BiGAN] Adversarial Feature Learning (ICLR 2017), J. Donahue et al. [pdf]
• [CycleGAN] Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks (ICCV 2017), J. Zhu et al. [pdf]
• [pix2pix] Image-to-Image Translation with Conditional Adversarial Nets (CVPR 2017), P. Isola et al. [pdf]
• [WGAN] Wasserstein GAN (ICML 2016), M. Arjovsky et al. [pdf]
• [PPGN] Plug & Play Generative Networks: Conditional Iterative Generation of Images in Latent Space (CVPR 2017), A. Nguyen et al. [pdf]

Control Theory

Papers

• Automatic LQR Tuning Based on Gaussian Process Global Optimization (ICRA 2016), A. Marco et al. [pdf]
• Bayesian Optimization for Learning Gaits Under Uncertainty (2015), R. Calandra et al. [pdf]
• Learning Quadrotor Dynamics Using Neural Network for Flight Control (CDC 2016), S. Bansal et al. [pdf]
• Goal-Driven Dynamics Optimization via Bayesian Optimization (CDC 2017), S. Bansal et al. [pdf]

Other

Papers

• How to Read a Paper (ACM SIGCOMM 2007), S. Keshav. [pdf]