Instructors: Sergey Levine, John Schulman, and Chelsea Finn.

This course will assume some familiarity with reinforcement learning, numerical optimization and machine learning. The course covers topics: Supervised learning and decision making; Basic reinforcement learning: Q-learning and policy gradients; Advanced model learning and prediction; Advanced deep reinforcement learning: trust region policy gradients, actor-critic methods, exploration; Open problems and research talks.

Table of Contents

Lectures

• Lecture 1: Introduction and course overview (Video) (Slides)

• Lecture 2: Supervised learning and imitation (Video) (Slides)

• Lecture 3: Reinforcement learning introduction (Video) (Slides)

• Lecture 4: Policy gradients introduction (Video) (Slides)

• Lecture 5: Actor-critic introduction (Video) (Slides)

• Lecture 6: Value functions introduction (Video) (Slides)

• Lecture 7: Advanced Q-learning algorithms (Video) (Slides)

• Lecture 8: Optimal control and planning (Video) (Slides)

• Lecture 9: Learning dynamical systems from data (Video) (Slides)

• Lecture 10: Learning policies by imitating optimal controllers (Video) (Slides)

• Guest Lecture: Advanced model learning and images (Video) (Slides)

• Lecture 11: Connection between inference and control (Video) (Slides)

• Lecture 12: Inverse reinforcement learning (Video) (Slides)

• Lecture 13 (Part 1): Advanced policy gradients (natural gradient, importance sampling) (Video) (Slides)

• Lecture 13 (Part 2): Exploration (Video) (Slides)

• Lecture 14: Exploration (part 2) and transfer learning (Video) (Slides)

• Lecture 15: Multi-task learning and transfer (Video) (Slides)

• Lecture 16: Meta-learning and parallelism (Video) (Slides)

• Lecture 17: Advanced imitation learning and open problems (Video) (Slides)

Assignments

• Homework 1: Imitation Learning (Problem) (Solution)
• Homework 2: Policy Gradients (Problem) (Solution)
• Homework 3: Q-Learning on Atari (Problem) (Solution)
• Homework 4: Model-Based RL (Problem) (Solution)