Reinforcement Learning by University of Waterloo

CS 885 Spring 2018 at University of Waterloo - Reinforcement Learning

Description

The course introduces students to the design of algorithms that enable machines to learn based on reinforcements. In contrast to supervised learning where machines learn from examples that include the correct decision and unsupervised learning where machines discover patterns in the data, reinforcement learning allows machines to learn from partial, implicit and delayed feedback. This is particularly useful in sequential decision making tasks where a machine repeatedly interacts with the environment or users. Applications of reinforcement learning include robotic control, autonomous vehicles, game playing, conversational agents, assistive technologies, computational finance, operations research, etc.

Topics

• Markov decision processes
• Bandits
• Model free reinforcement learning
• Model based reinforcement learning
• Partially observable reinforcement learning
• Deep reinforcement learning
• Hierarchical reinforcement learning

Potential applications

• Robotic control
• Game playing
• Conversational agents
• Operations research
• Assistive technologies
• Intelligent tutoring systems
• Computational finance
• Autonomous vehicles

Lectures

1. Course introduction (slides) (video)
2. Markov Processes (slides) (video)
3. Markov Decision Processes (slides) (video)
4. Value Iteration (slides) (video)
5. Policy Iteration (slides) (video)
6. Introduction to RL (slides) (video)
7. Deep neural networks (slides) (video)
8. Deep Q-Networks (slides) (video)
9. Guest lecture by Nabiha Asghar on RL for dialog systems (slides) (video)
10. Guest Lecture by Mike Rudd on OpenAI environments (slides) (video)
11. Guest Lecture by Timmy Tse on DQN and TensorFlow (slides) (video)
12. Policy Gradient (slides) (video)
13. Actor Critic (slides) (video)
14. Multi-armed bandits (slides) (video)
15. Bayesian and contextual bandits (slides) (video)
16. Model-based RL (slides) (video)
17. Bayesian RL (slides) (video)
18. Hidden Markov models (slides) (video)
19. Partially observable RL (slides) (video)
20. Deep recurrent Q-networks (slides) (video)
21. RL for video games. Playing FPS Games with Deep RL (slides) (video)
22. RL for video games. A Deep Hierarchical Approach to Lifelong Learning in Minecraft (slides) (video)
23. Adversarial Search (slides) (video)
24. RL for computer Go (slides) (video)
25. RL for board games (slides) (video)
26. Trust Region Methods (slides) (video)
27. Policy optimization. Trust region policy optimization (slides) (video)
28. Policy optimization. Proximal policy optimization (slides) (video)
29. Semi-Markov Decision Processes (slides) (video)
30. Hierarchical RL. The Option-Critic Architecture (slides) (video)
31. Hierarchical RL. FeUdal Networks for Hierarchical RL (slides) (video)
32. RL for robotics. Target-driven Visual Navigation in Indoor Scenes using Deep RL (slides) (video)
33. RL for robotics. Control of a Quadrotor with Reinforcement Learning (slides) (video)
34. Inverse Reinforcement Learning (slides) (video)
35. RL for autonomous vehicles. Safe, Multi-Agent, RL for Autonomous Driving (slides) (video)
36. RL for autonomous vehicles. Learning Driving Styles for Autonomous Vehicles from Demonstration (slides) (video)
37. RL for conversational agents. E2E lstm-based dialog control optimized with supervised and RL (slides) (video)
38. RL for conversational agents. Learning Cooperative Visual Dialog Agents with Deep RL (slides) (video)
39. Memory Augmented Networks (slides) (video)
40. Memory based RL. Neural Map: Structured Memory for Deep RL (slides) (video)
41. Memory based RL. Memory Augmented Control Networks (slides) (video)