This repository is dedicated to all things Reinforcement Learning, covering theory, practice, and applications.
This repository chronicles my exploration of the dynamic field of Reinforcement Learning (RL). As I embark on this learning adventure, I'll be diligently pushing all my work here, including scripts, projects, and a plethora of resources. From videos, lectures, and blog posts to reference articles and beyond, I'll document every step of my learning process. I'm committed to following the advice from experts and leveraging diverse learning materials to enrich my understanding and skills in RL. Join me as I dive deep into the fascinating world of Reinforcement Learning!
I am following the advice from Spinning Up as a Deep RL Researcher by Joshua Achiam summarised here.
- The Right Background
- Build up a solid mathematical background
- Build up a general knowledge of deep learning
- Become familiar with at least one deep learning library
- Get comfortable with the main concepts and terminology in RL
- Learn by Doing
- Write your own implementations
- Simplicity is critical. Implement the simplest algorithms first
- Start with vanilla policy gradient (also called REINFORCE), DQN, A2C (the synchronous version of A3C), PPO (the variant with the clipped objective), and DDPG
- Focus on understanding
- Implement an algorithm based on a paper but don’t overfit to paper details
- Study existing implementations for inspiration
- Iterate fast in simple environments like CartPole-v0, InvertedPendulum-v0, FrozenLake-v0, and HalfCheetah-v2
- Measure everything
- Scale experiments when things work
- Developing a Research Project
- Start by exploring the literature to become aware of topics in the field
- Approaches to idea-generation
- Improving on an Existing Approach
- Focusing on Unsolved Benchmarks
- Create a New Problem Setting
- Avoid reinventing the wheel
- Doing Rigorous Research in RL
- Set up fair comparisons
- Remove stochasticity as a confounder (3 - 10 random seeds)
- Run high-integrity experiments
- Check each claim separately
Read the entire article linked above for more insights.
This section will house all the resources I accessed during this journey.
I will endeavour to catalog all the resources I can though I may inadvertently overlook some.
| Resource Type | Name |
|---|---|
| Books | |
| Reinforcement Learning: An Introduction second edition by Richard S. Sutton and Andrew G. Barto | |
| Courses | |
| Reinforcement Learning beginner to master - AI in Python by Escape Velocity Labs | |
| Hugging Face's RL course | |
| Foundations of Deep RL -- 6-lecture series by Pieter Abbeel | |
| Deep Reinforcement Learning: CS 285 Fall 2020 and CS 285 at UC Berkeley website | |
| Google DeepMind x UCL RL Lecture Series 2021 | |
| RL Algorithms Implementation Study Group by Costa Huang | |
| Rainbow is all you need! - from DQN to Rainbow | |
| PG is all you need! from A2C to SAC | |
| Libraries | |
| Gym | |
| Gymnasium | |
| Stable Baselines3 | |
| RL Baselines3 Zoo | |
| Sample Factory | |
| CleanRL | |
| Optuna | |
| Articles | |
| A Map of Reinforcement Learning | |
| Custom Environments | |
| SnowballFight | |
| Huggy the Doggo | |
| VizDoom(Doom) | |
| PyBullet | |
| Projects | |
- Tabular methods
- Markov decision process
- Dynamic programming
- Monte Carlo methods
- Time difference methods (SARSA, Q-Learning)
- N-step bootstrapping
- Continuous state spaces
- State aggregation
- Tile Coding
- Deep Reinforcement Learning
- Deep SARSA
- Deep Q-Learning
- REINFORCE
- Advantage Actor-Critic / A2C
__Methods Mind Map__
Special thanks to the creators of the various RL courses, libraries, and environments included in this repository. Their efforts have been instrumental in advancing the field of Reinforcement Learning and making valuable resources accessible to learners worldwide.
Have questions, suggestions, or feedback? Feel free to reach out to me on LinkedIn. I'm always excited to engage with fellow RL enthusiasts and learners.
Happy Reinforcement Learning!
