Skip to content
ask4jubad edited this page Feb 15, 2023 · 106 revisions

This is the wiki for the reinforcement learning Discord.


  1. First Steps
  2. Debugging Advice
  3. Recommended Resources
  4. Links

First Steps

If you're entirely new to reinforcement learning, the best thing to do is to poke through the resources listed in Written Introductions and Video Introductions and see which ones suit your learning style.

All of these cover roughly the same material, and all have roughly the same prerequisites:

  • you should be comfortable with probability, statistics, linear algebra and multivariable calculus at a first-year-undergrad level,
  • you should be comfortable with programming in Python,
  • and you should be familiar with deep learning concepts like convolutions and gradient descent.

If you're entirely new to probability/statistics/linear algebra/multivariable calculus, we'll make a weak recommendation for Khan Academy's stats/lin alg/calc courses, or for the Mathematics for Machine Learning book. These resources suit a lot of people well.

If you're entirely new to programming in Python, the /r/python wiki has a range of introductory resources and we'll make a weak recommendation for Automate the Boring Stuff With Python as a book that suits many people well.

If you're entirely new to deep learning, the /r/ML wiki has a range of introductory resources, and we'll make a weak recommendation for as a course that suits many people well.

Debugging Advice

If you go through these and you're still stuck, come ask for help in #practical on the Discord. Keep in mind that debugging RL is extraordinarily tricky even when you've direct access, so don't expect miracles! It helps a lot if you can share a minimal example, describe the problem in detail, and explain how you've tried to fix it so far. The more detail you put into your question, the more likely it'll be a good answer!

Recommended Resources

These resources are things we have first-hand experience of and can testify to the quality of.

If you've had a good experience with a resource, edit this page and increment the 👍. Feel free to move stuff from the things-we-found section up to here if you've personally had a good experience with it. If you're uncertain, come discuss it in #wiki first.

Written Introductions

Video Introductions


Frameworks and Libraries

  • 2👍 Stable Baselines in Pytorch (SB3), a set of robust, reliable implementations of RL algorithms.
  • 2👍 RLlib by Ray, one of the better attempts at plug-n-play distributed RL.
  • 1👍 CleanRL, a set of single-file implementations of basic RL algorithms.
  • 1👍 AI Traineree, simple implementations with agents/envs being independent.

Other Communities

  • General RL
  • Specialist RL
    • L0 and Lc0: the Discords for Leela Zero and Leela Chess Zero, two open-source AlphaZero adaptations.
    • SC2 AI: Discord for StarCraft 2 bot development.
    • DonkeyCar League: a weekly friendly bot race league.
  • General ML
    • EleutherAI: language-model focussed, but a lot of good general AI safety and scaling discussion too.
    • TPU Podcast: GAN focussed, but a lot of good general ML discussion. Origin of the EleutherAI discord and project.
    • ML Collective: More general than the others mentioned so far, while still explicitly research-focussed.


This is a broad-strokes collection of things we've found in our around the internet that we think might be useful to others. It is much more lightly curated than the section above.

Lecture Videos


  • BAIR Berkeley Artificial Intelligence Research.
  • RL Weekly is a weekly newsletter highlighting important progress in reinforcement learning in research or industry.


RL Frameworks & Reference Implementations

RL Environments

The paperswithcode list is a good place to start.