RL-Inator

A hands-on project utilizing concepts from Reinforcement Learning, Linear Algebra, and Robotics.

Table of Contents

• About the Project
• Getting Started
  • Dependencies
  • Installations
• Usage Examples
• Future Work
  • Tasks
• Resources and Acknowledgements
  • Resources

About the Project

This project represents my journey in reinforcement learning. It includes scripts implementing
techniques to solve various problems posed in reinforcement learning ranging from solving a
simple version of the k-armed bandits problem to the full reinforcement learning problem, with
environments having observation spaces large enough to appear intractable to classical tabular
methods. In addition to these, the project contains some custom environments adhering to the
OpenAI Gym API.

Getting Started

Dependencies

• Python
• Jupyter
• NumPy
• SciPy
• OpenAI Gym v0.26.1
• OpenCV
• TensorFlow
• Keras

Versions included wherever they matter.

Installations

Just clone this repository - git clone https://github.com/virajbshah/rl-inator.git - and
you're ready to go! Dependencies can also be installed using pip install -r requirements.txt

Usage Examples

Each script has its own usage. For example, the each of the Jupyter Notebooks will initialize their
respective environments, solve them, and display their solutions upon executing all code cells.

Future Work

Tasks

• Go through material outlining any mathematical prerequisites, such as Linear Algebra.
• Get familiarized with the core concepts behind Reinforcement Learning (Purely Theoretical).
• Implement various policies for the Multi-Armed Bandits RL subproblem.
• Utilize Dynamic Programming to create a general solution for problems where the MDP is known.
• Solve problems with unknown MDPs using Monte Carlo methods.
• Work with Temporal Difference methods (Q-Learning) to solve control problems.
• Use Deep Q-Networks to adapt methods like Q-Learning to problems where using Q-Tables is not
  feasible as a result of the complexity of the problem.
• Implement various custom OpenAI Gym environments.

Resources and Acknowledgements

Resources

• Introduction to Linear Algebra (Gilbert Strang)
• Reinforcement Learning: An Introduction
• Algorithms for Reinforcement Learning
• DeepMind x UCL Deep Learning Lecture Series
• Foundations of Deep RL (Pieter Abbeel)
• OpenAI Gym Documentation

Maintained by Viraj Shah, as part of a program organized by SRA VJTI.