This repository features implementations of three reinforcement learning (RL) algorithms applied to the classic CartPole task from OpenAI Gym: Q-learning, SARSA, and Deep Q-Network (DQN). Each algorithm demonstrates a unique approach to learning how to balance a pole on a moving cart.
The CartPole task is a benchmark challenge in reinforcement learning. The agent controls a cart with a pole attached; the goal is to prevent the pole from falling over. The project showcases:
- Q-learning: An off-policy algorithm for learning the optimal action-value function.
- SARSA: An on-policy algorithm that updates Q-values using the action performed by the current policy.
- Deep Q-Network (DQN): Uses deep neural networks to approximate the Q-value function, with techniques like experience replay and fixed Q-targets for stability.
First, clone this repository: https://github.com/AlexanderAKM/Cartpole.git
Then, install the required dependencies:
pip install -r requirements.txtIf you want to run an agent, for example, SARSA for 4000 episodes, use:
python src/main.py --agent sarsa --episodes 4000If you want to plot the data from an experiment, for example, SARSA for 4000 episodes, use:
python src/plot.py -f data/input/rewards_sarsa_4000.csvThis project is licensed under the MIT License - see the LICENSE.md file for details.
- OpenAI Gym for providing the CartPole environment.
- Documentation of Pytorch on DQN implementation for cartpole: https://pytorch.org/tutorials/intermediate/reinforcement_q_learning.html
- Do much more hyperparameter tuning