🎮 Reinforcement Learning with Q-Learning and Deep Q-Learning

This project implements Q-Learning and Deep Q-Learning (DQN) agents on two environments from the RL Gym library: FrozenLake and Atari Pong.
The goal is to explore reinforcement learning techniques and understand the impact of key hyperparameters.

---

📚 Project Overview

• Frameworks: Gymnasium, NumPy, PyTorch
• Environments: FrozenLake, Atari Pong
• Parts:
  • Q-Learning on FrozenLake
  • Policy Iteration on FrozenLake
  • Deep Q-Learning (DQN) on Atari Pong

---

🛠 How to Run

1. Clone the repository

   git clone https://github.com/your-username/reinforcement-learning-q-dqn.git
   cd reinforcement-learning-q-dqn

2. Install dependencies

   pip install gymnasium[box2d,atari] numpy matplotlib torch

3. Run FrozenLake notebook

   jupyter notebook frozenlake_q_learning.ipynb

4. Run Atari Pong DQN notebook

   jupyter notebook atari_pong_dqn.ipynb

---

💡 Features & Highlights

• Q-Learning agent for FrozenLake with:
  • Learning rate (alpha)
  • Discount factor (gamma)
  • Exploration rate (epsilon)
• Policy Iteration implementation for FrozenLake
• Hyperparameter experiments with 3+ values per parameter
• Deep Q-Learning (DQN) agent for Atari Pong using a neural network
• Training and evaluation visualizations

---

📊 Results Summary

Environment	Algorithm	Performance Highlights
FrozenLake	Q-Learning	Learned to reach the goal with tuned epsilon, alpha, gamma
FrozenLake	Policy Iteration	Faster convergence vs Q-Learning
Atari Pong	Deep Q-Learning	Neural network agent learned to play Pong; improved performance over episodes

✅ Deep Q-Learning showed promising results in a complex visual environment.

---

✨ Dataset / Environment Links

• FrozenLake Environment
• Atari Pong Environment

---

📄 References

• FrozenLake Q-Learning Tutorial
• Gymnasium Docs
• Deep Q-Learning Papers & Guides

---

🎓 Key Learnings

• Understanding Q-Learning and Policy Iteration fundamentals
• Exploring the role of hyperparameters (alpha, gamma, epsilon)
• Applying deep learning to approximate Q-values
• Handling high-dimensional input in Atari games
• Visualizing and interpreting RL training performance

---

📜 License

© 2025 Anamol Khadka. All rights reserved.
This work is licensed under the Creative Commons Attribution 4.0 International (CC BY 4.0) license.

You are free to share and adapt the material for any purpose, even commercially, as long as appropriate credit is given.

For inquiries, please contact: khadkaanamol8@gmail.com