👾 QLearning: Berkeley Pacman Reinforcement Learning

An implementation of reinforcement learning algorithms — including Q-learning and value iteration — applied to the classic Pacman environment. This project is based on UC Berkeley’s CS188 AI coursework.

---

🧠 Architecture & Technology Breakdown

🧱 Core Modules

Module	Description
pacman.py	Main game loop and environment setup
qlearningAgents.py	Q-learning agent implementation
valueIterationAgents.py	Value iteration agent implementation
mdp.py	Defines the Markov Decision Process abstraction
learningAgents.py	Base classes for reinforcement learning agents
environment.py	Manages agent-environment interactions
featureExtractors.py	Extracts features for approximate Q-learning
graphicsDisplay.py	Renders the Pacman game visually
gridworld.py	Alternate environment for testing policies
autograder.py	Grades agent performance (used in coursework)

---

🔄 Reinforcement Learning Algorithms

🟨 Q-Learning

• Model-free algorithm
• Learns optimal policy by updating Q-values:

\[
  Q(s, a) \leftarrow Q(s, a) + \alpha \left[ r + \gamma \max_{a'} Q(s', a') - Q(s, a) \right]
  \]

• Implemented in qlearningAgents.py

🟦 Value Iteration

• Model-based algorithm
• Iteratively updates state values using Bellman equation
• Implemented in valueIterationAgents.py

---

🧪 Testing & Evaluation

• Test cases are located in test_cases/ and testClasses.py

• Run tests using:

  python autograder.py

  📁 Project Structure

QLearning/
├── pacman.py                  # Main game loop and environment setup
├── qlearningAgents.py         # Q-learning agent implementation
├── valueIterationAgents.py    # Value iteration agent implementation
├── mdp.py                     # Markov Decision Process abstraction
├── learningAgents.py          # Base classes for RL agents
├── environment.py             # Agent-environment interaction logic
├── featureExtractors.py       # Feature extraction for approximate Q-learning
├── gridworld.py               # Alternate environment for policy testing
├── graphicsDisplay.py         # Visual rendering of Pacman
├── autograder.py              # Performance and correctness grading
├── test_cases/                # Unit tests and evaluation scripts
├── layouts/                   # Maze layouts for Pacman
└── util.py                    # Utility functions

📄 License This project is based on UC Berkeley’s CS188 materials. Use for educational purposes only.

🙋‍♂️ Author Created by Stephen Blanchard