🎮 2048 AI - Deep Reinforcement Learning

AI-powered 2048 game player using Deep Q-Network (DQN). Watch the AI learn to play 2048 from scratch and achieve the 2048 tile consistently!

✨ Features

• Complete 2048 game engine with numpy-optimized board operations
• DQN reinforcement learning agent with experience replay and target networks
• Live gameplay visualization using pygame (watch the AI play in real-time)
• Training metrics dashboard with matplotlib (track learning progress)
• Comprehensive logging and checkpoint management
• Metrics export to CSV/JSON for external analysis
• Human-like code style - readable, slightly messy, authentic

🚀 Quick Start

Installation

# clone repo
git clone https://github.com/pc-style/2048_take2.git
cd 2048_take2

# install dependencies
pip install -r requirements.txt

Train the AI

# basic training (5000 episodes)
python3 -m ai.training

# custom number of episodes
python3 -m ai.training 1000

# with custom config
python3 -m ai.training --config my_config.json

Watch the AI Play

# Watch agent play (uses latest checkpoint)
python3 scripts/watch_agent.py

# Play the game yourself
python3 scripts/play_game.py

# Play with AI hints
python3 scripts/play_with_hints.py

📁 Project Structure

2048_take2/
├── game/              # Core 2048 game engine
│   ├── board.py       # Board state and move logic
│   └── __init__.py
├── ai/                # Reinforcement learning agent
│   ├── agent.py       # DQN agent implementation
│   ├── network.py     # Neural network architecture
│   ├── replay_buffer.py  # Experience replay
│   ├── rewards.py     # Reward function
│   ├── training.py    # Training loop
│   └── __init__.py
├── utils/             # Utilities
│   ├── config.py      # Configuration management
│   ├── logger.py      # Training logger
│   ├── checkpoint_manager.py  # Model checkpointing
│   ├── metrics_exporter.py    # Export to CSV/JSON
│   └── __init__.py
├── visualization/     # Visualization components
│   ├── game_renderer.py       # Live gameplay display
│   ├── metrics_dashboard.py   # Training metrics charts
│   └── __init__.py
├── scripts/           # User-facing scripts
│   ├── watch_agent.py  # Watch AI play
│   ├── play_game.py    # Play yourself
│   ├── play_with_hints.py  # Play with AI hints
│   ├── boost_epsilon.py # Utility to boost exploration
│   └── __init__.py
├── tests/             # Unit tests
├── checkpoints/       # Saved models (created during training)
├── logs/              # Training logs (created during training)
└── requirements.txt   # Python dependencies

🎯 How It Works

Game Engine

The 2048 game is implemented using numpy arrays for efficient operations:

• 4x4 grid representation
• Four directional moves (up, down, left, right)
• Tile merging with standard 2048 rules
• Random tile spawning (90% chance of 2, 10% chance of 4)

AI Agent

The agent uses Deep Q-Learning:

• State: Flattened 4x4 board with log2 normalization
• Actions: 4 possible moves (up, down, left, right)
• Rewards: Score increase + milestone bonuses + strategic bonuses
• Network: Fully connected layers (16 → 256 → 256 → 128 → 4)
• Training: Experience replay + target network for stability

Reward Function

• Merge reward: +tile_value for each merge
• Milestone bonuses: +500 for 512, +1000 for 1024, +2000 for 2048, etc.
• Empty cell bonus: +2 per empty cell
• Invalid move penalty: -10
• Game over penalty: -100

📊 Training Results

After ~2-4 hours of training (5000+ episodes):

• Average score: 3000-5000
• Success rate (2048 tile): 30%+
• Best tile achieved: 4096+

🔧 Configuration

Copy config.example.json to config.json and adjust parameters:

{
  "learning_rate": 0.001,
  "gamma": 0.99,
  "epsilon_start": 1.0,
  "epsilon_end": 0.01,
  "epsilon_decay": 0.995,
  "buffer_size": 10000,
  "batch_size": 64,
  "max_episodes": 5000
}

See docs/CONFIGURATION.md for detailed parameter explanations.

📚 Documentation

• Setup Guide - Installation and setup instructions
• Usage Guide - How to train and use the AI
• Configuration Guide - Hyperparameter tuning

🧪 Testing

Run all tests:

python3 -m unittest discover tests -v

Run specific test file:

python3 -m unittest tests.test_board -v

📈 Export Metrics

Export training metrics for analysis:

python3 -m utils.metrics_exporter --metrics logs/metrics.csv --output-dir exports

🤝 Contributing

This is a personal learning project, but suggestions and feedback are welcome!

📝 License

MIT License - feel free to use and modify

🎓 Learning Resources

• Deep Q-Learning Paper
• 2048 Game
• Reinforcement Learning Intro

---

Made with 🧠 and ☕ by a tired but determined student