Train RL Agents to Play Pokemon Red

Train reinforcement learning agents to play Pokemon Red using PyBoy and Stable Baselines 3. The V2 version uses a coordinate based exploration reward instead of frame KNN, trains faster, uses less memory, and is able to reach Cerulean City.

---

Running the Pretrained Model Interactively

Python 3.10+ is recommended. You also need to install ffmpeg and have it available in the command line.

Linux / MacOS

V2 is now recommended over the original version. You may follow all steps below but replace baselines with v2.

1. Copy your legally obtained Pokemon Red ROM into the base directory. Rename it to PokemonRed.gb if it is not already. The sha1 sum should be ea9bcae617fdf159b045185467ae58b2e4a48b9a, which you can verify by running shasum PokemonRed.gb.
2. Move into the baselines/ directory:

   cd baselines
   

3. Install dependencies:

   pip install -r requirements.txt
   

4. Run:

   python run_pretrained_interactive.py
   

Interact with the emulator using the arrow keys and the a and s keys (A and B buttons). You can pause the AI's input during the game by editing agent_enabled.txt.

Note: PokemonRed.gb must be in the main directory and your current directory must be baselines/.

---

Training the Model

V2

• Trains faster and with less memory
• Reaches Cerulean City
• Streams to map by default
• Replaces the frame KNN with a coordinate based exploration reward

1. Previous steps but in the v2 directory instead of baselines
2. Run:

   python baseline_fast_v2.py
   

---

Tracking Training Progress

Training Broadcast

Stream your training session to a shared global game map using the Broadcast Wrapper on your environment:

env = StreamWrapper(
    env,
    stream_metadata = {
        "user": "your-username",
        "env_id": id,
        "color": "#0033ff",
        "extra": "",
    }
)

Local Metrics

The current state of each game is rendered to images in the session directory. Track progress in tensorboard by moving into the session directory and running:

tensorboard --logdir .

Navigate to localhost:6006 in your browser to view metrics. To enable wandb integration, change use_wandb_logging in the training script to True.

---

Static Visualization

Map visualization code can be found in the visualization/ directory.