super-ml-pets

Framework for training and deploying AIs for Super Auto Pets

Train AIs for Super Auto Pets through a simulated environment and test the trained model against real opponents in the actual game! AI is trained using reinforcement learning and a machine vision system is used to capture the screen to give information to the AI.

Introduction

Framework supports Python 3.7-3.11 and works cross-platform (Ubuntu, Windows, macOS). Deployment is also compatible with the web app.

Training has also been tested with GitHub Codespaces and Google Colab. A demonstration of model training can be seen in this gist.

We recommend using Windows for deployment as the UNIX-based systems require root permissions to launch the program out-of-the-box.

Getting started

1. Clone the repo:

git clone https://github.com/andreped/super-ml-pets.git

2. Setup virtual environment:

cd super-ml-pets/
virtualenv -ppython3 venv --clear
./venv/Scripts/activate

To activate the virtual environment on UNIX-based systems, instead of the last line run source venv/bin/activate

3. Install requirements:

pip install -r requirements.txt

4. Download all pets, food, and misc icons

wget https://github.com/andreped/super-ml-pets/releases/download/pets-01-2024/pets.zip -O pets.zip; Expand-Archive pets.zip -DestinationPath ./; Remove-Item pets.zip
wget https://github.com/andreped/super-ml-pets/releases/download/food-01-2024/food.zip -O food.zip; Expand-Archive food.zip -DestinationPath ./; Remove-Item food.zip
wget https://github.com/andreped/super-ml-pets/releases/download/misc-01-2024/misc.zip -O misc.zip; Expand-Archive misc.zip -DestinationPath ./; Remove-Item misc.zip

Additional setup for Ubuntu only

sudo apt install python3-tk
sudo su
source venv/bin/activate
xhost +
export DISPLAY=:0.0

Note that the command sudo su enables administrator rights. This seems to be required by keyboard as mentioned in issue #23. The xhost + DISPLAY stuff is needed as the screen might not be found, hence, initializing one solves this issue.

Usage

This framework currently supports training and deploying RL models for SAP.

Training

For training in simulated environment, using default arguments, simply run:

python main.py --task train

Given an existing model, it is also possible to finetune it by (with example):

python main.py --task train --finetune /path/to/model_sap_gym_sb3_180822_checkpoint_2175_steps

The script supports other arguments. To see what is possible, run:

python main.py --help

Testing

1. To use a trained model in battle, start the game Super Auto Pets.

2. Ensure that the game is in full screen mode, disable all unneccessary prompts, enable auto name picker, and set speed to 200% (you might also have to enable auto battle which can only be done in the first battle - if this is the first time you are playing this game).

3. Change the UI style to classic for all options in customize including "Food art", "Background art", "Menu background", "Buff style", and "Held food".

4. Change UI style for pets to classic by going to the pets settings.

5. Enter the arena by clicking Arena mode.

6. Go outside the game and download a pretrained model from here, or use any pretrained model you might have. For simplicity, you can also run the following to download a example model:

wget https://github.com/andreped/super-ml-pets/releases/download/v0.0.6/model_sap_gym_sb3_280822_finetuned_641057_steps.zip

7. Then, simply start the AI by running this command from the terminal (with example path to pretrained model, without extension .zip):

python main.py --task deploy --infer_model /path/to/model_sap_gym_sb3_280822_finetuned_641057_steps

8. Go back into the game and press the Space keyboard button (when you are in the Arena (in team preparation, before battle).

It might take a few seconds, but you should now be able to see the AI start playing. Please, let it play in peace, or else it might get angry and you may have accidentally created Skynet. If you accidentally exit the game, or dont have the game in fullscreen, the machine vision system will fail, and you will have to start a completely new game to use the AI (properly).

Training history

To plot training history, run:

python smp/plot_history.py --log /path/to/history/history_rl_model/progress.csv

Troubleshoot

To install virtualenv, run:

pip install virtualenv

If you do not have virtualenv in the path, you can access it by:

python -m virtualenv -ppython3 venv --clear

To activate virtual environment on UNIX-based systems (e.g., macOS or Ubuntu), run:

source venv/bin/activate

If you are using newer versions of Python (e.g., 3.10), you might have issues with installing and/or using numpy with the other dependencies. If that happens, try downgrading numpy by:

pip install numpy==1.23.2 --force-reinstall

On both Ubuntu and macOS, it might require sudo permissions to run deployment. This has to do with keyboard events not being able to be recognized without sudo rights. On Windows, administrative rights is not needed. For more information, see here.

On macOS, when you are downloading the models (.zip files) from Releases, they might be unzipped automatically. This is bad as the model extension is .zip. To fix this, disable the Open safe files after downloading in the Safari Preferences (see here for more information).

If deployment fails to start (no mouse movements or events), it may be because your screen resolution differ from the expected resolution. The current machine vision system expects the screen resolution to be 1920x1080. Please, adjust the resolution to this. This will be fixed in the future.

Acknowledgements

This implementation is based on multiple different projects. The core implementation is derived from GoldExplosion, which further was based upon the super auto pets engine sapai and RL training through sapai-gym.

All credit to jpdefrutos for designing the amazing header figure.

Citation

If you found this project relevant for your research, please, cite the following:

@software{andre_pedersen_2023_7834142,
  author       = {André Pedersen and Javier Pérez de Frutos and laughinggaschambers and GoldExplosion},
  title        = {andreped/super-ml-pets: v0.0.9},
  month        = apr,
  year         = 2023,
  publisher    = {Zenodo},
  version      = {v0.0.9},
  doi          = {10.5281/zenodo.7834142},
  url          = {https://doi.org/10.5281/zenodo.7834142}
}