AI invaders is an AI that plays the game space invaders without human interaction while achieving better than average human performance
AI invaders uses a reinforcement learning algorithm known as Double Deep Q Network (DDQN) with Prioritized Replay an improvement over the Vanilla DQN architecture.
It’s the expected sum of discounted rewards for taking an action in a given state, for a given policy. As given below:
You can find a short video of the trained agent in the folder submitted. – Trained Agent Gameplay.mp4
There are two ways to run the application
- Training mode
- Play/Test mode
- Human mode
For both, start with the following steps
git clone https://github.com/updatesvc/AI-Invaders.git- Requires python with jupyter installed
- Required libraries to install tensorflow, keras, pygame, gym
Open the “Agent training notebook” in jupyter and run it. It will periodically save the model’s weights in a file
Open the “Notebook for running Agent” in jupyter and run it.
To stop the bot, click the stop button of the notebook – due to pygame limitations you can not stop using the X button of the game when testing the trained agent, and to exit completely restart the python kernel or alternatively stop it by exiting jupyter kernel
To play the game manually (as a human player)
- Open a terminal in the code directory
- Run python play.py
The trained.h5 contains trained model weights - trained for about 100hrs on colab K80. Reward modelling was difficult, but based on my research the optimal reward model was to limit the amount of bullets, and give a score based on ammo left. Reinforcing accurate and efficient shooting.
The repo also includes game.py , which is a custom implementation of space invaders that allows for reward modelling and fast environment sampling during training.
Sum Tree from : Jaromír Janisch In depth details on Double DQNs and Prioritised Replay : Jaromír Janisch Blog
Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.
it started with aliens, then ...