The Teeworlds Machine Learning Project's mission is to teach an AI to play the deathmatch mode of the 2D platformer/shooter Teeworlds.
This repository contains a modified version of the Teeworlds game server in the
teeworlds-tmlp submodule and an implementation of the A2C Reinforcement Learning algorithm in
a2c.py which makes use of the game servers.
The model did NOT achieve notable performance. It started taking completely probabilistic actions and learned to prevent killing itself which could happen by shooting at the ground. It did not learn to fight the opponent.
There could be many reasons for this:
- I had some specific questions concerning the state of the art implementation of the main training loop in A2C in combination with a state-based RNN which I didn't have the time to find answers to. So I just implemented it in the way which I thought made sense but one of those small details might be the critical factor.
- I did little to no hyperparameter tuning or grid search.
- The way the input and action spaces are presented to the network might be suboptimal.
- There might be a bug.
- Reinforcement learning might not work for this problem.
I will gladly cooperate with you if you want to work on this project. Concerning an A2C/PPO/A3C implementation the bulk of the code is already there especially on the C++ side. An expert in reinforcement learning would be of most value to the project. The main training loop would have to be reconsidered, hyperparameters would have to be carefully tuned and a lot of testing would be required to rule out bugs.
Building from source
git submodule init git submodule update
The software was only tested on Linux. Make sure you have all the dependencies from the building instructions of the vanilla Teeworlds server.
Building TensorFlow C++ API
Cd into the the repository and checkout a TensorFlow 1.x version. The software was developed and tested with
./configure bazel build --config=opt //tensorflow:libtensorflow_cc.so
Building the game server
copy-tensorflow-headers script with the path to your TensorFlow repository which contains the complete build and the
teeworlds-tmlp submodule directory. For example:
./copy-tensorflow-headers ../tensorflow/ teeworlds-tmlp/
build folder in the
teeworlds-tmlp directory and cd into it.
cmake -DCMAKE_BUILD_TYPE=Release -DCLIENT=OFF .. make teeworlds_srv