The Teeworlds Machine Learning Project's mission is to teach an AI to play the deathmatch mode of the 2D platformer/shooter Teeworlds.

This repository contains a modified version of the Teeworlds game server in the teeworlds-tmlp submodule and an implementation of the A2C Reinforcement Learning algorithm in a2c.py which makes use of the game servers.

The model did NOT achieve notable performance. It started taking completely probabilistic actions and learned to prevent killing itself which could happen by shooting at the ground. It did not learn to fight the opponent.

There could be many reasons for this:

• I had some specific questions concerning the state of the art implementation of the main training loop in A2C in combination with a state-based RNN which I didn't have the time to find answers to. So I just implemented it in the way which I thought made sense but one of those small details might be the critical factor.
• I did little to no hyperparameter tuning or grid search.
• The way the input and action spaces are presented to the network might be suboptimal.
• There might be a bug.
• Reinforcement learning might not work for this problem.

I will gladly cooperate with you if you want to work on this project. Concerning an A2C/PPO/A3C implementation the bulk of the code is already there especially on the C++ side. An expert in reinforcement learning would be of most value to the project. The main training loop would have to be reconsidered, hyperparameters would have to be carefully tuned and a lot of testing would be required to rule out bugs.

Building from source

Pull the teeworlds-tmlp submodule.

git submodule init
git submodule update

The software was only tested on Linux. Make sure you have all the dependencies from the building instructions of the vanilla Teeworlds server.

Building TensorFlow C++ API

Also take a look at the official building instructions. Clone the TensorFlow git repository and install Bazel. For Arch Linux there is the bazel package.

Cd into the the repository and checkout a TensorFlow 1.x version. The software was developed and tested with v1.14.0.

./configure
bazel build --config=opt //tensorflow:libtensorflow_cc.so

Building the game server

Run the copy-tensorflow-headers script with the path to your TensorFlow repository which contains the complete build and the teeworlds-tmlp submodule directory. For example:

./copy-tensorflow-headers ../tensorflow/ teeworlds-tmlp/

Create a build folder in the teeworlds-tmlp directory and cd into it.

cmake -DCMAKE_BUILD_TYPE=Release -DCLIENT=OFF ..
make teeworlds_srv