Skip to content
Teeworlds Machine Learning Project - A2C Python Implementation
Python Shell
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
teeworlds-tmlp @ f2c31b6
.gitignore
.gitmodules
README.md
TODO
a2c.py
autoexec.cfg
ckpt_to_savedmodel.py
copy-tensorflow-headers
inspect-model
old_nn.py
render_video.py
storage.cfg

README.md

The Teeworlds Machine Learning Project's mission is to teach an AI to play the deathmatch mode of the 2D platformer/shooter Teeworlds.

This repository contains a modified version of the Teeworlds game server in the teeworlds-tmlp submodule and an implementation of the A2C Reinforcement Learning algorithm in a2c.py which makes use of the game servers.

The model did NOT achieve notable performance. It started taking completely probabilistic actions and learned to prevent killing itself which could happen by shooting at the ground. It did not learn to fight the opponent.

There could be many reasons for this:

  • I had some specific questions concerning the state of the art implementation of the main training loop in A2C in combination with a state-based RNN which I didn't have the time to find answers to. So I just implemented it in the way which I thought made sense but one of those small details might be the critical factor.
  • I did little to no hyperparameter tuning or grid search.
  • The way the input and action spaces are presented to the network might be suboptimal.
  • There might be a bug.
  • Reinforcement learning might not work for this problem.

I will gladly cooperate with you if you want to work on this project. Concerning an A2C/PPO/A3C implementation the bulk of the code is already there especially on the C++ side. An expert in reinforcement learning would be of most value to the project. The main training loop would have to be reconsidered, hyperparameters would have to be carefully tuned and a lot of testing would be required to rule out bugs.

Building from source

Pull the teeworlds-tmlp submodule.

git submodule init
git submodule update

The software was only tested on Linux. Make sure you have all the dependencies from the building instructions of the vanilla Teeworlds server.

Building TensorFlow C++ API

Also take a look at the official building instructions. Clone the TensorFlow git repository and install Bazel. For Arch Linux there is the bazel package.

Cd into the the repository and checkout a TensorFlow 1.x version. The software was developed and tested with v1.14.0.

./configure
bazel build --config=opt //tensorflow:libtensorflow_cc.so

Building the game server

Run the copy-tensorflow-headers script with the path to your TensorFlow repository which contains the complete build and the teeworlds-tmlp submodule directory. For example:

./copy-tensorflow-headers ../tensorflow/ teeworlds-tmlp/

Create a build folder in the teeworlds-tmlp directory and cd into it.

cmake -DCMAKE_BUILD_TYPE=Release -DCLIENT=OFF ..
make teeworlds_srv
You can’t perform that action at this time.