Skip to content

Implementations for the Multi-Agent AI 'Pommermann' challenge @NIPS2018

License

Notifications You must be signed in to change notification settings

timlacroix/nips2018-agent

 
 

Repository files navigation

Multi-Agent Reinforcement Learning @ NIPS 2018

This repository is an open-source implementation of a Multi-Agent Reinforcement Learning Agent that will participate in the NIPS 2018 Pommerman challenge. For a full list of availble competitions this year at the NIPS conference, see the competition track.

This repository also provides a re-implementation for the environment. This is how the C++ backend currently looks like:

gif of the game

Prerequisites

To compile and run this project from source you will require

  • Linux Distribution (Tested on Ubuntu 18.04)
  • GCC 7.3.0
  • MAKE 4.1
  • CUDA 9 (not yet necessary, will be updated)

Setup

Download

This project uses the playground environment as a submodule. To fully clone the repository use

# git version 2.13+
$ git clone --recurse-submodules https://github.com/m2q/nips2018-agent.git

# git version 2.12 or less
$ git clone --recursive https://github.com/m2q/nips2018-agent.git

Compilation

Instead of using the shell scripts you can obviously use make commands and call/debug the binaries yourself. Here is a list:

Command What it does
make or make all Compiles and links both test and main source files
make main Compiles the main source to ./bin/exec
make test Compiles the test source to ./bin/test
make clean Removes ./bin and ./build
make mclean Removes ./bin/exec and ./build/src only

Tip: The makefile makes use of the MAKEFLAGS environment variable. Let's say you want to have -j n as the default job count, where n is the number of cores available on your system. Then just export an env variable like this

$ export MAKEFLAGS="-j $(grep -c ^processor /proc/cpuinfo)"

(or alternatively add it to your ${HOME}.profile)

Project Structure

All of the main source code is in src/* and all testing code is in unit_test/*. The source is divided into modules

src
 |
 |_ _ _ bboard
 |        |_ _ _ bboard.hpp
 |        |_ _ _ ..
 |
 |_ _ _ agents
 |        |_ _ _ agents.hpp
 |        |_ _ _ ..
 |
 |_ _ _ main.cpp

All environment specific functions (forward, board init, board masking etc) reside in bboard. Agents can be declared in the agents header and implemented in the same module.

All test cases will be in the module unit_test. The bboard should be tested thoroughly so it exactly matches the specified behaviour of Pommerman. The compiled test binary can be found in /bin

Testing

Want to test out how many steps can be simulated on your machine in 100ms?

# example of 4 threads on an Intel i5 (Skylake/4 cores)
$ ./performance.sh -t 4 

Activated Multi-Threading for performance tests. 
	Thread count:            4
	Max supported threads:   4

Test Results:
Iteration count (100ms):         586.332
Tested with:                     agents::HarmlessAgent

===============================================================================
All tests passed (1 assertion in 1 test case)

You can also directly run the test-binaries. For a list of command line arguments see the Catch2 CLI docs (or run ./test --help). Here are some typical examples I use a lot:

Command What it does
./test Runs all tests, including a performance report
./test "[step function]" Tests only the step function
./test ~"[performance]" Runs all test except the performance cases

Defining Agents

To create a new agent you can use the base struct defined in bboard.hpp. To add your own agent, declare it in agents/agents.hppand provide a source file in the same module. For example:

agents.hpp (excerpt)

/**
 * @brief Uses a hand-crafted FSM with stochastic noise
 */
struct MyNewAgent : bboard::Agent
{
    bboard::Move act(bboard::State* state) override;
};

fsm_agent.cpp

#include "bboard.hpp"
#include "agents.hpp"

namespace agents
{

bboard::Move MyNewAgent::act(bboard::State* state)
{
    // TODO: Implement your logic
    return bboard::Move::IDLE;
}

}

Citing This Repo

@misc{Alic2018nips,
  author = {Alic, Adrian},
  title = {Distributed Async MARL Competition Entry},
  year = {2018},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/m2q/nips2018-agent}}
}

About

Implementations for the Multi-Agent AI 'Pommermann' challenge @NIPS2018

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • C++ 99.5%
  • Other 0.5%