A brave new world

It's possible to use Multiagent RL with different simulators by creating a new package inside the examples/ directory. It must have the following modules:

• adapter.py: Class inheriting from BaseExperiment, implementing simulation-specific logic to connect the simulation and Multiagent RL.
• agents.py: Agents that will be instantiated by the simulation controller when running this simulation.
• plot.py: Logic to plot the simulation results after running this simulation.

Adapter

The adapter module contains an adapter class, inheriting from BaseExperiment, which controls the flow of the simulation. It must also contain the build_parser() function, to parse command-line arguments, and build_adapter_with_args(args), to build an adapter instance from the parsed arguments.

from multiagentrl import core
from multiagentrl import messages


class ExampleExperiment(core.BaseExperiment):
    def __init__(self, learn_games, test_games):
        super(ExampleExperiment, self).__init__(
            learn_games=learn_games,
            test_games=test_games)

    def execute_game(self):
        # Implement here the logic to run a simulation step


def build_parser():
    parser = argparse.ArgumentParser(description='Run example simulation.')
    parser.add_argument(
        '-l', '--learn-games', dest='learn_games', type=int, default=1,
        help='number of games to learn from')
    parser.add_argument(
        '-t', '--test-games', dest='test_games', type=int, default=1,
        help='number of games to test learned policy')
    return parser


def build_adapter_with_args(args):
    return ExampleExperiment(
        learn_games=args.learn_games,
        test_games=args.test_games)

Agents

The agents module contains agent classes to be used for action selection when the simulation is running. They must inherit from BaseControllerAgent and implement its virtual methods, namely start_game, stop_game, learn and act.

The following agent selects random actions for every simulation step.

import random

from multiagentrl import core


class RandomAgent(core.BaseControllerAgent):
    """Agent that randomly selects an action."""
    def __init__(self, agent_id, ally_ids, enemy_ids):
        super(RandomAgent, self).__init__(agent_id)

    def start_game(self):
        pass

    def finish_game(self):
        pass

    def learn(self, state, action, reward):
        pass

    def act(self, state, legal_actions, explore):
        if legal_actions:
            return random.choice(legal_actions)

Plot

The plot module generate graphs from simulation results. It must contain the build_parser() function, to parse command-line arguments, and plot(args), to plot graphs from the parsed arguments.