Multiagent reinforcement learning simulation framework - Undergraduate thesis in Mechatronics Engineering at the University of Brasília
Switch branches/tags
Nothing to show
Clone or download
matheusportela Use deep copy in learning algorithms
The learning algorithms, such as Q-learning and Q-learning with function approximation, had their previous state referring to the same object as the current state due to some Python nuances in its object model. Creating deep copies seems to quickly fix this bug.
Latest commit fea599d Feb 6, 2017



This repository contains the code used in the undergraduate thesis in Mechatronics Engineering, at the University of Brasilia, entitled "Behavior selection for multiple autonomous agents with reinforcement learning in stochastic environments".

The idea is to have multiple simulated robotic agents learning to select appropriate behaviors in a stochastic environment. The uncertainty of a state is handled through Bayesian Programming, and the agents learn by applying Q-learning with function approximation.

Currently, the approach is tested in a predator-prey problem using a modified version of the Pac-Man game with introduced uncertainties. Therefore, this simplified multi-agent situation aims to answer the following question: can the ghosts learn to get the Pac-Man?


The Pac-Man AI Projects provides six Pac-Man-like simulators that are free to use for educational purposes. The one we will be using is Project 5: Classification, which provides an arena mimicking the complete Pac-Man game, including various ghosts.

This project requires the following Python packages:


This assumes a GNU/Unix distribution (Ubuntu), but everything is in Pỳthon so the setup shouldn't be too different for other platforms.

Install by running the following commands.

sudo apt-get install python python-dev python-pip python-tk libzmq-dev python-matplotlib
sudo pip install pyzmq

Mac OS X

Installing on an OS X distribution requires a special setup using Homebrew and the XCode.

XCode shall be downloaded from the provided link or in your App Store. Then you need to run the following commands.

ruby -e "$(curl -fsSL"
brew install python --with-tcl-tk --enable-threads --with-x11
pip install matplotlib bumpy pyzmq


All documentation are kept in our wiki, with usage examples and modules explanations.