Beyond Playing to Win: Agent Behaviour Research (in GVGAI)

Personalised version of the GVGAI framework that allows providing the heuristic to use in the algorithm at the time of starting the game. This modification allows running the agents with different heuristics (behaviours) without having to update their core code. The heuristics are created in a external file and plugged into the agent during its instantiation.

This repository contains the code related to my research in agent behaviour. it covers the following topics:

• Heuristic diversification
• Use of MAP-Elites to generate agents that elicits diverse automated gameplay
• Automated gameplay
• Portability of the team of agents generated and their potential application in testing

Original code: GVGAI competition

The original code of the GVGAI Framework can be found at https://github.com/GAIGResearch/GVGAI. It has been duplicated in this repository and included some modifications to fit the needs of my experiments. Please refer to the original code for anything related to the GVGAI competition.

The code was duplicated on the 13th of January of 2021 so this repository does not include any modification/fix carried out in the main repository after this date.

Behaviour research code

The code related to the behaviour research is contained on its own folder: src/heuristic_diversification/

Heuristic diversification

The interests of my research is widening the use of existing algorithms when these are provided with goals that go beyond winning. The modifications included in this repository allow providing different heuristics to a same algorithm without having to modify its core, which was essential for my experiments.

The modifications are similar to the ones carried out in previous experiments but they have been integrated in the most recent version of GVGAI, cleaned, improved and polished.

Core modifications

1. Disable TIME_CONSTRAINED set for the competittion

2. Create new classes to provide the heuristic and the controller separately when running a game.

   • src/core/heuristic/StateHeuristics.java
   • src/core/player/AbstractHeuristicPlayer.java

3. Create new file to replace ArcadeMachine so the methdos related to the heuristic diversification are in their own file

   • src/heuristic_diversification/helper/ArcadeMachineHeuristic.java

4. Modify sample controllers (OSLA and MCTS) to support heuristic diversification.

   • src/heuristic_diversification/controllers/

5. Folder to contain the files with heuristics and helper classes

   • src/heuristic_diversification/heuristics/

6. Create GameStats class to collect final stats of the gameplay

   • src/heurstic_diversification/model/GameStats.java

7. Create test main to make sure the heuristic diversification works

   • src/heuristic_diversification/testDemo.java

Related paper

Cristina Guerrero-Romero, Annie Louis and Diego Perez-Liebana. "Beyond Playing to Win: Diversifying Heuristics for GVGAI." In 2017 IEEE Conference on Computational Intelligence and Games (CIG), pp. 118-125. IEEE, 2017.

Note: For the code and results obtained in the experiments presented in the paper, refer to the original repository.

List of heuristics (player behaviours) implemented

The heuristic implemented are general within the GVGAI framework. They can be used in any of the games supported by it, as long as the rules are well-formed and aligned with the assumptions made in each of the heuristics (detailed in the paper "MAP-Elites to Generate a Team of Agents that Elicits Diverse Automated Gameplay").

• Winning and Score: Winning the game while maximizing the score difference.
• Exploration: Maximizes the physical exploration of the map prioritizing those positions that haven’t been visited before or have been visited the least number of times.
• Curiosity: Maximizes the discovery and interaction with sprites in the game, prioritizing interactions with new sprites. Interactions are collisions between the avatar (or a sprite created by the avatar) and the elements of the game.
• Killing: Maximizes the destruction of Non-Playable Characters (NPCs).
• Collection: Maximizes the collection of resources available in the game.

A parent heuristic called TeamBehaviourHeuristic has also been implemented. It can be assigned to the agent to set different combinations of the previous behaviours, by assigning a weight to each of them. It's been necessary to include normalisation logic to the heuristics so their results are on a similar scale.

MAP-Elites to generate a team of agents with diverse behaviours

Implementation of the MAP-Elites algorithm to generate a team of agents that play a game eliciting different types of gameplay. Each of the cells of the map contains the description of an agent that implements the TeamBehaviourHeuristic given by a set of weights evolved by a stochastic hill climber mutation.

The candidates are assigned to the map by looking at the stats resulting from playing the game repeatedly. The performance used to replace elites in the map is not related to how well the agent performs in the game in terms of wins or score, as these are considered features, but by how fast the agents end up getting a pair of features in a similar range.

Contents

• src/heuristic_diversification/mapelites Contains MAPElites.java and the necessary classes and helpers to generate the map of elites. The features come from the resulting stats of playing a game, calculated by GameStats when playing the game.

• src/heuristic_diversification/framework/ Contains the classes related to the GVGAI framework so the MAP-Elites algorithm is as transparent to it as possible.

• Mutation: src/heuristic_diversification/mapelites/Generator.java contains the methods that generate random values and mutate the weights given to the agents. We use a stochastic hill climber mutation to get new values for the weights.

• src/heuristic_diversification/model/JSONManager.java Class with static methods to serialize the objects and generate the JSON files containing the results of the experiments (and read from them when necessary).

• src/heuristic_diversification/config/ Contains information about the agents, behaviours and games available for the experiments.

Main

• src/heuristic_diversification/MapElitesGameplay.java Contains the code used in the experiments. Reads the configuration files, sets up and prepares the agent, initializes the MAP-Elites and runs it based on the configuration provided. Creates logs and generates the final JSON with the results.

• src/heuristic_diversification/GenerateMapElitesGameplayConfigTemplate.java Generates a template to create valid configuration files.

• src/heuristic_diversification/mapElitesTest.java Test file used during development. Can be ignored.

Experiments

• The games and levels used in the experiments can be found in their own separate folder: examples/experiments/

The config files, executables and results of the experiments carried out have been uploaded and can be found in an OSF repository. The scripts to process and parse the resulting JSON files to generate the heatmap graphs, record the videos, etc are in another Github repository.

Demo

I have implemented an interactive tool that allows to visualize the gameplays of the agents generated by running the experiments. It allows checking the details and stats of the agents generated in each experiment, playing a pre­-recorded gameplay of each of them and to download the standalone and instructions to running them locally to witness their behaviour first hand.

The link to the live demo is: https://demo-visualize-diverse-gameplay-xqjmp.ondigitalocean.app/

Publication

Cristina Guerrero-Romero and Diego Perez-Liebana. "MAP-Elites to Generate a Team of Agents that Elicits Diverse Automated Gameplay" To be published in 2021 IEEE Conference on Games (CoG)

Related paper

Cristina Guerrero-Romero, Simon M. Lucas, and Diego Perez-Liebana. "Using a Team of General AI Algorithms to Assist Game Design and Testing." In 2018 IEEE Conference on Computational Intelligence and Games (CIG), pp. 1-8. IEEE, 2018

Automated gameplay

We have implemented an executable that allows triggering an automated gameplay of a game of the GVGAI framework by providing a config file with the details of the game, agent and the description of its behaviours. The code related to this standalone, as well as the final folder and downloadable zip can be found in the automated-gameplay-standalone branch.

Generalisation experiments: portability and gameplay testing

We have identified various behaviour-types agents from the teams generated in the experiments and run a test to check their potential portability to new levels, different from those used for their generation. These agents have also been used to test "broken" levels in an exploratory study.

The code for these experiments and related links can be found in the generalisation-experiments-automated-gameplay-stats branch

License

The original code and License of the GVGAI Framework can be found at: https://github.com/GAIGResearch/GVGAI

The original code has been extended to fit the needs of my research. All the modifications and extensions can be found in this repository, created by Cristina Guerrero-Romero in 2021.

Copyright (C) 2021 Cristina Guerrero-Romero

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program. If not, see https://www.gnu.org/licenses/.