NeuroEvolution

Introduction

Neuroevolution is a method for modifying neural network weights, topologies, or ensembles in order to learn a specific task. Evolutionary computation is used to search for network parameters that maximize a fitness function that measures performance in the task. Compared to other neural network learning methods, neuroevolution is highly general, allowing learning without explicit targets, with nondifferentiable activation functions, and with recurrent networks. It can also be combined with standard neural network learning to e.g. model biological adaptation. Neuroevolution can also be seen as a policy search method for reinforcement-learning problems, where it is well suited to continuous domains and to domains where the state is only partially observable.

In OpenNERO, neuroevolution is demonstrated in the Roomba environment and NERO Game environment. The particular neuroevolution method is rtNEAT, i.e. an evolutionary algorithm that adapts the weights and the topologies of neural network controllers. With a little bit of work (left as an exercise), rtNEAT can also be used to solve (instead of Q-learning) the Maze Running task, and perhaps even the Towers of Hanoi task.

Running the Roomba Demo

To run the demo, first:

1. Start OpenNERO
2. Start the Neuroevolution experiment
3. Select the type of bots you want to experiment with (Scripted or rtNEAT)
4. Set the number of agents you want
5. Click the "Add Robots" button to start the demo
6. Click "Pause" and then "Exit" when you want to stop the current bots

Scripted Bots

In this environment, one simple strategy is a greedy vacuum cleaner that detects and drives to the nearest piece of litter. The scripted robots use a hand-coded implementation of this strategy.

The bots have the following 4 sensors:

1. X coordinate of its own position
2. Y coordinate of its own position
3. X coordinate of the closest crumb
4. Y coordinate of the closest crumb

rtNEAT Bots

The second type of robots are the rtNEAT agents evolved by neuroevolution. They have the following sensors:

1. Angle of the closest crumb (between -180 and +180)
2. Fraction of all room crumbs that are in agent's detection radius (1/8 of the room's width)