Top down racing simulator that trains AI agents.
Originally, I started this project for my Interactive Entertainment Engineering class. It was just a top down car game that could be played online with friends. Sadly no one really wanted to play it with me. It was fun to mess around with, but I needed some competition. So I set out to make a bot that could beat me at my own game.
First, there needs to be a way for an AI model to interface with a car. So, each car is given 12 distance sensors as well as a speed sensor. Based off the sensor input, the AI model can control the car with these outputs:
- forward
- backward
- left
- right
But how will the AI know what to do? The AI will use a neural network(NN) to control the car. NN's aren't smart, they need to be trained in order to start returning reasonable outputs for each set of inputs. One of the simplest methods of training a NN, when there's no "right" output for a set of inputs, is a genetic algorithm.
The genetic algorithm is based off of the basic principles of evolution. Those that are fit survive and reproduce, sharing their genes with the next generation. In this case the unfit NN's are forgotten while the fit NN's are used to create the next generation of NN's.
The genes of the NN are its weights(the architecture remains constant). Each NN is generated by randomly drawing weight values from each parent with ~50/50 split. Then these weights are randomly mutated by adding or subtracting small values from each.
Generations are seperated into batchs and each has a set population size. For each batch the NN's are tested, with the top half being permitted to reproduce. In order to accelerate evolution, no NN can reproduce with another less fit NN.
WORKINPROGRESS
(this might be a bad thing?)
(how to determine fitness?)
Fitness function blah blah
Took a while, led to overfit, no left turns Changed up starting weights to cast wider search net, random starting directions (so they have to turn right and left!) Here is how it went. Throw in some charts
Checkout the branches with Qlearning and even a DRQN