Machine Learning for Flappy Bird using Neural Network and Genetic Algorithm
Here is the source code for a HTML5 project that implements a machine learning algorithm in the Flappy Bird video game using neural networks and a genetic algorithm. The program teaches a little bird how to flap optimally in order to fly safely through barriers as long as possible.
The complete tutorial with much more details and demo you can find here:
Here you can also watch a short video with a simple presentation of the algorithm:
Neural Network Architecture
To play the game, each unit (bird) has its own neural network consisted of the next 3 layers:
an input layer with 2 neurons presenting what a bird sees:
1) horizontal distance between the bird and the closest gap 2) height difference between the bird and the closest gap
a hidden layer with 6 neurons
an output layer with 1 neuron used to provide an action as follows:
if output > 0.5 then flap else do nothing
There is used Synaptic Neural Network library to implement entire artificial neural network instead of making a new one from the scratch.
The Main Concept of Machine Learning
The main concept of machine learning implemented in this program is based on the neuro-evolution form. It uses evolutionary algorithms such as a genetic algorithm to train artificial neural networks. Here are the main steps:
create a new population of 10 units (birds) with a random neural network
let all units play the game simultaneously by using their own neural networks
for each unit calculate its fitness function to measure its quality as:
fitness = total travelled distance - distance to the closest gap
when all units are killed, evaluate the current population to the next one using genetic algorithm operators (selection, crossover and mutation) as follows:
1. sort the units of the current population in decreasing order by their fitness ranking 2. select the top 4 units and mark them as the winners of the current population 3. the 4 winners are directly passed on to the next population 4. to fill the rest of the next population, create 6 offsprings as follows: - 1 offspring is made by a crossover of two best winners - 3 offsprings are made by a crossover of two random winners - 2 offsprings are direct copy of two random winners 5. to add some variations, apply random mutations on each offspring.
go back to the step 2
The entire game logic is implemented in gameplay.js file. It consists of the following classes:
App.Main, the main routine with the following essential functions:
- preload() to preload all assets
- create() to create all objects and initialize a new genetic algorithm object
- update() to run the main loop in which the Flappy Bird game is played by using AI neural networks and the population is evolved by using genetic algorithm
- drawStatus() to display information of all units
TreeGroup Class, extended Phaser Group class to represent a moving barrier. This group contains a top and a bottom Tree sprite.
Tree Class, extended Phaser Sprite class to represent a Tree sprite.
Bird Class, extended Phaser Sprite class to represent a Bird sprite.
Text Class, extended Phaser BitmapText class used for drawing text.
The genetic algorithm is implemented in genetic.js file which consists of the following class:
GeneticAlgorithm Class, the main class to handle all genetic algorithm operations. It needs two parameters: max_units to set a total number of units in population and top_units to set a number of top units (winners) used for evolving population. Here are its essential functions:
- reset() to reset genetic algorithm parameters
- createPopulation() to create a new population
- activateBrain() to activate the AI neural network of an unit and get its output action according to the inputs
- evolvePopulation() to evolve the population by using genetic operators (selection, crossover and mutations)
- selection() to select the best units from the current population
- crossOver() to perform a single point crossover between two parents
- mutation() to perform random mutations on an offspring