Informed Search/Reinforcement Learning Agents for Flappy Bird
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information.
FB White Paper.pdf

Flaippy Bird 101

Flappy Bird 101


  • Flappy Bird is a wildly successful game released for iPhone in 2014. This project examines AI solutions to the game along the lines of reinforcement learning and informed search. For a detailed description of the project, read our white paper in this repository.

Getting started

  • Make sure you've got PyGame 1.9+ installed and are running Python 2.7+. You may also need to update your PC's package of libpng.
  • For a demo of the TD-lambda learner on pre-trained weights, run python -d.
  • Note that training can take a while (esp. as the agent gets better and each episodes lasts longer). Our best performing agent was trained for well over 6 hours.

RL Statespace

  • 10 by 10 grid of 10 by 10 units immediately in front of the next gap.
  • More than 50 above or below the gap, the y-discretization increases to 100.
  • 100 horizontal units before the next gap the x-discretization increases to 100.
  • Vertical velocity [-10, -9, ... , 9, 10].


  • Clean up command line args and general IO.
  • shouldn't exist.
  • Make not totally awful.
  • Consolidate and clean up code. A lot is split between multiple files.
  • Those global variables and constants though...
  • The Q Learner commits suicide when prospects look grim. Work on getting it to be risky and aim for the gap even when failure is certain.