Meetup Barranquilla: Reinforcement learning
Deep Q-learning agent for replicating DeepMind's results in paper "Human-level control through deep reinforcement learning"
This project follows the description of the Deep Q Learning algorithm described in Playing Atari with Deep Reinforcement Learning  and shows that this learning algorithm can be further generalized to the notorious Flappy Bird, Pong and Space Invaders.
- Install Tensorflow:
pip install tensorflowIf you have NVIDIA GPU, you should do:
pip install tensorflow-gpu
- Install pygame:
pip install pygame
- Install opencv:
pip install opencv-python
- Install all gym dependencies:
pip install gym'[all]'
pip install -r requirements.txt
What is Deep Q-Network?
It is a convolutional neural network, trained with a variant of Q-learning, whose input is raw pixels and whose output is a value function estimating future rewards. For those who are interested in deep reinforcement learning, I highly recommend to read the following post: Demystifying Deep Reinforcement Learning
According to , I first preprocessed the game screens with following steps:
- Convert image to grayscale
- Resize image to 80x80
- Stack last 4 frames to produce an 80x80x4 input array for network
- First layer convolves the input image with an 8x8x4x32 kernel at a stride size of 4.
- The output is then put through a 2x2 max pooling layer.
- The second layer convolves with a 4x4x32x64 kernel at a stride of 2.
- The third layer convolves with a 3x3x64x64 kernel at a stride of 1.
- FullyConnected 1600x512
- Readout: FullyConnected 512x2
** Training ** For training, you should do the following:
cd FlappyBird python deep_q_network.py
There are examples developed in OpenAI gym for the pong game and SpaceInvaders. Just:
cd atari python pong.py python SpaceInvaders.py
 Mnih Volodymyr, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, Shane Legg, and Demis Hassabis. Human-level Control through Deep Reinforcement Learning. Nature, 529-33, 2015.
 Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing Atari with Deep Reinforcement Learning. NIPS, Deep Learning workshop
This work is highly based on the following repos: