DQN in Keras + TensorFlow + OpenAI Gym

This is an implementation of DQN (based on Mnih et al., 2015) in Keras + TensorFlow + OpenAI Gym.

Requirements

• gym (Atari environment)
• scikit-image
• keras
• tensorflow

Results

This is the result of training of DQN for about 28 hours (12K episodes, 4.7 millions frames) on AWS EC2 g2.2xlarge instance.



Statistics of average loss, average max q value, duration, and total reward / episode.

Usage

Training

For DQN, run:

python dqn.py

For Double DQN, run:

python ddqn.py

Visualizing learning with TensorBoard

Run the following:

tensorboard --logdir=summary/

Using GPU

I built an AMI for this experiment. All of requirements + CUDA + cuDNN are pre-installed in the AMI.
The AMI name is DQN-AMI, the ID is ami-c4a969a9, and the region is N. Virginia. Feel free to use it.

ToDo

• RMSPropGraves
• Dueling Network

References

• Mnih et al., 2013, Playing atari with deep reinforcement learning
• Mnih et al., 2015, Human-level control through deep reinforcement learning
• van Hasselt et al., 2016, Deep Reinforcement Learning with Double Q-learning
• devsisters/DQN-tensorflow
• spragunr/deep_q_rl