This is my course project for CMU-10703.
The code is implemented in Tensorflow and Keras.
Full detail can be found in this report.
This Tensorflow(Keras) implementation of Deep Reinforcement Learning contains:
[1] Playing Atari with Deep Reinforcement Learning
[2] Deep Reinforcement Learning with Double Q-learning
[3] Dueling Network Architectures for Deep Reinforcement Learning
[4] Deep Recurrent Q-Learning for Partially Observable MDPs
[5] DQN with Spatial Attention
[6] DQN with Temporal Attention
The code is based on Python 3. Install dependency by running:
pip install --user -r requirements.txt
There are two types of DQN implementation with gpu: Keras and Tensorflow.
You can choose different implementation by altering line 15 in
dqn_atari.py
Train original DQN:
python dqn_atari.py --task_name 'DQN'
Train Double DQN:
python dqn_atari.py --ddqn --task_name 'Double_DQN'
Train Dueling DQN:
python dqn_atari.py --net_mode=duel --task_name 'Dueling_DQN'
Train Recurrent DQN:
python dqn_atari.py --num_frames 10 --recurrent --task_name 'Recurrent_DQN'
Train Recurrent Temporal Attention DQN: (Using dqn_tf_temporalAt.py by uncommenting line 18 in dqn_atari.py)
python dqn_atari.py --num_frames 10 --recurrent --a_t --selector --task_name 'TemporalAt_DQN'
Train Recurrent Spatial Attention DQN: (Using dqn_tf_spatialAt.py by uncommenting line 21 in dqn_atari.py)
python dqn_atari.py --num_frames 10 --recurrent --a_t --selector --task_name 'SpatialAt_DQN'
Test trained model (e.g. Spatial Attention DQN):
python dqn_atari.py --num_frames 10 --recurrent --a_t --selector --test \
--load_network --load_network_path=PATH_TO_NET
Results of Seaquest-v0 for DQN (Blue), DRQN (yellow), DRQN-TemporalAttention (brown), DRQN-SpatialAttention (purple):
Analysis of Recurrent Temporal Attention DQN:
The number at the corner is the weight of this frame.
Analysis of Recurrent Spatial Attention DQN:
The attention focuses on the enemy before the action.
This project is a joint work with Rui Zhu, Bo Chen and Hanyue Liang.