h-DQN

Reproduction of "Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation" by Kulkarni et al. (2016) in Python: https://arxiv.org/abs/1604.06057

Disclaimer

This is a work in progress. I haven't been able to replicate the results yet.

Also, I haven't started on Montezuma's revenge yet. I intend to do this eventually, but I'm not sure when. Pull requests are welcomed and encouraged!

Comments/criticisms/suggestions/etc welcome, as always.

Progress

MDP Environment

• Create MDP Environment [Done]
• Create a non-hierarchical actor-critic agent as a baseline [Done]
• Evaluate the non-hierachical actor-critic by plotting which states it visits [Done]
• Create a h-DQN agent [Done]
• Evaluate the h-DQN agent by plotting which states it visits [Done]

Montezuma's Revenge

TODO (This might be a while. Pull requests welcome.)

Results

Stochastic MDP Environment

h-DQN

The h-DQN agent is located in ./agent/hDQN.py. Below is our replication of Figure 4 from the paper:

Requirements

• numpy
• tensorflow
• keras
• h5py
• matplotlib