Hulk Smash Thor: Indoor Scenes Visual Navigation with A3C & Imitation Learning

Introduction

This repository provides TensorFlow implementation of 2 extensions to the the deep Siamese target-driven actor-critic model for indoor scenes navigation, first proposed by Yuke Zhu.

Setup

This code was implemented in Tensorflow r1.0. This code has been tested with Python 3.6. Other dependencies can be install with pip: pip install -r requirements.txt.

Training and Evaluation

We include implementation for two separate models: A3C Target-driven with LSTM memory extension, and DAgger imitation learning-based model. The implementation of the former is under train.py and the class ActorCriticLSTMNetwork in network.py. The implementation of the latter is located in dagger_train.py and dagger_network.py.

To train a A3C-LSTM model, first activate the USE_LSTM tag in constants.py, then

python train.py

For the DAgger imitation-learning model,

python dagger_train.py

Acknowledgements

We would like to acknowledge the following references that have offered great help in the implementation.

• miyosuda's async_deep_reinforce repo
• yukezhu's implementation of the Target-driven model (currently unavailable online)

Citation

Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learning
Yuke Zhu, Roozbeh Mottaghi, Eric Kolve, Joseph J. Lim, Abhinav Gupta, Li Fei-Fei, and Ali Farhadi
ICRA 2017, Singapore

License

MIT