Cornell Instruction Following Framework
Switch branches/tags
Nothing to show
Clone or download
Latest commit b6bf08d Oct 3, 2018
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
README.md Update README.md Oct 3, 2018

README.md

Cornell Instruction Following Framework (CIFF)

CIFF is intended to provide an integrated framework for developing and experimenting with various natural language instruction following framework. Currently it provides a common interface for 3 datasets and simulators, several models and learning algorithm.

Contents of this repository

Contains simulators and dataset for 3 domains for natural language instruction following:

 a) Block World Dataset (Bisk et al. 2016): A fully-observed map for moving blocks around on a map.

 b) LANI (Misra et al. 2018): A partially observed domain for flying drone in an open space.

 c) CHAI (Misra et al. 2018): A partially observed domain for navigation and simple manipulation in a 3D house.

The code contains experiments for training and testing various models and baselines. Includes:

  a) Simple baselines like stop, random baseline, most frequent action.

  b) Models like Misra et al. 2017, Gated Attention Chaplot et al. 2018, etc.
    
  c) Training algorithms like behaviour cloning, A3C, Reinforce.

Dataset and Simulators: Dataset and simulators are available here: http://clic.nlp.cornell.edu/resources/Misra-EMNLP-2018/

Test scripts for visualizing the dataset and simulators will be available on the wiki soon.

Code To Come Soon: We will release a beta version soon.

Credits

Maintained by: Dipendra Misra (dkm@cs.cornell.edu)

Researchers and Developers: Dipendra Misra, Andrew Bennett, Max Shatkin, Eyvind Niklasson, Valts Blukis, and Yoav Artzi

Publications

Publications using models, data or simulators provided with CIFF.

  1. Mapping Navigation Instructions to Continuous Control Actions with Position Visitation Prediction Valts Blukis, Dipendra Misra, Ross A. Knepper, and Yoav Artzi, [CoRL 2018] (uses the LANI dataset)

  2. Mapping Instructions to Actions in 3D Environments with Visual Goal Prediction Dipendra Misra, Andrew Bennett, Valts Blukis, Eyvind Niklasson, Max Shatkhin, and Yoav Artzi, EMNLP 2018

  3. Scheduled Policy Optimization for Natural Language Communication with Intelligent Agents, Wenhan Xiong, Xiaoxiao Guo, Mo Yu, Shiyu Chang, Bowen Zhou, William Yang Wang, arXiv 2018

  4. Reinforcement Learning for Mapping Instructions to Actions with Reward Learning, Dipendra Misra and Yoav Artzi, AAAI Fall Symposium on Natural Language Communication for Human Robot Interaction. Paper

  5. Mapping Instructions and Visual Observations to Actions with Reinforcement Learning, Dipendra Misra, John Langford and Yoav Artzi, EMNLP 2017. Paper