No description, website, or topics provided.
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.

Unified Pragmatic Models for Generating and Following Instructions

Daniel Fried, Jacob Andreas, and Dan Klein

NAACL, 2018

This repository currently contains the code for follower and speaker models for the SCONE domains. SAIL is not currently integrated; please contact Daniel Fried if you'd like the code for it, or for other questions about the code.



  • python 3.6
  • dynet 2.0
  • numpy
  • scipy
  • pandas


  • The SCONE dataset. Place the rlong directory inside data/scone.


Train a base follower (aka listener, in the paper) or speaker model with <corpus> <seed> or <corpus> <seed>

where <corpus> is one of alchemy, tangrams, or scene, and <seed> is an integer. We used ensembles of models with seeds 1-10 for the results in the paper. Hyperparameters used in the paper (which achieved the best results on the dev set for each corpus) are hardcoded into these scripts.

Models and results will be output to expts/{follower,speaker}/<corpus>/<model_name>/<seed>.


These scripts will evaluate ensembles of the models, in both the literal setting (using only scores from the ensemble of models used to produce candidates) and the rational setting (using scores from both speakers and listeners). Results will be output to expts/rational_{speaker,follower}/<corpus>/<model>. <corpus> and <corpus>

By default, these scripts will look for 10 follower and speaker models trained with random seeds 1 through 10 (see Training above).