Skip to content


Switch branches/tags

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time

Event-to-Sentence Ensemble

Code for the paper "Story Realization: Expanding Plot Events into Sentences" Prithviraj Ammanabrolu, Ethan Tien, Wesley Cheung, Zhaochen Luo, William Ma, Lara J. Martin, and Mark O. Riedl

On arXiv:

Disclaimer: Code is not upkept

BibTex Citation:

    title = "Story Realization: Expanding Plot Events into Sentences",
    author = "Ammanabrolu, Prithviraj and Tien, Ethan and Cheung, Wesley and Luo, Zhaochen and Ma, William and Martin, Lara J. and
      Riedl, Mark O.",
    booktitle={{Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20)}},
    year = "2020",
    arxivId = {1909.03480},
    url = {},


Dataset: The dataset is now available on Hugging Face:

Event Creation (optional): If you wish to "eventify" your own sentences, you will run the code found in the EventCreation folder. This code was made using Python 3.6. Before running anything, you will want to get your favorite version of Stanford's parser and put it under EventCreation/tools/stanford. We used, stanford-english-corenlp-2016-10-31-models.jar, and, which were then unzipped into their respective directories.

The file shows you how to run the code if you want a sentence eventified at a time.

The process can also be done in two steps:

  1. Start the ./ server and run python to get a JSON of the parses/NER of the sentences. You will need to change the name of the files in, and once is finished running, remove the last comma in the JSON so that it can be read as a real JSON file.
  2. Run, passing in the parse file and the sentence file that you get from the first step (in addition to what you want to name the output file). Example: python parsed.json sentences.txt events.txt

Start RetEdit server:

cd E2S-Ensemble/RetEdit/
sudo bash

Takes a few minutes to set up, once you see flask output (says something like 'Running on') then proceed in a separate terminal.

Run Ensemble:

Edit or create a <config_file>.json (most recently used is config_drl_sents.json). Most likely the only thing that will need to be changed is the "test_src" entry under "data" to reflect a new input file. (Note: Gold standard dependencies have been removed)


cd E2S-Ensemble/
source activate e2s_ensemble
python --config <config_file>.json --sample --cuda --outf <outputfile>.txt

This will generate 3 output files:

<outputfile>.txt: Pure e2s output with ensemble thresholds

<outputfile>_verbose.txt: Each sentence is preceded by the model that generated the chosen sentence.

<outputfile>_more_verbose.tsv: Output of ALL 5 models, with their respective confidence scores (useful for quick threshold tuning)

After Ensemble

If no longer in use, ctrl+c out of the RetEdit flask instance, and run

sudo bash

to clean up the RetEdit docker instances in order to prevent excess memory consumption.

Slotfilling Code is in Slotfilling - edit to read in your data. Pass the event and the memory graph (responsible for keeping track of entities) through getAgentTurn.

Other potentially useful files:

python <outputfile>_more_verbose.tsv <reweighted_outf>.txt takes in a .tsv generated from and creates new output files based on the thresholds defined in (Generates <reweighted_outf>.txt and <reweighted_outf>_verbose.txt)

python <outputfile>_more_verbose.tsv <individual_outf>.txt takes in a .tsv generated from and creates 5 separate output files, one for each model and their respective outputs. (Generates <reweighted_outf>.txt and <reweighted_outf>_verbose.txt)

python <file>.txt calculates average number of words per sentence in the entire file.

python <outputfile>_verbose.txt takes in a <outputfile>_verbose.txt generated from and prints out the % utilization of each model in the output file.


Main code is under, which calls the seq2seq-based decoders (Vanilla, mcts, and FSM) that have been reimplemented under and the template decoder under, and also queries the Retreival-and-Edit flask instance, which must be previously instantiated.

It takes in an input event file, and runs 5 different threads, one for each model. There are tqdm progress bars for each model to see approxmiately how long each model will take (mcts will take the longest, followed by RetEdit).

Model files and data files have been (for the most part) omitted from this repo due to space issues, but to rebuild a working version of the ensemble from scratch, these are the steps that need to be followed:


Use e2s_ensemble.yml to create the correct conda environment by doing

conda env create -f e2s_ensemble.yml
source activate e2s_ensemble

Preparing the Retrival-and-Edit (RetEdit) model (based on this paper)


docker / nvidia-docker

How to train

Work in folder E2S-Ensemble/RetEdit/

  1. Prepare data ⋅⋅* create a dataset folder within /datasets ..* create tab-separated dataset: train.tsv, valid.tsv, and test.tsv ..* initialize word vectors within /word_vectors (using helper script ..* set up config file in /editor_code/configs/editor
  2. Run sudo bash
  3. Run export COPY_EDIT_DATA=$(pwd); export PYTHONIOENCODING=utf8; cd cond-editor-codalab; python {CONFIG-FILE.TXT}

How to use Either modify and run or (which calls to handle requests outside of the docker container)

Preparing the Sequence-to-Sequence based models (Vanilla, FSM, mcts)

Work in the folder E2S-Ensemble/mcts Modify or create a <config_file>.json to incorporate the intended training and testing sets for training. To run training, under the e2s_ensemble conda environment, run python --config config.json. To get output for the vanilla seq2seq model, run python --config config.json.

The FSM and mcts model needs to point to this seq2seq trained model, but does not require any further training.

Preparing the Templates model

This code adapted from the AWD-LSTM codebase. Work in the folder E2S-Ensemble/Templates

Although the dataset, is generalized, we generalize it a bit further in an attempt to yield better training by removing the numbers from the named entities. To do this, run python input.txt output.txt to create a file that has the numbers on the named entities removed.

Then, prepare a training/validation/test split, each named train.txt, valid.txt, test.txt respectively. To train the model, python --data data/scifi/ --model BiGRU --batch_size 20 --nlayers 1 --epochs 60 --save --lr 1

To incorpate for use in, update the parameters in the first few argument statements.


No description, website, or topics provided.






No releases published


No packages published