Skip to content
Non-Monotonic Sequential Text Generation
Python Jupyter Notebook
Branch: master
Clone or download
Latest commit d9e19f5 May 2, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
notebooks upload May 2, 2019
tree_text_gen upload May 2, 2019
.gitignore upload May 2, 2019
README.md upload May 2, 2019
requirements.txt upload May 2, 2019
setup.py upload May 2, 2019

README.md

Non-Monotonic Sequential Text Generation

PyTorch implementation of the paper:

Non-Monotonic Sequential Text Generation
Sean Welleck, Kiante Brantley, Hal Daume III, Kyunghyun Cho
ICML 2019

We present code and data for training models described in the paper, and notebooks for evaluating pre-trained models.

Installation

python setup.py develop

Data

For downloading the datasets below, it may be helpful to use gdown.pl.

Persona Chat (Unconditional Generation, Word-Reordering)

  • Google drive
  • Put the .jsonl files into a directory {PCHAT_DIR}.

Machine Translation

  • Google drive
  • Unzip the dataset, e.g. to /path/to/iwslt. Then {MT_DIR} below will be /path/to/iwslt/IWSLT/en-de/.

Using a Pretrained Model

You can use and evaluate pre-trained models in one of the provided notebooks:

Task Models Notebook
Word Reordering Google drive notebooks/word_reorder_eval.ipynb
Unconditional Generation Google drive notebooks/unconditional_eval.ipynb
Translation (Transformer) Google drive notebooks/translation_eval.ipynb

The word-reordering and translation notebooks reproduce the evaluation metrics (e.g. BLEU) in the paper.

The unconditional notebook demos the models via interactive sampling and tree completion.

Training

First download and unzip GloVe vectors into a directory {GLOVE_DIR}.

Word-Reordering

python tree_text_gen/binary/bagorder/train.py --glovepath {GLOVE_DIR}/glove.840B.300d.txt \
                                              --datadir {PCHAT_DIR} 

Unconditional Generation

python tree_text_gen/binary/unconditional/train.py --glovepath {GLOVE_DIR}/glove.840B.300d.txt \
                                                   --datadir {PCHAT_DIR} 

Machine Translation (Transformer)

python tree_text_gen/binary/translation/train_transformer.py --datadir {MT_DIR}

Use --multigpu for multi-GPU.

Machine Translation (LSTM)

python tree_text_gen/binary/translation/train.py --datadir {MT_DIR} --model-type translation \
                                                 --beta-burnin 2 --beta-step 0.05 \
                                                 --self-teach-beta-step 0.05

By default these commands train policies with the annealed oracle. See tree_text_gen/{bagorder, unconditional, translation}/args.py for hyper-parameters and arguments.

You can’t perform that action at this time.