Skip to content
Code for the paper "Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networks"
Branch: master
Clone or download
yikangshen Merge pull request #7 from ArkAung/master
Update function name in ON_LSTM.py
Latest commit 46d63cd May 22, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
EVALB add evalb Feb 18, 2019
data/penn initial code Jul 25, 2018
.gitattributes Initial commit Jul 25, 2018
.gitignore initial code Jul 25, 2018
LICENSE
ON_LSTM.py Update function name for __main__ in ON_LSTM May 17, 2019
README.md Update README.md Feb 8, 2019
data.py reinitialize code Nov 12, 2018
data_ptb.py add evalb Feb 18, 2019
embed_regularize.py reinitialize code Nov 12, 2018
locked_dropout.py reinitialize code Nov 12, 2018
main.py add evalb Feb 18, 2019
model.py readme & rename function Nov 15, 2018
parse_comparison.py reinitialize code Nov 12, 2018
requirements.txt Update requirements.txt May 9, 2019
splitcross.py reinitialize code Nov 12, 2018
test_phrase_grammar.py add evalb Feb 18, 2019
utils.py add evalb Feb 18, 2019
weight_drop.py reinitialize code Nov 12, 2018

README.md

ON-LSTM

This repository contains the code used for word-level language model and unsupervised parsing experiments in Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networks paper, originally forked from the LSTM and QRNN Language Model Toolkit for PyTorch. If you use this code or our results in your research, we'd appreciate if you cite our paper as following:

@article{shen2018ordered,
  title={Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networks},
  author={Shen, Yikang and Tan, Shawn and Sordoni, Alessandro and Courville, Aaron},
  journal={arXiv preprint arXiv:1810.09536},
  year={2018}
}

Software Requirements

Python 3.6, NLTK and PyTorch 0.4 are required for the current codebase.

Steps

  1. Install PyTorch 0.4 and NLTK

  2. Download PTB data. Note that the two tasks, i.e., language modeling and unsupervised parsing share the same model strucutre but require different formats of the PTB data. For language modeling we need the standard 10,000 word Penn Treebank corpus data, and for parsing we need Penn Treebank Parsed data.

  3. Scripts and commands

    • Train Language Modeling python main.py --batch_size 20 --dropout 0.45 --dropouth 0.3 --dropouti 0.5 --wdrop 0.45 --chunk_size 10 --seed 141 --epoch 1000 --data /path/to/your/data

    • Test Unsupervised Parsing python test_phrase_grammar.py --cuda

    The default setting in main.py achieves a perplexity of approximately 56.17 on PTB test set and unlabeled F1 of approximately 47.7 on WSJ test set.

You can’t perform that action at this time.