Simple LSTM-based word-level language model in PyTorch
Switch branches/tags
Nothing to show
Clone or download
Latest commit 5a0f888 Jun 12, 2017
Permalink
Failed to load latest commit information.
.gitignore Initial commit Jun 12, 2017
LICENSE update readme Jun 12, 2017
README.md update readme Jun 12, 2017
lm.py Initial commit Jun 12, 2017
ptb-lm.py Initial commit Jun 12, 2017
reader.py Initial commit Jun 12, 2017

README.md

Simple Word-based Language Model in PyTorch

This model is directly analagous to this Tesnsorflow's LM. In fact, the reader is directly taken from its older version

See this blogpost.

How to RUN:

python ptb-lm.py --data=[PATH_TO_DATA]

Default params should result in Test perplexity of ~78.04. Your actual result will vary due to random initialization. This basically matches results from TF's tutorial, only faster.

On GTX 1080 I am getting around 7,400 wps.

Files

  • lm.py - language model description
  • reader.py - slightly older version of TF's PTB reader which yields numpy arrays as batches
  • ptb-lm.py - driver script

Requirements

  • Python 3 (I used Anaconda distribution)
  • PyTorch (I used 0.1.12)