Skip to content
master
Go to file
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 

README.md

This is the code for the EACL '17 paper "Using the Output Embedding to Improve Language Models" by Ofir Press and Lior Wolf.

The folder BayesianRNN implements the weight tying method (with and without weight decay) over the code of Yarin Gal's BayesianRNN language model (commit 7b923cc84f48f2dc65113aa012c3b99dfa6abed8 ). Notice that the number of units in each LSTM layer was changed to 1500.

The folder ptb_word_lm implements the weight tying method and projection regularization on the language model from the official tensorflow tutorials . (The full original code is available here ).

For an implementation of weight tying and three way weight tying in neural machine translation models, see Nematus. (For decoder weight tying use the flag "--tie_decoder_embeddings", and for three way weight tying also use "--tie_encoder_decoder_embeddings").

If you use these methods in your research, please cite our paper:

@InProceedings{press2017using,
  author    = {Press, Ofir  and  Wolf, Lior},
  title     = {Using the Output Embedding to Improve Language Models},
  booktitle = {Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers},
  month     = {April},
  year      = {2017},
  address   = {Valencia, Spain},
  publisher = {Association for Computational Linguistics},
  pages     = {157--163},
}

About

Code for the EACL paper "Using the Output Embedding to Improve Language Models" by Ofir Press and Lior Wolf

Resources

Releases

No releases published

Packages

No packages published
You can’t perform that action at this time.