Skip to content
No description, website, or topics provided.
Branch: master
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
README.md Update README.md Mar 12, 2019
cal_ppl.py initial commit Mar 11, 2019
data.py initial commit Mar 11, 2019
embed_regularize.py initial commit Mar 11, 2019
finetune.py initial commit Mar 11, 2019
generate.py
get_data.sh initial commit Mar 11, 2019
locked_dropout.py initial commit Mar 11, 2019
main.py
model.py initial commit Mar 11, 2019
partial_shuffle.py Update partial_shuffle.py Mar 12, 2019
utils.py initial commit Mar 11, 2019
weight_drop.py initial commit Mar 11, 2019

README.md

Partially Shuffling the Training Data to Improve Language Models

This repository contains the code for the Partial Shuffle method, and a modified version of the DOC language model that utilizes this method.

If you'd like to run the DOC + Partial Shuffle models, use the same commands as in the original DOC model, presented here.

The code for the Partial Shuffle method itself is in partial_shuffle.py. If you'd like to use this method in your own language model, simply import partial_shuffle.py, and call it before each epoch, as in line 196 in main.py. No other modifications are required.

You can’t perform that action at this time.