Cannot reproduce perplexities in the word language model

[lezcano]

I've trained a few of the example models present in the folder word_language_model, and I have not been able to reproduce the results given in the README.
python main.py --cuda --epochs 6 # I get test ppl 142.07.
python main.py --cuda --epochs 6 --tied # I get a test perplexity of 130.32

I do get similar training perplexities to the ones given in the README, but I believe that this is not necessarily a good measure of how well the model is performing.

For the model
python main.py --cuda --emsize 650 --nhid 650 --dropout 0.5 --epochs 40
I get a training perplexity of about 52, and a test perplexity of 94.30. None of these numbers are even close to the supposed test perplexity of 80.97.

Can anyone reproduce these results or the results given in README.md?

[tvogels]

For

python3 main.py --cuda --emsize 650 --nhid 650 --dropout 0.5 --epochs 40 --tied

I get a test perplexity of 86.44 as opposed to the 75.96 mentioned in README.md.

[hunterlang]

Likewise, for

python main.py --emsize 1500 --nhid 1500 --dropout 0.65 --epochs 40 --tied

I get test perplexity 84.17 instead of 72.3 in README. Has anyone been able to close the gap to the reported perplexities?

[ahug]

For

python main.py --emsize 1500 --nhid 1500 --dropout 0.65 --epochs 40 --tied

I also get a test perplexity of 84.12.

I observed similar discrepancies for the other parameter settings.

[HolyLow]

For

python main.py --emsize 1500 --nhid 1500 --dropout 0.65 --epochs 40 --tied

I also get a test perplexity of 83.62. Have no idea why the reported performance is much better than mine.

[soumith]

About reported perplexities on the larger models, please see 97e3e13

As @HolyLow pointed out in another thread, the given perplexity numbers for the larger models were on Penn_Treebank and not on WikiText2, so removed those references from README for now

[soumith]

The numbers even for the small models were not for WikiText2, but for PTB, so removed them as well.

Closed via 97e3e13 and e0929a4