Skip to content
Branch: master
Find file History
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
..
Failed to load latest commit information.
Pretrain LM Wiki.ipynb
README.md
Tokens to ID.ipynb
create tokens.ipynb
merge wiki.ipynb
prepare_wik.sh
pretrain_lm.ipynb
pretrain_lm.py
sampled_sm.py

README.md

General Language Model

Only wikipedia articles with > 1000 words were used for training.

Implementation based on Sebastian Ruder's open source scripts.

You can’t perform that action at this time.