Skip to content

Knight-H/thai-lm-training

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

24 Commits
all-data-bytebpe-20000
all-data-bytebpe-20000
 
 
senior_proj_itos
senior_proj_itos
 
 
thwiki-seniorproj-bytebpe-30522
thwiki-seniorproj-bytebpe-30522
 
 
.gitignore
.gitignore
 
 
DataLoader Parallel.ipynb
DataLoader Parallel.ipynb
 
 
Pretrain Model Original Bert v1 July 27 2020 - edited August 25 2020.ipynb
Pretrain Model Original Bert v1 July 27 2020 - edited August 25 2020.ipynb
 
 
Pretrain Model Original Bert v2 August 29 2020 - RERUN Dont know why stop.ipynb
Pretrain Model Original Bert v2 August 29 2020 - RERUN Dont know why stop.ipynb
 
 
Pretrain Model v1 - June 17 2020.ipynb
Pretrain Model v1 - June 17 2020.ipynb
 
 
Pretrain Model v2 - June 23 2020.ipynb
Pretrain Model v2 - June 23 2020.ipynb
 
 
Pretrain Model v3 - Simple Transformers June 28 2020.ipynb
Pretrain Model v3 - Simple Transformers June 28 2020.ipynb
 
 
Pretrain Model v3.5 - Simple Transformers June 29 2020.ipynb
Pretrain Model v3.5 - Simple Transformers June 29 2020.ipynb
 
 
Pretrain Model v4 - Combined Roberta Dataloader June 28 2020.ipynb
Pretrain Model v4 - Combined Roberta Dataloader June 28 2020.ipynb
 
 
Pretrain Model v4.1 - Combined Roberta Dataloader DEBUG FP16 PARALLEL June 29 2020.ipynb
Pretrain Model v4.1 - Combined Roberta Dataloader DEBUG FP16 PARALLEL June 29 2020.ipynb
 
 
Pretrain Model v4.2 - Combined Roberta Dataloader +RUN CONTINUE July 2 2020.ipynb
Pretrain Model v4.2 - Combined Roberta Dataloader +RUN CONTINUE July 2 2020.ipynb
 
 
README.md
README.md
 
 
SelectorDataSource-v2-knight see.ipynb
SelectorDataSource-v2-knight see.ipynb
 
 
SelectorDataSource-v2.ipynb
SelectorDataSource-v2.ipynb
 
 
Test Mask Filling v1.ipynb
Test Mask Filling v1.ipynb
 
 
Test Mask Filling v2 - Original Bert September 16,2020.ipynb
Test Mask Filling v2 - Original Bert September 16,2020.ipynb
 
 
Tokenizing Senior Project v1 - newmm - July 27 2020.ipynb
Tokenizing Senior Project v1 - newmm - July 27 2020.ipynb
 
 
Tokenizing v1 - June 16 2020.ipynb
Tokenizing v1 - June 16 2020.ipynb
 
 
Tokenizing v2 - CharLevelBPE - June 17 2020.ipynb
Tokenizing v2 - CharLevelBPE - June 17 2020.ipynb
 
 
Tokenizing v3 - WordPiece - June 17 2020.ipynb
Tokenizing v3 - WordPiece - June 17 2020.ipynb
 
 
Tokenizing v4 - Aggregated - June 24 2020.ipynb
Tokenizing v4 - Aggregated - June 24 2020.ipynb
 
 
Tokenizing v5 - Byte Level Only - June 27 2020.ipynb
Tokenizing v5 - Byte Level Only - June 27 2020.ipynb
 
 
Tokenizing v5.2 - WordPiece Only - July 5 2020.ipynb
Tokenizing v5.2 - WordPiece Only - July 5 2020.ipynb
 
 
all-data-bytebpe-20000.tokenizer.json
all-data-bytebpe-20000.tokenizer.json
 
 
make_tok_cache_senior_proj.py
make_tok_cache_senior_proj.py
 
 
replace_dict.pkl
replace_dict.pkl
 
 
runGuru_pretrain_originalBert.py
runGuru_pretrain_originalBert.py
 
 
runHK_pretrain_originalBert.py
runHK_pretrain_originalBert.py
 
 
run_pretrain_originalBert.py
run_pretrain_originalBert.py
 
 
senior_project_util.py
senior_project_util.py
 
 
temp_vocab.txt
temp_vocab.txt
 
 
thwiki-seniorproj-bytebpe-30522.tokenizer.json
thwiki-seniorproj-bytebpe-30522.tokenizer.json
 
 
thwiki-wordpiece-30522.tokenizer.json
thwiki-wordpiece-30522.tokenizer.json
 
 

Repository files navigation

thai-lm-training

python -m torch.distributed.launch --nproc_per_node=2 runGuru_pretrain_originalBert.py

About

No description, website, or topics provided.

Resources

Readme
Activity

Stars

1 star

Watchers

2 watching

Forks

1 fork
Report repository

Releases

No releases published

Packages

No packages published

Languages