GitHub - vat99/Megatron-LM: Ongoing research training transformer language models at scale, including: BERT & GPT-2

vat99 / Megatron-LM Public

forked from NVIDIA/Megatron-LM

Notifications You must be signed in to change notification settings
Fork 0
Star 0

Ongoing research training transformer language models at scale, including: BERT & GPT-2

0 stars 2k forks Branches Tags Activity

Notifications

Name		Name	Last commit message	Last commit date
Latest commit History 1,299 Commits
12022_gpt_distributed_40gb_8socket_mbs16_bs128_gbs1024.txt		12022_gpt_distributed_40gb_8socket_mbs16_bs128_gbs1024.txt
12022_gpt_distributed_40gb_8socket_mbs16_bs64_gbs512.txt		12022_gpt_distributed_40gb_8socket_mbs16_bs64_gbs512.txt
12022_gpt_distributed_40gb_8socket_mbs2_bs128_gbs1024.txt		12022_gpt_distributed_40gb_8socket_mbs2_bs128_gbs1024.txt
12022_gpt_distributed_40gb_8socket_mbs2_bs64_gbs512.txt		12022_gpt_distributed_40gb_8socket_mbs2_bs64_gbs512.txt
12022_gpt_distributed_40gb_8socket_mbs2_bs64_gbs512_new.txt		12022_gpt_distributed_40gb_8socket_mbs2_bs64_gbs512_new.txt
12022_gpt_distributed_40gb_8socket_mbs2_bs64_gbs512_retry.txt		12022_gpt_distributed_40gb_8socket_mbs2_bs64_gbs512_retry.txt
12022_gpt_distributed_40gb_8socket_mbs32_bs128_gbs1024.txt		12022_gpt_distributed_40gb_8socket_mbs32_bs128_gbs1024.txt
12022_gpt_distributed_40gb_8socket_mbs32_bs64_gbs512.txt		12022_gpt_distributed_40gb_8socket_mbs32_bs64_gbs512.txt
12022_gpt_distributed_40gb_8socket_mbs4_bs128_gbs1024.txt		12022_gpt_distributed_40gb_8socket_mbs4_bs128_gbs1024.txt
12022_gpt_distributed_40gb_8socket_mbs4_bs64_gbs512.txt		12022_gpt_distributed_40gb_8socket_mbs4_bs64_gbs512.txt
12022_gpt_distributed_40gb_8socket_mbs64_bs64_gbs512.txt		12022_gpt_distributed_40gb_8socket_mbs64_bs64_gbs512.txt
12022_gpt_distributed_40gb_8socket_mbs8_bs128_gbs1024.txt		12022_gpt_distributed_40gb_8socket_mbs8_bs128_gbs1024.txt
12022_gpt_distributed_40gb_8socket_mbs8_bs64_gbs512.txt		12022_gpt_distributed_40gb_8socket_mbs8_bs64_gbs512.txt

About

Ongoing research training transformer language models at scale, including: BERT & GPT-2

Report repository

Releases

Packages

No packages published