bert_from_scratch

This directory contains all the scripts and code you need to train BERT from scratch on a TPU on Google Cloud.

Installation

TODO: Dockerize.

bash ./scripts/prepare_training_data.sh [BUCKET NAME] [TRAIN FILE NAME]

bash ./scripts/train_from_scratch.sh [BUCKET NAME] [TPU NAME]

bash ./scripts/convert.sh [BUCKET NAME]

Name		Name	Last commit message	Last commit date
Latest commit History 36 Commits
bert_from_scratch		bert_from_scratch
scripts		scripts
.gitignore		.gitignore
README.md		README.md
requirements.txt		requirements.txt