v0.0.1: Training on AWS Trainium
The following architectures can be trained on AWS Trainium instances (trn1.2xlarge and trn1.32xlarge) :
- ALBERT
- BERT
- DistilBERT
- RoBERTa
- XLM-RoBERTa
- CamemBERT
- Electra
- GPT-2
- GPT-Neo
- MarianMT
- T5
- BART
- ViT
Training examples for many tasks are provided here.