title | date | categories | tags | ||||||
---|---|---|---|---|---|---|---|---|---|
What is the T5 Transformer and how does it work? |
2021-02-15 |
|
|
The Text-to-Text Transfer Transformer or T5 is a type of Transformer that is capable of being trained on a variety of tasks with a uniform architecture. It was created by Google AI and was published about in the paper “Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer“. Here, we’ll take a look at T5 architecture, pretraining, finetuning — including variations and the conclusions that can be derived from them. It effectively summarizes the above linked paper.