Skip to content

Files

Latest commit

 

History

History
14 lines (13 loc) · 923 Bytes

what-is-the-t5-transformer-and-how-does-it-work.md

File metadata and controls

14 lines (13 loc) · 923 Bytes
title date categories tags
What is the T5 Transformer and how does it work?
2021-02-15
buffer
deep-learning
nlp
t5
transformer
transformers

The Text-to-Text Transfer Transformer or T5 is a type of Transformer that is capable of being trained on a variety of tasks with a uniform architecture. It was created by Google AI and was published about in the paper “Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer“. Here, we’ll take a look at T5 architecture, pretraining, finetuning — including variations and the conclusions that can be derived from them. It effectively summarizes the above linked paper.