Links: Transformer Networks
Dave Touretzky edited this page Dec 18, 2023
·
18 revisions
Transformer networks are deep neural networks now widely used for neural natural language processing, including handling search queries, question answering, image captioning, and translating between languages.
- video (9:10) and text: Transformers Explained: Understand the Model Behind GPT, BERT, and T5
- Transformer: A Novel Neural Network Architecture for Language Understanding, Google AI blog. Very accessible introduction.
- Language Processing with BERT: The 3 Minute Intro (Deep learning for NLP)
- How Transformers Work in Deep Learning and NLP
- Getting Meaning From Text
- A deep dive into BERT: How BERT launched a rocket into natural language understanding
- Transformers From Scratch (Rohrer)
- Transformers From Scratch (Bloem)
- The Annotated Transformer
- Transformer model for language understanding (TensorFlow tutorial on language translation)
- Language modeling with nn.Transformer and TorchText (PyTorch tutorial)
- The Narrated Transformer Language Model
- Tensor2Tensor Transformers
- GPT-3: Language Models are Few-Shot Learners (Paper Explained) (1:04:29)
- A Visual Guide to Transformer Neural Networks (series):
- Rasa Algorithm Whiteboard - Transformers & Attention 1: Self Attention
- Google BERT demo [direct link]
- ML4K BERT Q&A model [direct link]
- Talk to Transformer [direct link]
- TextSynth [direct link]
- Attention Is All You Need, Vaswani et al. 2017.
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, Devlin et al. 2019.
- Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation Wu et al. 2016.
- Google's AI Is Something Even Stranger Than Conscious, Stephen Marche, The Atlantic, June 19, 2022
- How Does ChatGPT Work? Tracing the Evolution of AIGC DTonomy, December 31, 2022
- Simple Transformer Language Model (Python notebook in CoLab)
- SQuAD: Stanford Question Answering Dataset used to train some BERT models