The Transformer - Attention Based Sequence Transduction

Slides of a 25 min deep-dive into the transformer sequence-to-sequence architecture (Attention Is All You Need, Vaswani et al., 2017). I gave this talk at the Data Science Meetup Münster on September 19, 2019.

Links to Resources

Fun:

• https://talktotransformer.com/
• https://transformer.huggingface.co/ (Write With Transformer)

Blogs:

• http://jalammar.github.io/
• http://nlp.seas.harvard.edu/2018/04/03/attention.html (The Annotated Transformer)
• https://lilianweng.github.io/lil-log/2018/06/24/attention-attention.html

PyTorch:

• https://huggingface.co/pytorch-transformers/
• https://github.com/pytorch/fairseq
• https://docs.fast.ai/text.models.html

Tensorflow:

• https://www.tensorflow.org/beta/tutorials/text/transformer
• https://github.com/google-research/bert