PyTorch Tutorials

This repo contains my implementations and notes on some tutorials for PyTorch.

Tutorials

0 - Notes

• PyTorch Notes
• TorchText Notes - 1: Notes on TorchText before version 0.7, including Field, Example, BucketIterator, etc.
• TorchText Notes - 2: Notes on TorchText after version 0.7, which introduces a new dataset abstraction (see TorchText Release Notes 0.7.0).
• NN for Regression and SVM

1 - Learning PyTorch follows the Official Tutorials in the Learning PyTorch collection.

• 60 Minute Blitz 1
• 60 Minute Blitz 2
• Learning PyTorch with Examples
• What is torch.nn Really
• Visualization with TensorBoard

2 - Text follows the Official Tutorials in the Text collection.

• Intro to PyTorch
• Word Embeddings
• LSTM and Other RNNs
• BiLSTM CRF: Conditional Random Field (CRF) implementation.
• Classifying Names: Char-level one-hot embeddings into an RNN, to predict name categories.
• Generating Names: Char-level and category one-hot embeddings into an RNN, to recurrently predict next characters.
• Translation with Seq2Seq and Attention: Encoder-decoder seq2seq framework with attention.
• Classifying Text with TorchText: An implementation of FastText using TorchText.
• Translating Language with TorchText: Encoder-decoder seq2seq framework with attention using TorchText.

3 - Sentiment Analysis follows the PyTorch Sentiment Analysis repo by Ben Trevett.

• Simple Sentiment Analysis: Word embeddings into an RNN, to predict a binary sentiment target.
  • Intro to Field, Example, Dataset and BucketIterator of TorchText.
• Upgraded Sentiment Analysis: Pre-trained embeddings, bidirectional multi-layer LSTM, dropout and <pad> token handling.
  • Intro to Vectors and Vocab of TorchText.
• Faster Sentiment Analysis: An implementation of FastText.
• Convolutional Sentiment Analysis: Word embeddings into a CNN, to predict a binary sentiment target.
• Multi-Class Sentiment Analysis: Word embeddings into a CNN, to predict a multi-class sentiment target.
• Transformers for Sentiment Analysis: Using pre-trained BERT (by huggingface) as contextual word embeddings, followed by an RNN, to predict a binary sentiment target.

4 - Seq2Seq follows the PyTorch Seq2Seq repo by Ben Trevett.

• Seq2Seq with Neural Networks: Basic encoder-decoder framework.
• Learning Phrase Representations: Relieving information compression by using context vector in every decoding step.
• Learning to Align and Translate: Further relieving information compression by attention.
• Convolutional Seq2Seq Learning: Using CNN instead of RNN for encoder and decoder.
• Attention Is All You Need: An implementation of Transformer.
• Transformer by PyTorch: Using torch.nn.Transformer.

5 - Sequence-Labeling follows the PyTorch PoS Tagging repo by Ben Trevett.

• BiLSTM for PoS Tagging: Word embeddings into a BiLSTM, to predict the tag sequence (position by position).
• Fine Tuning Pretrained Transformers for PoS Tagging: Using pre-trained BERT (by huggingface) as contextual word embeddings, to predict the tag sequence.
• BiLSTM-CRF for PoS Tagging: Word embeddings into a BiLSTM followed by a CRF layer, to predict the tag sequence.
• CRF by torchcrf: Using pytorch-crf package for CRF.
• BiLSTM for NER: BiLSTM and BiLSTM-CRF on NER corpus.

6 - Image Classification follows the PyTorch Image Classification repo by Ben Trevett.