PyTorch book

handbook by

Introduction

This is an open source book by AI SUMMER, the goal is to help those who want and use PyTorch for deep learning development and research.

The technology of deep learning is developing rapidly, and PyTorch is constantly updated, and I will gradually improve the relevant content.

Release Notes

As the PyTorch version changes, the tutorial version will be the same as the PyTorch version.

PyTorch has released the latest version 1.6.0.

TODO

Organize chapters and notebooks

!? benchmarks on data loading, weight initialization, optimizers

Custom dataloaders

Templates for better coding in pytorch

Updates of tensorboard instead of tensorboardX

useful functions

Custom Losses and losses from numpy tensors (as in cython), networks types of fusion,

Table of Contents

Chapter 1: Getting Started with PyTorch

1. Introduction to PyTorch
2. PyTorch environment setup
3. PyTorch Deep Learning: 60-minute quick start (official)
   • Tensor
   • Autograd: Automatic Derivation
   • Neural Network
   • Train a classifier
   • Selected Reading: Data Parallel Processing (Multi-GPU)
4. Related Resource Introduction

Chapter 2 Basics

2.1 PyTorch Basics

1. Tensor
2. Automatic Derivation
3. Neural Network Package nn and Optimizer optm
4. Data loading and preprocessing

2.2 Deep Learning Basics and Mathematical Principles

• Deep learning basics and mathematical principles

2.3 Introduction to Neural Networks

• Introduction to Neural Networks Note: This chapter will crash when opened locally with Microsoft Edge, please open Chrome Firefox to view

2.4 Convolutional Neural Networks

• Convolutional Neural Networks

2.5 Recurrent Neural Networks

• Recurrent Neural Networks

Chapter 3 Practice

3.1 Logistic regression binary classification

• Logistic regression binary classification

3.2 CNN: MNIST dataset handwritten digit recognition

• CNN: Handwritten digit recognition of MNIST dataset

3.3 RNN Examples: Predicting Cosine by Sin

• RNN Example: Predicting Cos by Sin

Chapter 4 Improvement

4.1 Fine-tuning networks

• Fine-tuning

• Loss functions

• Optimizers

4.2 Visualization

• visdom

• tensorboardx (tensorboard support now)

• Visual Understanding Convolutional Neural Network

4.3 Fast.ai

• Fast.ai

4.4 Training Skills

4.5 Multi-GPU Parallel Training

• Multi-GPU parallel computing

Chapter 5 Applications

5.1 Introduction to Kaggle

Introduction to Kaggle

5.2 Structured Data

Pytorch processing structured data

5.3 Computer Vision

Fashion MNIST image classification

5.4 Natural Language Processing

• Transformers
• RNN encoder-decoder
• with attention
• Language models

5.5 GANs

5.6 Action Recognition

5.7 Segmentation 2D, 3D

5.8 Medical Imaging, Health & AI

5.5 Collaborative Filtering

Chapter 6 Mobile , IoT

• Raspberry Pi with Pytorch
• Export to mobile
• ONNX

Chapter 7 Appendix

Summary of common operations of transforms

pytorch's loss function summary

pytorch's optimizer summary

License