Deep Learning Basics Using Pytorch
If you, like me, learn by doing, this is the course for you. You'll experiment on a series of toy data sets and who knows, maybe in the process learn the subject. Beginners may continue down the rabbit hole; all others may proceed to arxiv.
Teachers' Tips and Notes
The accompanying slides DeepLearningBasics.pptx are for teachers who intend to teach a course on deep learning. The slides are colour-coded as instructions to teachers ("red"), narratives to follow ("yellow") and slides to show ("that dark bluish colour").
Getting the Lessons
If you have git installed, download all the exercises by issuing the following command:
If you do not have git installed, just download everything as a zip file using the "Clone or Download" link.
Start the notebook server from the directory where you downloaded the lessons by issuing the command:
These lessons will open up in your browser!
Install Pytorch. Our lessons are based on Pytorch.
Start with the slides DeepLearningBasics.pptx.
As you go through the slides, do the exercises in the Jupyter Notebook files (.ipynb).
The Jupyter Notebook Exercises
This course is organized around a series of Pytorch exercises. Click on the files ending in .ipynb and they'll open up in your browser.
You'll be able to run the programs in the exercises directly from your browser.
Understanding Pytorch Tensors
How to create and manipulate tensors using Pytorch and how to visualize a multidimensional space.
Layer of Neurons = Affine Transform
Building a simple neural network layer (an affine transform) using Pytorch.
Turning a neural network layer into a classifier - just add a decision rule!
Introducing toy problem 1.
That was easy? Try toy problem 2.
Understanding Casting Learning as an Optimization Problem
Tool for generating data for toy problems 1 and 2
Computing the cross-entropy loss for a single data point
Computing the cross-entropy loss for a batch of data
Computing the logistic loss for a single data point (an alternative to cross-entropy)
Understanding Single-Layer Neural Networks
Using the 'loss function' $x^2+3$ to illustrate gradient descent.
Gradient descent over generated data set for problem 1.
Gradient descent over generated data set for problem 2.
Motivating and Learning about Multi-Layer Neural Networks!
Generating data-set 3 - the XOR function.
Can you compute the XOR function using a single-layer neural network?
Using non-linear activation functions.
Creating a multi-layer neural network using non-linear activation functions. Can you compute the XOR function using a multi-layer neural network (using a ReLU)?
Can you compute the XOR function using a multi-layer neural network (using a Sigmoid activation function)?
Can you compute the XOR function using a multi-layer neural network with a bias term (using a Sigmoid activation function)?
Can you improve the XOR function using a multi-layer neural network with a bias term (using a ReLU activation function)?
The Back-Propagation Algorithm
Let's start learning about backprop.
Backprop computation for a different data point (a different x and y -> input and label).
Backprop computation for a batch of data points (a set of x and y -> input and label).
Backprop using a single data point for a multilayer neural network.
Backprop using a batch of data points for a multilayer neural network.
Image Classification and Convolutional Neural Networks
Classifying images. Fetching training data (MNIST).
Classification of MNIST images using a single-layer neural network.
Classification of MNIST images using a multi-layer neural network.
Classification of MNIST images using 7 layers in the multi-layer neural network.
Classification of MNIST images using convolutional layers (a rough implementation of LeNet Model 5) in the neural network.
Text Processing and Recurrent Neural Networks
Creating data for demonstrating sequential classification.
Sequential classification using an RNN.
A sequence to sequence model.