If you, like me, learn by doing, this is the course for you. You'll experiment on a series of toy data sets and who knows, maybe in the process learn the subject. Beginners may continue down the rabbit hole; all others may proceed to arxiv.
The accompanying slides DeepLearningBasics.pptx are for teachers who intend to teach a course on deep learning. The slides are colour-coded as instructions to teachers ("red"), narratives to follow ("yellow") and slides to show ("that dark bluish colour").
If you have git installed, download all the exercises by issuing the following command:
git clone https://github.com/aiaioo/DeepLearningBasicsTutorial/
If you do not have git installed, just download everything as a zip file using the "Clone or Download" link.
Start the notebook server from the directory where you downloaded the lessons by issuing the command:
jupyter notebook
These lessons will open up in your browser!
Install Pytorch. Our lessons are based on Pytorch.
Start with the slides DeepLearningBasics.pptx.
As you go through the slides, do the exercises in the Jupyter Notebook files (.ipynb).
This course is organized around a series of Pytorch exercises. Click on the files ending in .ipynb and they'll open up in your browser.
You'll be able to run the programs in the exercises directly from your browser.
How to create and manipulate tensors using Pytorch and how to visualize a multidimensional space.
Building a simple neural network layer (an affine transform) using Pytorch.
Turning a neural network layer into a classifier - just add a decision rule!
Introducing toy problem 1.
That was easy? Try toy problem 2.
Tool for generating data for toy problems 1 and 2
Computing the cross-entropy loss for a single data point
Computing the cross-entropy loss for a batch of data
Computing the logistic loss for a single data point (an alternative to cross-entropy)
Using the 'loss function'
Gradient descent over generated data set for problem 1.
Gradient descent over generated data set for problem 2.
Generating data-set 3 - the XOR function.
Can you compute the XOR function using a single-layer neural network?
Using non-linear activation functions.
Creating a multi-layer neural network using non-linear activation functions. Can you compute the XOR function using a multi-layer neural network (using a ReLU)?
Can you compute the XOR function using a multi-layer neural network (using a Sigmoid activation function)?
Can you compute the XOR function using a multi-layer neural network with a bias term (using a Sigmoid activation function)?
Can you improve the XOR function using a multi-layer neural network with a bias term (using a ReLU activation function)?
Let's start learning about backprop.
Backprop computation for a different data point (a different x and y -> input and label).
Backprop computation for a batch of data points (a set of x and y -> input and label).
Backprop using a single data point for a multilayer neural network.
Backprop using a batch of data points for a multilayer neural network.
Classifying images. Fetching training data (MNIST).
Classification of MNIST images using a single-layer neural network.
Classification of MNIST images using a multi-layer neural network.
Classification of MNIST images using 7 layers in the multi-layer neural network.
Classification of MNIST images using convolutional layers (a rough implementation of LeNet Model 5) in the neural network.
Creating data for demonstrating sequential classification.
Sequential classification using an RNN.
A sequence to sequence model.