Dive into Deep Learning in 1 Day

Last updated：|today|

Information

• Speaker： Alex Smola

Overview

Did you ever want to find out about deep learning but didn't have time to spend months? New to machine learning? Do you want to build image classifiers, NLP apps, train on many GPUs or even on many machines? If you're an engineer or data scientist, this course is for you. This is about the equivalent of a Coursera course, all packed into one day. The course consists of four segments of 90 minutes each.

1. Deep Learning Basics
2. Convolutional Neural Networks for computer vision
3. Best practices (GPUs, Parallelization, Fine Tuning, Transfer Learning)
4. Recurrent Neural Networks for natural language (RNN, LSTM, GRU)

Prerequisites

You should have some basic knowledge of Linear Algebra, Calculus, Probability, and Python (here's another book to learn Python). Moreover, you should have some experience with Jupyter notebooks, or with SageMaker notebooks. To run things on (multiple) GPUs you need access to a GPU server, such as the P2, G3, or P3 instances.

Syllabus

• This course relies heavily on the Dive into Deep Learning book. There's a lot more detail in the book (notebooks, examples, math, applications).
• The crash course will get you started. For more information also see other courses and tutorials based on the book.
• All notebooks below are availabe at d2l-ai/1day-notebooks, which contains instructions how to setup the running environments.

Time	Topics
8:00---9:00	Setup clinic for laptops
9:00---10:30	Part 1: Deep learning basic
10:30---11:00	Coffee break
11:00---12:30	Part 2: Convolutional neural networks
12:30---2:00	Lunch break
2:00---3:30	Part 3: Performance
3:30---4:00	Coffee break
4:00---5:30	Part 4: Recurrent neural networks

Part 1: Deep Learning Basic

Slides: [keynote], [pdf]

Notebooks:

1. Data Manipulation with Ndarray [ipynb] [slides]
2. Automatic Differentiation [ipynb] [slides]
3. Linear Regression Implementation from Scratch [ipynb] [slides]
4. Concise Implementation of Linear Regression [ipynb] [slides]
5. Image Classification Data (Fashion-MNIST) [ipynb] [slides]
6. Implementation of Softmax Regression from Scratch [ipynb] [slides]
7. Concise Implementation of Softmax Regression [ipynb] [slides]
8. Implementation of Multilayer Perceptron from Scratch [ipynb] [slides]
9. Concise Implementation of Multilayer Perceptron [ipynb] [slides]

Part 2: Convolutional neural networks

Slides: [keynote], [pdf]

Notebooks:

1. GPUs [ipynb] [slides]
2. Convolutions [ipynb] [slides]
3. Pooling [ipynb] [slides]
4. Convolutional Neural Networks (LeNet) [ipynb] [slides]
5. Deep Convolutional Neural Networks (AlexNet) [ipynb] [slides]
6. Networks Using Blocks (VGG) [ipynb] [slides]
7. Inception Networks (GoogLeNet) [ipynb] [slides]
8. Residual Networks (ResNet) [ipynb] [slides]

Part 3: Performance

Slides: [keynote], [pdf]

Notebooks:

1. A Hybrid of Imperative and Symbolic Programming [ipynb] [slides]
2. Multi-GPU Computation Implementation from Scratch [ipynb] [slides]
3. Concise Implementation of Multi-GPU Computation [ipynb] [slides]
4. Fine Tuning [ipynb] [slides]

Part 4: Recurrent neural networks

Slides: [keynote], [pdf]

Notebooks:

1. Text Preprocessing [ipynb] [slides]
2. Implementation of Recurrent Neural Networks from Scratch [ipynb] [slides]
3. Concise Implementation of Recurrent Neural Networks [ipynb] [slides]
4. Gated Recurrent Units (GRU) [ipynb] [slides]
5. Long Short Term Memory (LSTM) [ipynb] [slides]