Material and references from the MLT Dive into Deep Learning coding sessions
These bi-weekly sessions aim to provide code-focused sessions by reimplementing selected models from the book Dive into Deep Learning. These sessions are meant for people interested in implementing models from scratch. We hope to help participants either get started in their Machine Learning journey or deepen their knowledge if they already have previous experience.
We will try to achieve this by:
-
Helping participants to create their models end-to-end by reimplementing models from scratch and discussing what modules/elements need to be included (e.g., data preprocessing, dataset generation, data transformation, etc.) to train an ML model.
-
Discussing and resolving coding questions participants might have during the sessions.
Even though we welcome everybody to join the sessions, it is highly recommended to have at least intermediate Python skills as we will be using PyTorch to implement models. We also recommend participants have a foundational knowledge of calculus, linear algebra, and statistics/probability theory.
- 30 min – Introduction
- 60 min – Live Coding
- 30 min – Discussion
- Coding env setup example and book presentation
- A quick review of ML domains (supervised/unsupervised/RL)
- General Architecture/Components of ML code
- Implementations:
- simple MLP-model
- Basics of Convolution and related Layers
- Implementations:
- LeNet
- VGG
- Inception Block
- Resnet Block
- Basics of Recurrent Neural Networks
- Tokenization, Vocabulary
- Backpropagation through time
- Implementations:
- RNN
- GRU
- LSTM
- Attention mechanism #1
- Sequence to Sequence Models
- Bahdanau attention
- Luong Attention
Attention mechanism (Transformer) implementation
Generative adversarial networks (DCGAN) implementation
MLT promotes an inclusive environment that values integrity, openness, and respect. https://github.com/Machine-Learning-Tokyo/MLT_starterkit