This repo contains a set of DL algorithms implemented from first principles. These notebooks help to understand the inner workings of modern DL libraries such as PyTorch and Keras. Detailed analysis with a hint of mechanistic interpretability is provided along with the code.
For a better experience with notebooks, it is recommended to view the repository at https://nbviewer.org/github/nveshaan/deep_learning/tree/main/
Core
- Vanilla Neural Network
- Weights Initialization
- Normalization
- Residual Connections
- Optimizers
Inductive Bias
- CNN: Convolutional Neural Network
- RNN: Recurrent Neural Network
- GRU: Gated Recurrent Unit
- LSTM: Long Short-Term Memory
- SSM: State Space Model
- GNN: Graph Neural Network
Attention
- Attention Mechanisms
- Positional Encodings
- Transformer
Discriminative
Implemented in architectures
Generative
- GPT: Generative Pre-trained Transformer
- VAE: Variational Autoencoder
- GAN: Generative Adversarial Network
- Diffusion
- Flow Matching
Joint Embedding
- Contrastive Learning
- Self-Supervised Learning
- Grokking
- Double Descent
- Loss Landscape Visualization