IN PROGRESS...

A complete set of deep learning concepts

ML Notes

Contents

ANN

• Basics of Neural Networks

• Perceptron

  - Understanding about Perceptron with intution
  
  - Plotting a simple perceptron
  - Building a perceptron from scratch
  - Understanding about loss function
  - Proof of Perceptron Convergence Theorem
  
  - Worked on Real World Datasets to understand ANN:
        - Customer Churn Prediction (Binary Classification)
        - MNIST Dataset (Multi-Class Classification)
        - Graduation Admission Prediction (Regression)
  

• Loss functions and cost functions

  - Understanding about loss function in detail
  - Types of loss function
  

• Multi-Layer Perceptron

  - Understanding about MLP along with forward propagation
  

• BackPropagation

  - Understanding BackPropagation from scratch
  - Keras implementation of BackPropagation
  

• Memoization

  - Understanding why it is importatnt
  - Implementing Memoization 
  

• Gradient Descent

  - Understanding Gradient Descent
  
  - Types:
        1. Batch Gradient Descent
        2. Stochastic Gradient Descent
        3. Mini-Batch Gradient Descent
  
  - Vanishing/Exploding Gradient Problem
  

• Improving performance of Neural Networks

  - Different ways to improve the performance of Neural Networks
  

• Problems with a Neural Network

• Early Stopping

  - Intuition behind Early Stopping
  - Implementing Early Stopping in Keras
  

• Feature Scaling

  - Standardization
  - Normalization
  

• Dropout

  - Understanding Dropout
  - Implementing Dropout in Keras
  

• Regularization

  - Understanding Regularization
  - Types of Regularization
        1. L1 Regularization
        2. L2 Regularization
        3. Elastic Net Regularization
  

• Bias variance tradeoff

• Activation Functions

  - Understanding Activation Functions
  - Types of Activation Functions
        1. Sigmoid
        2. Tanh
        3. ReLU
        4. Leaky ReLU
        5. Linear
        6. Step
        7. Parametric ReLU
        8. Exponential Linear Unit (ELU)
        9. Scaling Exponential Linear Unit (SELU)
  

• Weight Initialization techniques:

  - Understanding Weight Initialization techniques
  - Types of Weight Initialization techniques
        1. Zero Initialization
        2. Non-zero constant Initialization
        3. Random Initialization
        4. Xavier Initialization
        5. He Initialization
  

• Batch Normalization

  - Understanding Batch Normalization
  

• Optimizers

  - Understanding Optimizers
  - Types of Optimizers
  - Exponential Moving Weighted Average
        1. Batch Gradient Descent
        2. Stochastic Gradient Descent
        3. Mini-Batch Gradient Descent
        4. Momentum
        5. Nesterov Accelerated Gradient (NAG)
        6. Adaptive Gradient (AdaGrad)      
        7. Root Mean Square Propagation (RMSProp)
        8. Adaptive Moment Estimation (Adam)
  

• Hyperparameter Tuning

  - Understanding Hyperparameter Tuning
  - Using keras-tuner for Hyperparameter Tuning
  

CNN

• Basics of Convolutional Neural Networks

• Convolutional Neural Networks

  - Understanding about Convolutional Neural Networks
  - Convolutional Neural Networks Architecture
  - Convolutional Neural Networks Intuition
  - Convolutional Neural Networks Layers
        1. Convolutional Layer
        2. Pooling Layer
        3. Fully Connected Layer
        4. Output Layer
  

• Convolutional Operation

  - Understanding about Convolutional Operation
  - Convolutional Operation Intuition
  

• Padding

  - Understanding about Padding
  - Types of Padding
        1. Valid Padding
        2. Same Padding
  - Implementing Padding in Keras
  

• Strides

  - Understanding about Strides
  - Implementing Strides in Keras
  

• Pooling

  - Understanding about Pooling
  - Types of Pooling
        1. Max Pooling
        2. Min Pooling
        3. Average Pooling
        4. L2-norm Pooling
        5. Global Pooling
  - Implementing Pooling in Keras
  

• LeNet-5

  - LeNet-5 Architecture
  - LeNet-5 Layers
        1. Convolutional Layer
        2. Pooling Layer
        3. Fully Connected Layer
        4. Output Layer
  - Implementing LeNet-5 in Keras