Advanced Learning Algorithms

This repository contains comprehensive notes and materials for the Advanced Learning Algorithms course from Stanford and Deeplearning.AI, focusing on neural networks, model evaluation, and decision trees. Props to Andrew Ng for creating such a great course!

Contents

• course notes.pdf: Document containing detailed course notes
• advanced_learning_algorithms.tex: LaTeX source file for the document
• figures/: Directory containing generated images for the document
• NOTEBOOKS/: Original course notebooks with implementations and examples

How to Use

Compiling the LaTeX Document

To compile the LaTeX document into a PDF and give the document your personal touch, you need a LaTeX distribution installed (such as TeX Live, MiKTeX, or MacTeX).

Once installed, compile the document using:

pdflatex advanced_learning_algorithms.tex
pdflatex advanced_learning_algorithms.tex

You may need to run this command multiple times to ensure all references are properly resolved. You can also use Overleaf to compile the document.

Course Topics

The course notes cover the following topics:

1. Neural Networks Fundamentals

   • Neurons and Layers
   • Activation Functions (Linear, Sigmoid, ReLU, Softmax)
   • Network Architecture
   • Forward Propagation

2. Training Neural Networks

   • Loss Functions (MSE, Binary Cross-Entropy, Categorical Cross-Entropy)
   • Backpropagation Algorithm
   • Optimization Algorithms (SGD, Momentum, Adam)
   • Derivatives and Chain Rule

3. Model Evaluation and Selection

   • Train/Validation/Test Sets
   • Bias and Variance Tradeoff
   • Learning Curves
   • Regularization Techniques (L1/L2, Dropout, Early Stopping)

4. Decision Trees and Ensemble Methods

   • Decision Tree Algorithms
   • Information Gain and Entropy
   • Random Forests
   • Boosting Methods (AdaBoost)

5. Advanced Topics

   • Convolutional Neural Networks (CNNs)
   • Practical Implementation with TensorFlow
   • Model Diagnostics and Debugging

Jupyter Notebooks

The repository includes the following Jupyter notebooks from the course:

• C2W3_Lab_01_Model_Evaluation_and_Selection.ipynb - Model evaluation techniques
• C2W3_Lab_02_Diagnosing_Bias_and_Variance.ipynb - Bias and variance diagnosis
• C2_W1_Assignment.ipynb - Neural networks assignment
• C2_W1_Lab01_Neurons_and_Layers.ipynb - Building neurons and layers
• C2_W1_Lab02_CoffeeRoasting_TF.ipynb - TensorFlow implementation
• C2_W2_Assignment.ipynb - Neural network training assignment
• C2_W2_Backprop.ipynb - Backpropagation implementation
• C2_W2_Derivatives.ipynb - Understanding derivatives
• C2_W2_Multiclass_TF.ipynb - Multiclass classification
• C2_W2_Relu.ipynb - ReLU activation function
• C2_W2_SoftMax.ipynb - Softmax function implementation
• C2_W3_Assignment.ipynb - Model evaluation assignment
• C2_W4_Decision_Tree_with_Markdown.ipynb - Decision trees with explanations
• C2_W4_Lab_01_Decision_Trees.ipynb - Decision tree implementation
• C2_W4_Lab_02_Tree_Ensemble.ipynb - Ensemble methods

Please also note that this is not the entirety of the course, but rather a selection of key topics and implementations mentioned in the course that I have personally picked out.