Optimization Algorithms from Scratch

This project implements and compares optimization algorithms used in neural network training, including:

• Gradient Descent
• Momentum
• Adam
• Learning Rate Decay

The notebook visualizes loss curves and decision boundaries to compare optimizer behavior.

Files

• optimization_algs.ipynb – main notebook implementation

Technologies

• Python
• NumPy
• Matplotlib
• Jupyter Notebook