Machine learning from scratch

Implementing different machine learning and optimization algorithms from scratch and visualizing them without any machine learning/optimization library.

Implemented machine learning algorithms

• Perceptron
• Linear regression
• Logistic regression
• Polynomial regression
• Multiclass classification (One-vs-rest, One-vs-one)
• Nonlinear transformation

Implemented optimization algorithms

Unconstrained optimization :

• Gradient descent
• Newton
• Line search (Bisection, Newton raphson, False position)
• Backtracking line search (Armijo, Goldstein, Weak-Wolfe, Strong-Wolfe)

Constrained optimization :

• Frank-wolfe
• Feasible directions
• Projected gradient
• Reduced gradient