This is my project undertaken as a part of the course CS 782, titled Machine Learning. The aim of the project is to study and implement optimizers for Support Vector Machine and a variant called Least Square Support Vector Machine and compare the two models on several heads such as training time, accuracy and behaviour towards various types of datasets.
Support Vector Machines were optimized using Stochastic Gradient Descent and Sequential Minimal Optimization. Least Square Support Vector Machine was optimized using Conjugate Gradient method.
It is noted that LS-SVM is faster to train, but heavily suffers from overfitting as the mathematical foundation is flawed. SVM, on the other hand is slower to train but is much more reliable and accurate. Below are the results