Support Vector Machines (SVMs) are a remarkable group of machine learning algorithms that are widely used in all fields that require classification or prediction. There are many modified SVM algorithms developed by researchers to improve on the results they yield. To further study those modified SVM algorithms, this project implemented the Linear SVM, Kernel SVM, K-means+SVM, and Clustered SVM (CSVM) in Python 3.
The whole code was implemented in Python 3 with the use of libraries such as Numpy and PyTorch since they provide structures that assist in matrix manipulation. It was decided to achieve a full implementation without the use of functions directly related to SVM implementations that are provided in the libraries. The only exception made was for the use of a QP solver from the CVXOPT library for Kernel SVM. Some libraries that might be install externally are listed in the following.
- PyTorch
- CVXOPT
The project applied SVM Variations on four datasets: Minst odd/even, Minst 3/8, SVM Guide 1, and KDD. The description of the datasets are listed as follows.
| Dataset | Train samples | Test samples | Features | Classes |
|---|---|---|---|---|
| Minst odd/even | 60,000 | 10,000 | 784 | 2 |
| Minst 3/8 | 11,982 | 1,984 | 784 | 2 |
| SVM Guide 1 | 3,089 | 4,000 | 4 | 2 |
| KDD | 63,209,277 | 10,000,000 | 3 | 2 |