This is my Master's thesis for receiving the degree of Master of Science in Computer Engineering with specialization in Artificial Intelligence.
Full PDF file and summary are provided in this repository.
In the past decade, machine learning has been used for solving problems with complex patterns. Classification is one of the main learning types which solves problems such as face recognition, text classification, and disease recognition. Support vector machine (SVM) is a state-of-the-art classification method which has good generalization and accuracy. In recent years, classifiers based on SVM have been proposed. Among these are twin support vector machine (TSVM) which has received more attention. The central idea of TSVM is to find two non-parallel hyperplanes for binary classification. Therefore, it solves two smaller-sized Quadratic Programming Problems (QPPs) as opposed to one large QPP in standard SVM. As a result, TSVM is four times faster than standard SVM in theory. Even though TSVM has better prediction accuracy and time complexity than SVM, it has several drawbacks such as high sensitivity to outliers and noise, overfitting, and high computational cost for large datasets. In this study with the aim of addressing the drawbacks, we propose two classifiers, called KNN-based least squares twin support vector machine (KNN-LSTSVM) and a regularized KNN-based twin support vector machine (RKNN-TSVM). The proposed KNN-LSTSVM and RKNN-TSVM classifiers construct K-nearest neighbor graph to give weight to each training samples. Also, margin points of each class are determined using these graphs. This further makes the proposed classifiers more robust to noise and outliers than standard TSVM. The proposed classifiers were comprehensively evaluated on synthetic and benchmark datasets. The experimental results validate the effectiveness of proposed KNN-LTSVM and RKNN-TSVM in terms of classification accuracy and computational time.
Keywords: Twin support vector machine, Nearest neighbor graph, Least squares, Structural risk, Classification