The goal of our project is to develop an accurate tagger for questions posted on Stack Exchange. Our problem is an instance of the more general problem of developing accurate classifiers for large scale text datasets. We are tackling the multilabel classification problem where each item (in this case, question) can belong to multiple classes (in this case, tags). We are predicting the tags (or keywords) for a particular Stack Exchange post given only the question text and the title of the post. In the process, we compare the performance of Support Vector Classification (SVC) for different kernel functions, loss function, etc.
We found linear SVC with Crammer Singer technique produces best results.
Testing Error for SVC with different kernel functions where number of iterations = 10,000
| Kernel | C = 1000(hard-margin) | C = 0.001(soft-margin) |
|---|---|---|
| RBF | 43.1 % | 48.5 % |
| Linear | 51.9 % | 45.2 % |
| Polynomial (n=2) | 54.4 % | 65 % |
| Polynomial (n=3) | 72.2 % | 84.4 % |
| Sigmoid | 84.4 % | 84.4 % |
Testing Error for Linear SVC with different techniques where C = 0.001 (soft-margin) and number of iterations = 10,000
| Technique | Hinge Loss Function | Square Hinge Loss Function |
|---|---|---|
| One-vs-rest | 47.59 % | 68 % |
| Crammer Singer | 45.25 % | 45.25% |
#Report
Our detailed report and results are available here.
#Team
This work has been done as a part of a course project for Artificial Neural Network (IEE-03) at IIT Roorkee.