Welcome to the DD2421 ML Challenge. In short, you must build and train a classifier given a labeled dataset and then use it to infer the labels of a given unlabeled evaluation dataset. You must then submit the inferred labels in a specified format, which will be compared to the ground truth.
Without any context, it is important to understand the data. For that, I used a heatmap diagram to check the correlation between features, which would be useful for dimensionality reduction in case some features are heavily correlated.
X3 and X4 have a strong negative correlation (almost opposites of one another), so one possible solution would be to remove one of them. However, the model accuracy wasn't really affected by this, as well as the processing time, so there was no advantage in removing it.
The usual process, transform all nulls and '?' cells into NaN and replace them with the columns's mean value. For columns with a missing Boolean, it would be possible to apply a generative algorithm to "predict" this values but the easy solution was just to remove those rows (3/4 in total).
Several approaches were considered, including applying a Deep Feature Synthesis (DFS). Using DFS with multiplication operation between columns, the number of features increased to 34. To reduce this number, Principal Component Analysis (PCA) was used but to no avail. All in all, no changes in the features were made.
The final model was a Voting Classifier, which combined the classifiers: LightGBM, ExtraTrees and XGBoost.
| Classifier | Accuracy |
|---|---|
| Voting Classifier | 87.5% |
