This analysis is from data presented in the research paper: Challenges in benchmarking stream learning algorithms with real-world data
This project analyses a streaming dataset of insects. The Machine Learning algorithm then decides if it is a pest or not. Classification algorithms applied include:
- No-change Classifier
- Majority Class Classifier
- Hoeffding Trees
- SAM-KNN
- Hoeffding Adaptive Trees(HAT)
- Adaptive Random Forest(ARF)
- Leverage bagging
The dataset is seperated based on the kind of drifts the dataset showcases. The above algorithms are applied to the datasets with 3 different drift patterns(Incremental, Abrupt and Gradual)
The accuracies of the above algorithms on Incremental dataset is shown below
As expected the No-change classifier and Majority Class Classifier being very basic ones deliver poor performance. The ARF delivers the best performance on this dataset
Similarly, The accuracies of the above algorithms on Gradual dataset is shown below
A similar trend is followed except the No-Change and Majority classifiers peak for a period of time. This could be due to the fact the data remains constant for a certain period of time resulting in no-change in the result and the majority of the resulting classes are same. However, over time we observe that Leverage Bagging performs the best predictions.
Also, The accuracies of the above algorithms on abrupt dataset is shown below
A similar trend is followed except the No-Change and Majority classifiers peak for a period of time. This could be due to the fact the data remains constant for a certain period of time resulting in no-change in the result and the majority of the resulting classes are same.
However, over time we observe that Leverage Bagging performs the best predictions.
ADWIN is used as drift detection method to accomodate for changes over seasons(different insects populate over different seasons)
The ADWIN drift detection algorithm does not detect any drifts in the incremental dataset. The occurence of drifts for the Gradual and Abrupt dataset has been visualised below:
The final accuracies of different algorithms on different dataset is shown below
