Implementation of Cost Effective Active Learning

[Vishu26]

This is a modified implementation of the algorithm Cost Effective Active Learning (Pl. refer - CEAL). This version not only picks up the top K uncertain samples but also picks up the top N highly confident samples that may represent information and diversity. It is better than the original implementation as it does not involve tuning the confidence threshold parameter for every dataset.

[cosmic-cortex]

Thanks for the contribution! I am really glad that you took the time to add this to the examples!

I have some small change requests. As I indicated in my review comment, if there are no rigorous studies showing that this implementation is indeed measurably better, then the phrasing better should be replaced with different in the docstring.

I am fine with the implementation having modifications from the original one, but can you please include comments in the example where the differences are? I am thinking about

1. clarifications in the leading docstring,
2. one-liner comments in the body where you are omitting or adding new steps.

Thanks!

[cosmic-cortex]

Do you have any references that can demonstrate that this implementation is better than the original? If no such studies were done, it is fine to leave the example implementation as you provided here, but in this case, please replace the word better with different.

[Vishu26]

Thank you for the review. I have made necessary changes in the comments. Would you also suggest to have a separate implementation of the original CEAL algorithm? It would be quite easy to do so.

Also, I have implemented bayesian active learning by disagreement with monte carlo dropouts. The file is present in the latest commit. It would be nice if you can review that too.

[cosmic-cortex]

I don't think that the original implementation is necessary.

However, the Bayesian active learning with Monte Carlo dropouts is already added as an example (https://github.com/modAL-python/modAL/blob/master/examples/deep_bayesian_active_learning.py). Can you remove your one from the PR and perhaps enhance the already existing example where you see fit? (For instance, replacing the model with LeNet model as you did would be great!)

[Vishu26]

Thank you for the comment. I have deleted Bayesian Active Learning By Disagreement file. I think the already implemented Bayesian Active Learning with Monte Carlo Dropouts uses Max Entropy. I can modify it to use BALD.