[MRG+2] LOF algorithm (Anomaly Detection)

[ngoix]

Local Outlier Factor implementation.

Motivated by previous discussions
http://sourceforge.net/p/scikit-learn/mailman/message/32485020
and
#4163

benchmarks of LOF on:
https://github.com/ngoix/scikit-learn/blob/AD_benchmarks/benchmarks/bench_lof_vs_iforest.py

[agramfort]

thanks for the early PR

let me know when you need a review ie when you addressed the standard things (tests, example, some basic doc)

[agramfort]

thanks for the early PR

let me know when you need a review ie when you addressed the standard things (tests, example, some basic doc)

[jmschrei]

I'd also be interested in reviewing this when you've moved past the WIP stage.

[jmschrei]

I'd also be interested in reviewing this when you've moved past the WIP stage.

[ngoix]

I think it is ready for a first review @agramfort @jmschrei !

[ngoix]

I think it is ready for a first review @agramfort @jmschrei !

[jmschrei]

I would like to see some more extensive unit tests, particularly in cases where the algorithm should fail (wrong dimensions or other incorrect types of data passed in). I'll be able to look more at the performance of the code once you merge the mixin with the other class, and change the API to always take in an X matrix.

[jmschrei]

I would like to see some more extensive unit tests, particularly in cases where the algorithm should fail (wrong dimensions or other incorrect types of data passed in). I'll be able to look more at the performance of the code once you merge the mixin with the other class, and change the API to always take in an X matrix.

[jmschrei]

I'd also like to see an example of it performing against a/many current algorithm(s), so that it is clear it is a valuable contribution.

[jmschrei]

I'd also like to see an example of it performing against a/many current algorithm(s), so that it is clear it is a valuable contribution.

[ngoix]

If you have a dataset X and want to remove outliers from it, you don't want to do

fit(X)
predict(X)

because then each sample is considered in its own neighbourhoud: in predict(X), X is considered as 'new observations'.

What the user wants is:

for each x in X,
fit(X-{x})
predict(x)

which is allowed by

fit(X)
predict()

It is like looking for k-nearest-neighbors of points in a dataset X: you can do:

neigh = NearestNeighbors()
neigh.fit(X)
neigh.kneighbors()

which is different from

neigh = NearestNeighbors()
neigh.fit(X)
neigh.kneighbors(X)

I can make predict() have as signature

def predict(self, X):

and allows taking X=None in argument... Is it allowed ?

[ngoix]

If you have a dataset X and want to remove outliers from it, you don't want to do

fit(X)
predict(X)

because then each sample is considered in its own neighbourhoud: in predict(X), X is considered as 'new observations'.

What the user wants is:

for each x in X,
fit(X-{x})
predict(x)

which is allowed by

fit(X)
predict()

It is like looking for k-nearest-neighbors of points in a dataset X: you can do:

neigh = NearestNeighbors()
neigh.fit(X)
neigh.kneighbors()

which is different from

neigh = NearestNeighbors()
neigh.fit(X)
neigh.kneighbors(X)

I can make predict() have as signature

def predict(self, X):

and allows taking X=None in argument... Is it allowed ?

[agramfort]

If you have a dataset X and want to remove outliers from it, you don't want to do

fit(X)
predict(X)

implement a fit_predict(X) method is the way to go.

[agramfort]

If you have a dataset X and want to remove outliers from it, you don't want to do

fit(X)
predict(X)

implement a fit_predict(X) method is the way to go.

[ngoix]

Ok thanks !

[ngoix]

Ok thanks !

[ngoix]

I merged the mixin with LOF class, changed the API and added a comparison example.
@agramfort @jmschrei what do you think?

[ngoix]

I merged the mixin with LOF class, changed the API and added a comparison example.
@agramfort @jmschrei what do you think?

[ngoix]

Thanks @agramfort I've done the changes.

[ngoix]

Thanks @agramfort I've done the changes.

[agramfort]

@ngoix please update what's new and let's merge this !

[agramfort]

@ngoix please update what's new and let's merge this !

[ngoix]

done!

[ngoix]

done!

[agramfort]

@amueller want to take a final look?

for me it's good enough to merge

[agramfort]

@amueller want to take a final look?

for me it's good enough to merge

[amueller]

I think caching the LRD on the training set would be good (and actually make the code easier to follow). I think either predict and decision_function should both be private or neither. I kinda tend towards both, as making public is easier than hiding.
The rest is mostly minor, though how to tune n_neighbors seems pretty important.

[amueller]

It's kinda odd that this example lives in this folder... but whatever..

[ngoix]

Yes very weird! it is the folder of the first outlier detection algorithm in scikit-learn.

[amueller]

I don't feel that the example illustrates the point that was just made about the different densities. I'm fine to leave it as-is but I don't get a good idea of the global vs local. It would be nice to also illustrate a failure mode maybe?

[ngoix]

No global vs local anymore!

[amueller]

Wait, I don't understand this. Please elaborate.

[amueller]

thanks :)

[amueller]

thanks :)

[raghavrv]

Hurray 🍻 Thanks @ngoix !!

[raghavrv]

Hurray 🍻 Thanks @ngoix !!

[tguillemot]

Youpi 🍻 !!

[tguillemot]

Youpi 🍻 !!

[albertcthomas]

Thanks @ngoix !!

[albertcthomas]

Thanks @ngoix !!

[GaelVaroquaux]

Merged #5279.

Whoot!! Thanks to everybody involved.

[GaelVaroquaux]

Merged #5279.

Whoot!! Thanks to everybody involved.

[agramfort]

Congrats !

[agramfort]

Congrats !