Benchmark KMeans against pydaal KMeans

[amueller]

Pydaal cited speedups of about 40x for a single cpu between their lloyd k-means and ours.
It would be great to reproduce and identify bottlenecks. Also, we should compare against our elkan and see what the differences are.
I think this was done for several low-dim spaces, n_clusters=100 and varying n_samples.

[souravsingh]

@amueller Which dataset should we use for the purposes of the benchmark and do we make use of intel python distribution for the benchmark?

[sangeet259]

I am a newcomer to Open Source contribution . I know python can you tell me how to get started with this bug ?

[amueller]

try using random data with varying dimensionality and number of samples. Use data that is large enough so that the overhead won't outweight the computation, so maybe use something that takes an order of a minute.
I think you need to use their python distribution to use pydaal.

[amueller]

Their benchmarks are here: https://github.com/dvnagorny/sklearn_benchs

[vrishank97]

Can I take this up as my first contribution here?

[souravsingh]

@amueller I have started the benchmark by making the modifications to the repository given. I will be posting my results soonish.

[amueller]

Creating a intel python environment can be done by

conda create -n idp intelpython3_full python=3 -c intel

[amueller]

ok so it looks like they ignore n_init, which means they only run one random iteration, which gives them a 10x speedup to begin with lol. That's pretty ridiculous. Though running 10 random initializations is a bit arbitrary, to be fair....

[amueller]

Actually, they might do 5 random restarts? Though it's not entirely obvious.

[amueller]

It looks like a lot of their speedups were due to the restarts and they did a better benchmark which showed only fractional improvements (I don't remember, like 20% or 40% speedup or something).

I think this issue is now well covered by #10744.