Fix severe bug in distance matrix -- found/fixed by Bernd Fritzke. #2
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
In Nov. 2021, Bernd Fritzke [reported & fixed](https://www.mail-archive.com/r-help@r-project.org/msg263971.html) a severe bug in the distance matrix calculation. The bug resulted in poor performance equivalent to naive init (rather than k-means++). Bernd reports: > In its current form, the results were much worse than those of Hartigan-Wong (the default for k-means in the stats package) for all test problems I tried. However, after fixing the bug, kmeanspp found better results than Hartigan-Wong for all those problems. In summary: > The bug concerns a distance computation which should be a matrix of distances of all data vectors and all current codebook vectors, but is not. The code and an example illustrating the problem is shown below. Basically, to subtract a vector from a matrix, one has to convert the vector into a matrix where all rows are just copies of the vector. ... The fix is trivial. See also this [post by Paul Harrison](https://logarithmic.net/pfh-files/blog/01618034037/k-means-better.html), who also noticed sub-optimal results. When I [compared these to scikit-learn](https://ctwardy.micro.blog/2021/06/10/on-kmeans-clustering.html), I discovered the results were as if the inits were still naive (random). Bernd's analysis would explain that -- and appears to fix it.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In Nov. 2021, Bernd Fritzke reported & fixed a severe bug in the distance matrix calculation. The bug resulted in poor performance equivalent to naive init (rather than k-means++).
Bernd reports:
In summary:
See also this post by Paul Harrison, who also noticed sub-optimal results. When I compared these to scikit-learn, I discovered the results were as if the inits were still naive (random).
Bernd's analysis would explain that -- and appears to fix it.