planned change of default value for n_init parameter of KMeans to auto will silently worsen results of existing code

[gittar]

In version 1.2 the new value auto will be introduced for the KMeans parameter n_init and in version 1.4 auto is planned to become the default value for n_init:

scikit-learn/sklearn/cluster/_kmeans.py

Lines 1199 to 1206 in a7698a8

	When `n_init='auto'`, the number of runs will be 10 if using
	`init='random'`, and 1 if using `init='kmeans++'`.
	.. versionadded:: 1.2
	Added 'auto' option for `n_init`.
	.. versionchanged:: 1.4
	Default value for `n_init` will change from 10 to `'auto'` in version 1.4.

For uses of KMeans with init='k-means++' (which is the default) this will silently change the default value of n_init from 10 to 1.

This can (and will in many cases) strongly worsen the results of existing code. The code below illustrates the difference of having 1 vs. 10 as value of n_init. In each call of check() a new data set is of 1000 points is generated and KMeans with default method kmeans++ is run 10 times with n_init=10 and 10 times with n_init=1. Finally the mean percentage deterioration from having n_init=1 instead od n_init=10 is computed. It is always a deterioration and is between 1.81% and 3.52% in the shown experiments. This can be quite significant depending on the application.

My question is wether it is a good idea to make this change which will likely worsen the result quality of every call of KMeans with default parameters (or with explicit setting of init='k-means++'). While not a breaking change in the syntactic sense it is a breaking change numerically since many KMeans results can be expected to deteriorate. If this change is made as planned this should IMO be communicated very clearly.

Opinions?

from sklearn.cluster import KMeans
import numpy as np
def check():
    err={}
    # random data set from unit square
    X=np.random.random(size=(1000,2))
    for n_init in [10,1]:
        err[n_init]=0
        for _ in range(10):
            km=KMeans(n_clusters=100, n_init=n_init)
            km.fit(X)
            err[n_init]+=km.inertia_
    print(f"mean error for n_init=1 is {err[1]/err[10]-1:.2%} higher than for n_init=10")
        
for i in range(10):
    check()

Example Output:
mean error for n_init=1 is 2.85% higher than for n_init=10
mean error for n_init=1 is 3.52% higher than for n_init=10
mean error for n_init=1 is 3.25% higher than for n_init=10
mean error for n_init=1 is 1.81% higher than for n_init=10
mean error for n_init=1 is 2.68% higher than for n_init=10
mean error for n_init=1 is 2.91% higher than for n_init=10
mean error for n_init=1 is 2.01% higher than for n_init=10
mean error for n_init=1 is 3.02% higher than for n_init=10
mean error for n_init=1 is 2.25% higher than for n_init=10
mean error for n_init=1 is 2.75% higher than for n_init=10

[glemaitre]

I open #25022 since this is a real issue and we would need to act on it before the next release.