ENH Make KNeighborsClassifier.predict handle X=None

[dkobak]

Reference Issues/PRs

Fixes #29722 and #27747.

This makes predict(), predict_proba(), and score() of KNeighborsClassifier and RadiusNeighborsClassifier take X=None. In this case predictions for all training set points are returned, and points are NOT included into their own neighbors. This makes it consistent with the current behaviour of KNeighborsRegressor and RadiusNeighborsRegressor.

New functionality is described in docstrings (and added to the docstrings of KNeighborsRegressor and RadiusNeighborsRegressor).

Tests are added confirming that knn.fit(X, y).predict(None) is equivalent to cross_val_predict(knn, X, y, cv=LeaveOneOut()).

Sanity checking

@ogrisel asked me to add a notebook showing that this is a statistically sane thing to do (#29722 (comment)):

It would also be great to check in a side simulation study (in a notebook) to show that the variance of the LOO CV scheme is not significantly larger than the one obtained fors k-fold CV with k in {5, 10, 50} and for n_neighbors in {1, 10, 100} to confirm the outcome of the literature review in #27747 (comment). This side notebook should not part of the scikit-learn repo but linked in the PR to help assert that the new feature exposed in the PR is a methodologically sane thing to do.

I did this here: #29722 (comment). Is it sufficient to simply leave it there?

[github-actions]

✔️ Linting Passed

All linting checks passed. Your pull request is in excellent shape! ☀️

_{Generated for commit: 6fc430b. Link to the linter CI: here}

[adrinjalali]

LGTM.

[OmarManzoor]

Thanks for the PR @dkobak

[dkobak]

@OmarManzoor After I commited your one-line change to v1.6.rst some checks started failing, and I don't understand why :-/

[OmarManzoor]

It seems like there are some conflicts with main. Can you merge main in your branch and resolve the conflicts?

[dkobak]

Oh! Turns out that upcoming changes should not go into v1.6.rst anymore but into separate files in upcoming_changes folder. This was implemented only yesterday: #30081. I adapted my PR accordingly and now everything seems to pass. Thanks!

[OmarManzoor]

LGTM. Thanks @dkobak