MNT clean-up deprecations for 1.7: multi_class in LogisticRegression

[jeremiedbb]

When cleaning-up the deprecation of the multi_class parameter, see #28703, I found that we haven't been raising a warning the last 2 versions for the following situation:

from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression

iris = load_iris()

# multiclass problem with solver="liblinear" and multi_class left to default
LogisticRegression(solver="liblinear").fit(iris.data, iris.target)

In this setting it falls back to OVR because liblinear doesn't support multinomial. However the deprecation of multi_class implied that all multiclass problems would be fitted using the multinomial loss and that an error would be raised if a solver doesn't support it. See

From then on, the recommended 'multinomial' will always be used for `n_classes >= 3`.
Solvers that do not support 'multinomial' will raise an error.
Use `sklearn.multiclass.OneVsRestClassifier(LogisticRegression())` if you still want to use OvR.

That's an issue because doing it for 1.7 would break code that was working until now without any FutureWarning.

A pragmatic solution is to delay the deprecation clean-up and raise a warning for this specific case for 1 more release (I don't think we really need 2 here). This is what I implemented in this PR. What do you think @lorentzenchr @ogrisel ?
Do you think it's worth a new specific changelog entry ? I feel like the one in v1.5.rst is enough.

[github-actions]

✔️ Linting Passed

All linting checks passed. Your pull request is in excellent shape! ☀️

_{Generated for commit: 519b315. Link to the linter CI: here}

[lorentzenchr]

First of all, sorry for this mistake.

I would actually dare to carry out the deprecation in 1.7 and see if anybody complains. I have heard so many complaints about scikit-learn, but never about deprecations and their settlement .

[jeremiedbb]

First of all, sorry for this mistake.

Well I think I reviewed your PR so we share it :)

I have heard so many complaints about scikit-learn, but never about deprecations and their settlement .

Maybe because we're cautious about backward compatibility and because we take care to break their code without notice 😉

I would actually dare to carry out the deprecation in 1.7 and see if anybody complains

It crossed my mind, but I think it's better to do the right thing. It could be a pain for some users while keeping the deprecated code 1 more release for us is not an major issue (it might delay some refactorings in logistic regression if planned though).
Let's see what @ogrisel and @adrinjalali think. If needed, I have a branch with the full clean-up ready 😄 .

[thomasjpfan]

I prefer this PR's approach of raising a deprecation warning first.

Let's also include a changelog entry. One cycle for a deprecation warning is shorter than normal.

We should still add to changelog so people know about it. One cycle for a deprecation warning is shorter than normal.

[jeremiedbb]

Let's also include a changelog entry

Done

[lorentzenchr]

@jeremiedbb It seems I forgot to update the solver table in the docstring of LogisticRegression: newton-cholesky now supports multiclass!!!
Could you update that here?

[lorentzenchr]

@jeremiedbb thanks

[pras529]

@jeremiedbb please review this minor documentation correction.