ENH check_scoring() has raise_exc for multimetric scoring

[StefanieSenger]

Reference Issues/PRs

#28360
#28359

What does this implement/fix? Explain your changes.

This PR suggests to expose raise_exc in check_scoring to facilitate a public API for multrimetric scoring.
Thus check_scoring can also be used in cross_validate in a simpler way (this is actually how I came across it).

[github-actions]

✔️ Linting Passed

All linting checks passed. Your pull request is in excellent shape! ☀️

_{Generated for commit: 2a30caf. Link to the linter CI: here}

[StefanieSenger]

If this is wanted, then we need to add a changelog entry. And since the CI "Check Changelog" did not alarm, should we move check_scoring() somewhere where it's recognised by the CI as part of the public API?

[adrinjalali]

This would need a test. And since it's touching public API, it needs a changelog. Otherwise LGTM.

[StefanieSenger]

Thanks @adrinjalali, I have now added a test.

[glemaitre]

There is something weird here maybe the "to" should not be here.

[StefanieSenger]

Hm, I believe it's correct English. Whether to do A if condition or to do B.

[glemaitre]

I see, do you mind inverting both part around the "or":

Whether to return an error code or to raise an ...

Having a really long sentence before the "or" make my parsing algorithm fail :).

[StefanieSenger]

I can see what you mean.

However, I really think that with those kinds of speaking params (raise_exc) we should document what happens when True first, because this is the natural meaning (and the default). Otherwise we force people to switch signs while reading.

What about using parenthesis like so:

Whether to raise an exception (if a subset of the scorers in multimetric scoring fails) or to return an error code.

[glemaitre]

The parenthesis make the trick for me.

[glemaitre]

It looks good apart from little nitpicks.

[StefanieSenger]

Thank you @glemaitre. I have added an example, but had to exclude it from doctest. Would you mind having another look?

[StefanieSenger]

Hm, I believe it's correct English. Whether to do A if condition or to do B.

[glemaitre]

With the two suggestions, I think that it will be enough.

[adrinjalali]

minor nit, otherwise LGTM.

[adrinjalali]

should we replace list, tuple, or set to iterable of strings?

[StefanieSenger]

Hm, I actually think we should tell the users explicitly what the can pass in. Any other than those wouldn't show the correct behaviour in li. 1054-56:

    if isinstance(scoring, (list, tuple, set, dict)):
        scorers = _check_multimetric_scoring(estimator, scoring=scoring)
        return _MultimetricScorer(scorers=scorers, raise_exc=raise_exc)

And, any other iterable of strings would raise during the @validate_params().

So I would rather leave it as it is.