CI/FIX Enable numpy dev in [scipy-dev] build and fix Bayesian linear models for numpy 2 compat

[ogrisel]

While reviewing #27075, I realized that the current state of our Linux_Nightly pylatest_pip_scipy_dev does not actually test against numpy dev without us upper-bounding the version of numpy explicitly (yet):

https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=61365&view=logs&j=dfe99b15-50db-5d7b-b1e9-4105c42527cf&t=eb5122d5-ab7e-5479-a8ce-245b4d64938b&l=442

The goal of this PR is to trigger a run to:

• check if scipy and pandas actually break at import time when running against numpy 2 dev,
• if not, check if scikit-learn itself can import successfully and then run its own tests,
• fix scikit-learn itself in case of small problems revealed by the test results.

[github-actions]

✔️ Linting Passed

All linting checks passed. Your pull request is in excellent shape! ☀️

_{Generated for commit: ed58999. Link to the linter CI: here}

[ogrisel]

Good news. Most scikit-learn tests pass (thanks to the fixes contributed earlier by @seberg). We just need to fix a test with unexpected dtype values. I suspect the changes in dtype promotion rules: https://numpy.org/neps/nep-0050-scalar-promotion.html.

I will have a look.

[glemaitre]

LGTM. If the fix is simply in a test, I could imagine to add it inside this PR as well.

[ogrisel]

LGTM. If the fix is simply in a test, I could imagine to add it inside this PR as well.

Not it's not just the test. The dtype of some fitted attribute is upcasted. I have a fix, I will push to the PR and update the title.

[ogrisel]

I milestoned this PR 1.4 so that scikit-learn would not have any numpy 2 incompatible code in it.

[glemaitre]

LGTM.

[lesteve]

LGTM

It seems like float32 dtype was maintained in numpy<2 by relying on float64_scalar * float32_array => float32_array (try executing np.float64(1.) * np.ones(2, dtype=np.float32) if you want to double-check)

The code in this PR is more careful about dtype.

[lesteve]

And also it seems like pandas dev wheel is the one that wants numpy<2 ...