Test failures due to `asarray(..., copy=False)` semantics change in numpy main

[rgommers]

gh-19997 anticipated this change and fixed most things, however that fix was incomplete (or the NumPy implementation changed after it landed). What is left is now visible in wheel builds (example log):

   ERROR io/_harwell_boeing/tests/test_hb.py - ValueError: Unable to avoid copy ...
  ERROR linalg/tests/test_sketches.py - ValueError: Unable to avoid copy while ...
  ERROR sparse/csgraph/tests/test_shortest_path.py - ValueError: Unable to avoi...
  ERROR sparse/linalg/_isolve/tests/test_iterative.py - ValueError: Unable to a...

We need to fix these and then retrigger wheel builds to fix the current nightlies, which are not usable because they were not built against numpy main (see numpy/numpy#25907 (comment)).

Cc @mtsokol for visibility.

[matthewfeickert]

@rgommers @mtsokol @tupui @larsoner As a temporary fix until this Issue gets resolved, you could also download the wheels manually from the nightly build from yesterday's (2024-02-29) 03:13 CST build and then reupload those manually to https://anaconda.org/scientific-python-nightly-wheels/scipy/files. Then once the building workflow is fixed and you trigger it again these older wheels will get overwritten.

If this is a good idea in your mind but are busy enough with trying to fix this I can do this, like I did for the NumPy wheels earlier today.

[matthewfeickert]

Given feedback from @stefanv I've uploaded the 2024-02-29 wheels from GitHub Actions, but the aarch64 and arm64 wheels are built elsewhere and so still need to be uploaded. If someone can direct me to where those are built I can see if I can still get the 2024-02-29 artifacts and upload them (they get uploaded via @tupui's token, so he could probably tell me).

edit: Ah those wheels are built on Cirrus CI.

@tupui On the Cirrus CI main branch logs https://cirrus-ci.com/github/scipy/scipy/main I found this 2024-02-17 build (https://cirrus-ci.com/build/5758922498244608) and uploaded those artifacts to https://anaconda.org/scientific-python-nightly-wheels/scipy/files to overwrite the ones uploaded on 2024-03-01 that were built against the wrong numpy.

Once this current build issue gets fixed, can you please manually trigger a new build on Cirrus CI to overwrite the 2024-02-17 wheels?

[andyfaff]

https://cirrus-ci.com/build/6417540230938624

[matthewfeickert]

https://cirrus-ci.com/build/6417540230938624

@andyfaff My understanding is that is the build that was built against the wrong numpy (numpy/numpy#25907 (comment)).

[andyfaff]

The macos_arm64 nightlies are specified to be built against numpy>=2.0.0.dev0, and have been for a while.

[matthewfeickert]

Ah excellent! Thanks very much for pointing this out @andyfaff! I had missed that the 2024-03-01 00:02 was UTC, and so the timing of that build was before the 2024-03-01 02:05 UTC scientific-python/upload-nightly-action removal job (obvious in text, yes, but that's me writing it out knowing both times now and not being shown CST times on screen) that removed the numpy wheels and broke things. So my apologies, as I overwrote those aarch64 and arm64 wheels for no reason. 😬

I've gone ahead and manually downloaded the 2024-03-01 00:02 UTC wheels and uploaded them to overwrite things again, and so the aarch64 and arm64 wheels that were there at the start of 2024-03-01 are now back up and we're back to where we started. 👍

Example on Apple silicon of all good post upload:

% micromamba env create --name scipy-numpy-version-check --yes python=3.12 && micromamba activate scipy-numpy-version-check
% python -m pip install --pre --upgrade --index-url https://pypi.anaconda.org/scientific-python-nightly-wheels/simple scipy
Looking in indexes: https://pypi.anaconda.org/scientific-python-nightly-wheels/simple
Collecting scipy
  Downloading https://pypi.anaconda.org/scientific-python-nightly-wheels/simple/scipy/1.13.0.dev0/scipy-1.13.0.dev0-cp312-cp312-macosx_12_0_arm64.whl (31.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 31.2/31.2 MB 33.5 MB/s eta 0:00:00
WARNING: Cache entry deserialization failed, entry ignored
Collecting numpy>=1.22.4 (from scipy)
  Downloading https://pypi.anaconda.org/scientific-python-nightly-wheels/simple/numpy/2.0.0.dev0/numpy-2.0.0.dev0-cp312-cp312-macosx_14_0_arm64.whl (5.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.1/5.1 MB 35.0 MB/s eta 0:00:00
Installing collected packages: numpy, scipy
Successfully installed numpy-2.0.0.dev0 scipy-1.13.0.dev0
% python -c "import scipy.special"  # no 'A module that was compiled using NumPy 1.x...', yay!

I will stop touching the wheels now unless I get explicit requests from a SciPy dev to help with something. But it seems that everything should be good in the interim while the wheel building workflow gets fixed. 👍

The macos_arm64 nightlies are specified to be built against numpy>=2.0.0.dev0, and have been for a while.

Also, thanks @andyfaff, for securing that in PR #19217, as that's nice to know the aarch64 and arm64 wheel builds are already guarded! 🚀

[rgommers]

Fixed in main; wheels are rebuilding in https://github.com/scipy/scipy/actions/runs/8125502787 now.

Thanks @matthewfeickert and @andyfaff for dealing with the missing wheels.