Skip to content

Conversation

@mattip
Copy link
Collaborator

@mattip mattip commented Oct 30, 2025

  • I updated the package version in pyproject.toml and made sure the first 3 numbers match git describe --tags --abbrev=8 in OpenBLAS at the OPENBLAS_COMMIT. If I did not update OPENBLAS_COMMIT, I incremented the wheel build number (i.e. 0.3.29.0.0 to 0.3.29.0.1)

Towards numpy/numpy#5520, scipy/scipy#23686. The last commit in OpenBLAS should fix them.

@mattip
Copy link
Collaborator Author

mattip commented Oct 30, 2025

Grrr. There are build failures:

  • arm64 manylinux is failing tests with SBGEMM FAILURES: 28800, arm64 musllinux is passing. Strange.
  • windows x64 is failing when trying to move the wheel, the names scipy_openblas64-0.3.30.322.0-py3-none-win_amd64.whl are now the same. Somehow the naming change since use gfortran-11 on macos-x86_64 build #225?

# move the mis-named scipy_openblas64-none-any.whl to a platform-specific name
for f in dist/*.whl; do mv $f "${f/%any.whl/$WHEEL_PLAT.whl}"; done

  • win-arm64 is failing. The build log is (like on previous builds) huge, with many warning: macro name is a reserved identifier [-Wreserved-macro-identifier] and warning: unsafe pointer arithmetic [-Wunsafe-buffer-usage] warnings. It is so big I cannot get to the end to see the error.

@mattip
Copy link
Collaborator Author

mattip commented Oct 31, 2025

The win-arm64 builds which use CMake are failing somewhere around the link step (it is difficult to isolate which command since CMake has a POST_BUILD command with gensymbol.pl, building dllnit.c, and then linking) with

lld-link: error: <root>: undefined symbol: shgemm_
lld-link: error: <root>: undefined symbol: shgemv_
lld-link: error: <root>: undefined symbol: cblas_shgemm

I think this is due to some missing name mangling. We set -DSYMBOLPREFIX="scipy_" -DLIBNAMEPREFIX="scipy_" but I don't see scipy_ in those symbol names. This is new to latest develop, v0.3.30 succeeded. @Harishmcw as our resident windows-arm64 expert, do you have any idea what is going on and why there are so many warnings in the windows-arm64 build? here is the latest run, note I added set CFLAGS=-Wno-reserved-macro-identifier -Wno-unsafe-buffer-usage to cut the spurious warnings down by a lot.

@Harishmcw
Copy link
Contributor

Harishmcw commented Nov 1, 2025

Hi @mattip,

I've opened OpenMathLib/OpenBLAS#5524 to track this.

TL;DR: The generated .def file includes shgemm_, shgemv_, and cblas_shgemm symbols even when BUILD_HFLOAT16=OFF. Since these are only compiled when that flag is ON, the linker fails.

@mattip
Copy link
Collaborator Author

mattip commented Nov 2, 2025

I tried adding the patch from upstream to fix win-arm64, but the batch file did not actually use it. I opened #228 to refactor the build on win-arm64 and added patching.

@mattip
Copy link
Collaborator Author

mattip commented Nov 2, 2025

OK, down to the aarch64-manylinux failure. That one is weird: musllinux passes, manylinux fails.

@mattip
Copy link
Collaborator Author

mattip commented Nov 2, 2025

The manylinux2014 image is using gcc-10 (and gfortran-10). The musllinux_1_2 image uses gcc/gfortran 14. gcc-10 is really out of date. FWIW, NumPy has dropped manylinux2014 in favor of manylinux_2_28 (manylinux2014 is equivalent to manylinux_2_17). Scipy still is releasing manylinux2014 wheels. We could

  • use manylinux_2_28 for aarch64. This would then preclude using the updated wheel on scipy until they change their policy
  • try to update gcc on the aarch64 manylinux2014 image. This will probably fail, according to this comment in the manylinux repo.
  • Try to figure out which kernel is failing and disable it. I see on the NumPy workflow that the runner machines support all the possible SIMD extensions "NEON", "NEON_FP16", "NEON_VFPV4", "ASIMD", "ASIMDHP", "ASIMDDP", "ASIMDFHM", "SVE". I tried limiting the aarch64 kernels. We had started something like this in order to limit the package size in Shrink aarch64 wheels #170 but decided the ROI was not worth the effort.

@mattip
Copy link
Collaborator Author

mattip commented Nov 2, 2025

Cool. Limiting the aarch64-manylinux2014 build to ARMV8 only "solved" the problem.

@mattip mattip mentioned this pull request Nov 2, 2025
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants