PPC build fails on 10.6.8 Rosetta: unsupported cflag used, lowlevel_strided_loops failure

[barracuda156]

Describe the issue:

py310-numpy fails to build on 10.6.8 Rosetta. There are two errors:

1. Usage of -march=native, which is unsupported on PPC: -march=native option not supported on powerpc mfem/mfem#216

:info:build extra options: '-march=native'
:info:build WARN: CCompilerOpt.dist_test[630] : CCompilerOpt._dist_test_spawn[764] : Command (/opt/local/bin/gcc-mp-12 -Wno-unused-result -Wsign-compare -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -pipe -Os -arch ppc -isysroot/ -Inumpy/core/src/common -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/src/_simd -I/opt/local/Library/Frameworks/Python.framework/Versions/3.10/include/python3.10 -Ibuild/src.macosx-10.6-ppc-3.10/numpy/core/src/common -Ibuild/src.macosx-10.6-ppc-3.10/numpy/core/src/npymath -c /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py310-numpy/work/numpy-1.23.1/numpy/distutils/checks/test_flags.c -o /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py310-numpy/work/.tmp/tmp6lznyx3c/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py310-numpy/work/numpy-1.23.1/numpy/distutils/checks/test_flags.o -MMD -MF /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py310-numpy/work/.tmp/tmp6lznyx3c/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py310-numpy/work/numpy-1.23.1/numpy/distutils/checks/test_flags.o.d -march=native) failed with exit status 1 output -> 
:info:build gcc-mp-12: error: unrecognized command-line option '-march=native'; did you mean '-mcpu=native'?
:info:build WARN: CCompilerOpt.cc_test_flags[1077] : testing failed

2. lowlevel_strided_loops.c failure, not sure for which reason:

error: Command "/opt/local/bin/gcc-mp-12 -Wno-unused-result -Wsign-compare -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -pipe -Os -arch ppc -isysroot/ -DNPY_INTERNAL_BUILD=1 -DHAVE_NPY_CONFIG_H=1 -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE=1 -D_LARGEFILE64_SOURCE=1 -DHAVE_CBLAS -Ibuild/src.macosx-10.6-ppc-3.10/numpy/core/src/multiarray -Ibuild/src.macosx-10.6-ppc-3.10/numpy/core/src/common -Ibuild/src.macosx-10.6-ppc-3.10/numpy/core/src/umath -Inumpy/core/include -Ibuild/src.macosx-10.6-ppc-3.10/numpy/core/include/numpy -Ibuild/src.macosx-10.6-ppc-3.10/numpy/distutils/include -Ibuild/src.macosx-10.6-ppc-3.10/numpy/core/src/npysort -Inumpy/core/src/common -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/src/_simd -I/opt/local/Library/Frameworks/Python.framework/Versions/3.10/include/python3.10 -Ibuild/src.macosx-10.6-ppc-3.10/numpy/core/src/common -Ibuild/src.macosx-10.6-ppc-3.10/numpy/core/src/npymath -c build/src.macosx-10.6-ppc-3.10/numpy/core/src/multiarray/lowlevel_strided_loops.c -o build/temp.macosx-10.6-ppc-cpython-310/build/src.macosx-10.6-ppc-3.10/numpy/core/src/multiarray/lowlevel_strided_loops.o -MMD -MF build/temp.macosx-10.6-ppc-cpython-310/build/src.macosx-10.6-ppc-3.10/numpy/core/src/multiarray/lowlevel_strided_loops.o.d" failed with exit status 1

Reproduce the code example:

Well, it is a build error. Provided you have 10.6.8 with Rosetta and build for `ppc`, running `sudo port -v install py310-numpy` will lead to the error reported.

Error message:

Build fails with:

:info:build Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py310-numpy/work/numpy-1.23.1" && /opt/local/Library/Frameworks/Python.framework/Versions/3.10/bin/python3.10 setup.py --no-user-cfg build -j6 
:info:build Exit code: 1
:error:build Failed to build py310-numpy: command execution failed
:debug:build Error code: CHILDSTATUS 56631 1
:debug:build Backtrace: command execution failed
:debug:build     while executing
:debug:build "system {*}$notty {*}$callback {*}$nice $fullcmdstring"
:debug:build     invoked from within
:debug:build "command_exec -callback portprogress::target_progress_callback build"
:debug:build     (procedure "portbuild::build_main" line 8)
:debug:build     invoked from within
:debug:build "$procedure $targetname"

### NumPy/Python version information:

:debug:main Starting logging for py310-numpy @1.23.1_0+gfortran+openblas
:debug:sysinfo Mac OS X 10.6.8 (darwin/10.8.0) arch powerpc
:debug:sysinfo MacPorts 2.7.2
:debug:sysinfo Xcode 3.2.6
:debug:sysinfo SDK 10.6
:debug:sysinfo MACOSX_DEPLOYMENT_TARGET: 10.6

[rgommers]

Thanks for the report @barracuda156. Do I understand it right that you are trying to build on macOS 10.6 for PowerPC? That seems unsupported by now, it's like 15 years old hardware and OS.

[barracuda156]

Thanks for the report @barracuda156. Do I understand it right that you are trying to build on macOS 10.6 for PowerPC? That seems unsupported by now, it's like 15 years old hardware and OS.

@rgommers Yeah, but Macports support all OSs starting from Tiger (10.4).

This problem seems specific to Rosetta, apparently the build system gets some setting wrong (confusing cpu_arch for build_arch). From what I recall, I have built py310-numpy on 10.5.8 and 10.6 PPC (10A190).

P. S. I opened the issue to get some suggestions, not that I expect a ready solution. I will also try fixing it on my side in the meanwhile.

[rgommers]

CCompilerOpt is fairly new, I think from well after we dropped PowerPC support for macOS. I think you'll have to patch out all the failing SIMD-related flags until things work.

[barracuda156]

The bug is still there in py311-numpy.

@rgommers -march= flag should never be used on PPC, regardless of the OS. It is not supported for the arch itself, be it PPC7400 or POWER9.

[rgommers]

@barracuda156 we're not going to solve that. This is basically unsupported usage, macOS 10.6 was the last OS version to support PowerPC (see https://en.wikipedia.org/wiki/MacOS_version_history).

The way we can close this issue is to raise a clearer error at the start of the build, to avoid the confusing build failure later on.

[barracuda156]

@barracuda156 we're not going to solve that. This is basically unsupported usage, macOS 10.6 was the last OS version to support PowerPC (see https://en.wikipedia.org/wiki/MacOS_version_history).

The way we can close this issue is to raise a clearer error at the start of the build, to avoid the confusing build failure later on.

This can trivially be fixed if you fix your compilation flags. It is not Darwin-specific. On any POWER/PowerPC cpu -march= cannot be used; -mtune= should be used instead.
There is literally nothing about OS here. Whenever the build arch is any suptype of PPC, native flag is -mtune=.

P. S. Please do not break it further :)
If I find a neat fix and confirm it working, I will make a PR.

[rgommers]

The distutils and numpy.distutils build flags are all hardcoded for macOS per architecture (and yes that's bad - the whole build system is obsolete and will be removed soon). You won't see this problem on other other OSes with PowerPC (we have CI for Linux on ppc64le).

[rgommers]

Note that we have now dropped the numpy.distutils based build for NumPy, and are using Meson as the build system. I expect it is getting the build flags for PowerPC on macOS right. If not, you can bypass it as a workaround with the -Ddisable-optimization flag to the configure stage of the build. E.g. when you're triggering the build with pip:

pip install numpy -Csetup-args=-Ddisable-optimization

This is probably fixed and if it isn't, the root cause is now elsewhere. Hence I will close this issue. If there's still a problem, please comment and we can reopen the issue.