Compiling numpy 1.17 on cygwin fails with `Error: invalid register for .seh_savexmm

[DWesl]

Attempting to install numpy 1.17.3 on cygwin fails, with the only reported error message being invalid register for .seh_savexmm.
I can't figure out where this would be getting called, since the reported file is standard input and grepping the source for savexmm does not produce any obvious clues.

Reproducing code example:

python3 -m pip install numpy==1.17.3

Error message:

Error message on third and subsequent compilations

...
gcc: build/src.cygwin-3.0.7-x86_64-3.5/numpy/core/src/umath/loops.c
{standard input}: Assembler messages:
{standard input}:4612: Error: invalid register for .seh_savexmm
{standard input}:4614: Error: invalid register for .seh_savexmm
{standard input}:4616: Error: invalid register for .seh_savexmm
{standard input}:4618: Error: invalid register for .seh_savexmm
{standard input}:4620: Error: invalid register for .seh_savexmm
{standard input}:4622: Error: invalid register for .seh_savexmm
{standard input}:6493: Error: invalid register for .seh_savexmm
{standard input}:6495: Error: invalid register for .seh_savexmm
{standard input}:6497: Error: invalid register for .seh_savexmm
{standard input}:6499: Error: invalid register for .seh_savexmm
{standard input}:6501: Error: invalid register for .seh_savexmm
{standard input}:6503: Error: invalid register for .seh_savexmm
{standard input}:6505: Error: invalid register for .seh_savexmm
numpy/core/src/umath/loops.c.src: In function ‘PyUFunc_e_e’:
numpy/core/src/umath/loops.c.src:77:1: warning: visibility attribute not supported in this configuration; ignored [-Wattributes]
 }
 ^
...
numpy/core/src/umath/loops.c.src: In function ‘OBJECT_sign’:
numpy/core/src/umath/loops.c.src:2919:1: warning: visibility attribute not supported in this configuration; ignored [-Wattributes]
 }
 ^
error: Command "gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -ggdb -O2 -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fstack-protector-strong --param=ssp-buffer-size=4 -fdebug-prefix-map=/usr/src/ports/python36/python36-3.6.9-1.x86_64/build=/usr/src/debug/python36-3.6.9-1 -fdebug-prefix-map=/usr/src/ports/python36/python36-3.6.9-1.x86_64/src/Python-3.6.9=/usr/src/debug/python36-3.6.9-1 -ggdb -O2 -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fstack-protector-strong --param=ssp-buffer-size=4 -fdebug-prefix-map=/usr/src/ports/python36/python36-3.6.9-1.x86_64/build=/usr/src/debug/python36-3.6.9-1 -fdebug-prefix-map=/usr/src/ports/python36/python36-3.6.9-1.x86_64/src/Python-3.6.9=/usr/src/debug/python36-3.6.9-1 -DNPY_INTERNAL_BUILD=1 -DHAVE_NPY_CONFIG_H=1 -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE=1 -D_LARGEFILE64_SOURCE=1 -DNO_ATLAS_INFO=1 -DHAVE_CBLAS -I/usr/local/include -I/usr/include -I/usr/include/suitesparse -Ibuild/src.cygwin-3.0.7-x86_64-3.6/numpy/core/src/umath -Ibuild/src.cygwin-3.0.7-x86_64-3.6/numpy/core/src/npymath -Ibuild/src.cygwin-3.0.7-x86_64-3.6/numpy/core/src/common -Inumpy/core/include -Ibuild/src.cygwin-3.0.7-x86_64-3.6/numpy/core/include/numpy -Inumpy/core/src/common -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/usr/include/python3.6m -Ibuild/src.cygwin-3.0.7-x86_64-3.6/numpy/core/src/common -Ibuild/src.cygwin-3.0.7-x86_64-3.6/numpy/core/src/npymath -Ibuild/src.cygwin-3.0.7-x86_64-3.6/numpy/core/src/common -Ibuild/src.cygwin-3.0.7-x86_64-3.6/numpy/core/src/npymath -c build/src.cygwin-3.0.7-x86_64-3.6/numpy/core/src/umath/loops.c -o build/temp.cygwin-3.0.7-x86_64-3.6/build/src.cygwin-3.0.7-x86_64-3.6/numpy/core/src/umath/loops.o -MMD -MF build/temp.cygwin-3.0.7-x86_64-3.6/build/src.cygwin-3.0.7-x86_64-3.6/numpy/core/src/umath/loops.o.d" failed with exit status 1
...

The first and second compilations produce different output before the block of errors, but are otherwise identical.

Numpy/Python version information:

Compiling numpy 1.17.3 on python 3.5.7 fails as described above.

Compiling numpy 1.17.0 on python 3.7.4 fails in the manner described above.
Numpy 1.16.5 built just fine and passed most tests.

How would I go about debugging this?

[charris]

Sounds like a hardware detection or compiler problem. What GCC version do you have? Are you running 32 or 64 bits?

[DWesl]

64-bit Cygwin with GCC 8.3.0.

[DWesl]

Numpy 1.17.3 installs fine on 32-bit Cygwin with gcc 7.4.0 and python 3.6.9

[DWesl]

Installing numpy 1.17.3 from source with pip on 64-bit WSL-debian using python 3.7.3 and gcc 8.3.0 worked fine.
All trials here were on the same machine.

[DWesl]

Running git bisect points to the following commit:

651e03c0019d4c4c6ca8c43cb7d7c0d344a72cc1 is the first bad commit
commit 651e03c0019d4c4c6ca8c43cb7d7c0d344a72cc1
Author: Raghuveer Devulapalli
Date:   Thu Mar 28 15:07:35 2019 -0700

    BUG: Adding macro HAVE_ATTRIBUTE_TARGET_@ISA@_WITH_INTRINSICS

    1) use __builtin_cpu_supports("avx512f") only for gcc ver >= 5

    2) Introduced two new macro's:
    HAVE_ATTRIBUTE_TARGET_@ISA@_WITH_INTRINSICS for ensuring compiler can
    compile functions that use intrinsics and are compiled with avx2/avx512f
    attributes

:040000 040000 17c469bd32baf93f8306d76ea3d7bcec9b98f286 5c129001b713ed2edf09f2e98c8c0fa61316d985 M      numpy
bisect run success

I restricted attention to numpy/core/src/umath/loops.c.src, so it might also be a tad before this commit.

[charris]

What is your cpu? I think this is a compiler/configuration problem. Are you running in a virtual environment by any chance.

[DWesl]

Four commits before that one works; three before does not.
That is, it would appear 9754a20 is the first to fail in this way.
I don't know what to do next.

[DWesl]

No virtual environment, Intel core i5. /proc/cpuinfo lists avx2 but not avx512f. I think the chip is 6th generation/Skylake. Manually running the probes for avx2 and avx512f target attribute support both succeed.

[mattip]

In order to freeze the build, you can do this, the -b should then have the results of the failed build.

python3 -m pip install numpy==1.17.3 -b /tmp/numpy

This stack overflow suggests adding -fno-asynchronous-unwind-tables. You could try just running the line that fails once with that flag and once without

[DWesl]

Thank you, -fno-asynchronous-unwind-tables fixed it. I'll report back once the tests have run.

[mattip]

According to gcc bug report, it has not been fixed. We could add the flag unconditionally when cygwin is detected, but then it would get lost inside distutils and we would not know to update it when gcc fixes the problem.

[eric-wieser]

Should we add a config test for the need for this flag, which is what I did last time I got a compiler bug?

[mattip]

sure, maybe we should have done that with the -std=c99 fix as well. It is a flag, not a "HAVE" test. Where would we add that?

[eric-wieser]

Hmm, I hadn't thought of that. This is the change I'd made before I was thinking about: https://github.com/numpy/numpy/pull/13739/files#diff-33315af1a22893e97676ee134db5ce1cR466-R471

[mattip]

We could some new moredefs.append('NPY_DO_NOT_OPTIMIZE_XXX'). While I don't think we should add a cygwin Azure run, maybe I can set up an environment somewhere

[DWesl]

NumPy 1.17.5 is compiling and has no new test failures compared to 1.16.6. I did have to move a few "#include <Python.h>" lines before system includes so it could define "_GNU_SOURCE" before "#include"ing system headers. I will submit those as a separate pull request if I still need to do that for "master".

[DWesl]

I got 1.18.4 to pass all tests expected, with

CFLAGS="-fno-asynchronous-unwind-tables
-ffixed-xmm16 -ffixed-xmm17 -ffixed-xmm18 -ffixed-xmm19
-ffixed-xmm20 -ffixed-xmm21 -ffixed-xmm22 -ffixed-xmm23
-ffixed-xmm24 -ffixed-xmm25 -ffixed-xmm26 -ffixed-xmm27
-ffixed-xmm28 -ffixed-xmm29 -ffixed-xmm30 -ffixed-xmm31"

I think I also tested it with just the -ffixed-xmm16-30 flags, and that also worked.
I'll get my environment set up to submit the patches for that to work: I'm not sure where I would add the flags.

[DWesl]

I forgot to pass the flags when testing a PR, so apparently this is gone. The GCC bug report above now says "fixed", so I'm not sure whether it's GCC or NumPy that did something to make this go away.

Current NumPy master (will be 1.19) on 64-bit Cygwin 3.1.4 with GCC 9.3.0.

1.17.3 compiles just fine now as well, so I'm suspecting this is a GCC fix.

[DWesl]

This is still an issue on older GCC (<8.4), see #16290.

[DWesl]

Given #17548 and friends, I think this issue is resolved in trunk and a few older release branches, so it's time to close this issue.