Failed to build OpenBLAS on skylake-x target.

I can't build OpenBLAS with gcc-10.0.1 .
This happened just recently (after thursday). and I don't know if it is bug with gcc or openblas.

In file included from /OpenBLAS/kernel/x86_64/sgemm_kernel_16x4_skylakex_2.c:380,
                 from /OpenBLAS/build/kernel/CMakeFiles/sgemm_kernel.c:7:
/OpenBLAS/kernel/x86_64/sgemm_direct_skylakex.c: In function 'sgemm_kernel_direct':
/OpenBLAS/kernel/x86_64/sgemm_direct_skylakex.c:16:40: warning: AVX512F vector return without AVX512F enabled changes the ABI [-Wpsabi]
   16 | #define DECLARE_RESULT_512(N,M) __m512 result##N##M = _mm512_setzero_ps()
      |                                        ^~~~~~
/OpenBLAS/kernel/x86_64/sgemm_direct_skylakex.c:162:4: note: in expansion of macro 'DECLARE_RESULT_256'
  162 |    DECLARE_RESULT_256(0, 0);
      |    ^~~~~~~~~~~~~~~~~~
In file included from /usr/local/lib/gcc/x86_64-linux-gnu/10.0.1/include/immintrin.h:55,
                 from /OpenBLAS/kernel/x86_64/sgemm_kernel_16x4_skylakex_2.c:379,
                 from /OpenBLAS/build/kernel/CMakeFiles/sgemm_kernel.c:7:
/usr/local/lib/gcc/x86_64-linux-gnu/10.0.1/include/avx512fintrin.h:310:1: error: inlining failed in call to 'always_inline' '_mm512_setzero_ps': target specific option mismatch
  310 | _mm512_setzero_ps (void)
      | ^~~~~~~~~~~~~~~~~
In file included from /OpenBLAS/kernel/x86_64/sgemm_kernel_16x4_skylakex_2.c:380,
                 from /OpenBLAS/build/kernel/CMakeFiles/sgemm_kernel.c:7:
/OpenBLAS/kernel/x86_64/sgemm_direct_skylakex.c:16:55: note: called from here
   16 | #define DECLARE_RESULT_512(N,M) __m512 result##N##M = _mm512_setzero_ps()
      |                                                       ^~~~~~~~~~~~~~~~~~~

this is just a part of the error messages that I got.

[martin-frbg]

This rather looks as if the -march=skylake-avx512 flag was not supplied to gcc (should be set automatically in Makefile.x86_64 when the c_check script detected AVX512 capability). I do not think the single change committed last thursday has anything to do with it, perhaps your snapshot of gcc10 mishandles the skylake-avx512 flag and/or needs an additional -mavx512f option to enable AVX512F. (Or you built an older version of OpenBLAS by mistake, there were some problems with autodetection and option setting for SkylakeX in early summer when I did not yet have the hardware.)

Just to clarify.
That issue comes from GitHub Actions where I did configure it to run every Monday and Thursday to build a docker image that contains gcc-trunk, OpenBLAS-develop and some other tools.
The image was successfully built until this morning, when it gives me those error messages.
May be the bug is related to gcc-trunk.
My repository : https://github.com/ChinouneMehdi/debian-gfortran-testing
Build log : https://github.com/ChinouneMehdi/debian-gfortran-testing/runs/409938240

[wjc404]

@ChinouneMehdi Maybe this modified version of sgemm_direct_skylakex.c can work (see below). For trying, please change the suffix to ".c", replace the old "sgemm_direct_skylakex.c" in /OpenBLAS/kernel/x86_64 with this file and recompile.
sgemm_direct_skylakex.txt

@wjc404 It builds now successfully (with your modified file).
Thanks.

[martin-frbg]

Interesting, looks like gcc just got pickier overnight ? (That file was last touched two weeks ago to remove a redundant #include, the actual lines of code that wjc404 just fixed for you have been there for a few months)

[brada4]

target specific option mismatch

Thats when gcc cannot emit assembly output due to uncommon combo of ISA exts flags like -mmmx
It is not per-se code problem, though lightly shuffling it may work better (say removing all -m flags)

It failed this morning, even with the modified file!!
Maybe because GitHub Actions not using the same CPU each time.
https://help.github.com/en/actions/automating-your-workflow-with-github-actions/virtual-environments-for-github-hosted-runners
https://docs.microsoft.com/en-us/azure/virtual-machines/windows/sizes-general#dsv2-series

[martin-frbg]

Do you have access to the build logs ? Even if the build environment changes to a non-AVX512 (Broadwell or Haswell generation) host from time to time, the build system should choose the appropriate target automatically (unless the switch happens in the middle of a build, of course)

It fails with Skylake-X
fail.log
and It succeeds with Hasewell
success.log

[martin-frbg]

Success with Haswell is understandable as that will not even try to use AVX512, not sure what happens in the Skylake-X build. The build system appears to have been content with the AVX512 capability of both hardware and compiler (else it would have changed the target to Haswell), and It seems to compile the daxpy and sdot microkernels which also have AVX512 instructions (though protected by an ifdef __AVX512CD__) just fine.
I do not notice anything suspicious in the recent gcc change log (unless it is a weird side effect of Jakub Jelinek's "i386: prefer vpermilpd over vpermpd [PR93395]" from friday - at least this would be one something changed "after thursday").

Sorry to not verifying before reporting, but it fails with gcc-8.3.0, gcc-9.2.1 and gcc-trunk with skylake-x target. ( I was using OpenBLAS from distribution repositories with gcc-8.3.0 and gcc-9.2.1 ).
gcc-8.3.0 :
fail_8.3.0.txt
gcc-9.2.1 :
fail_9.2.0.txt

[brada4]

Probably -m option combination problem https://www.felixcloutier.com/x86/vpermilpd

[martin-frbg]

Passes with 9.2.0 (and default build options) on my SkylakeX, do you set any special CFLAGS for compilation ?

No, I didn''t set any CFLAGS.
Take a look https://github.com/ChinouneMehdi/OpenBLAS/tree/bug
https://github.com/ChinouneMehdi/OpenBLAS/blob/bug/.github/workflows/testing.yml