Failed to build on Windows & AMD CPU

[moderato]

Hi there, I'm trying to build OpenBLAS on a Windows machine with AMD CPU on Anaconda Prompt but failed due to a bug probably related to BF16. Anyone knows how to fix this issue?

Related issue (run on WSL): #4673

Error message:

  sbstobf16_k.c
C:\Users\moderato\Documents\repos\OpenBLAS\kernel\x86_64\tobf16.c(130,63): error C1001: Internal
 compiler error. [C:\Users\moderato\Documents\repos\OpenBLAS\build\kernel\kernel.vcxproj
]
C:\Users\moderato\Documents\repos\OpenBLAS\kernel\x86_64\tobf16.c(130,63): error C1001: (compile
r file 'D:\a\_work\1\s\src\vctools\Compiler\CxxFE\sl\p1\c\symbols.c', line 23601) [C:\Users\moderato\Documents\repos\Op
enBLAS\build\kernel\kernel.vcxproj]
C:\Users\moderato\Documents\repos\OpenBLAS\kernel\x86_64\tobf16.c(130,63): error C1001:  To work
 around this problem, try simplifying or changing the program near the locations listed above. [C:\Users\moderato\Docum
ents\repos\OpenBLAS\build\kernel\kernel.vcxproj]
C:\Users\moderato\Documents\repos\OpenBLAS\kernel\x86_64\tobf16.c(130,63): error C1001: If possi
ble please provide a repro here: https://developercommunity.visualstudio.com  [C:\Users\moderato\Documents\repos\OpenBL
AS\build\kernel\kernel.vcxproj]
C:\Users\moderato\Documents\repos\OpenBLAS\kernel\x86_64\tobf16.c(130,63): error C1001: Please c
hoose the Technical Support command on the Visual C++  [C:\Users\moderato\Documents\repos\OpenBL
AS\build\kernel\kernel.vcxproj]
C:\Users\moderato\Documents\repos\OpenBLAS\kernel\x86_64\tobf16.c(130,63): error C1001:  Help me
nu, or open the Technical Support help file for more information [C:\Users\moderato\Documents\repos\3rdpa
rty\OpenBLAS\build\kernel\kernel.vcxproj]

System info:

Architecture:            x86_64
  CPU op-mode(s):        32-bit, 64-bit
  Address sizes:         48 bits physical, 48 bits virtual
  Byte Order:            Little Endian
CPU(s):                  12
  On-line CPU(s) list:   0-11
Vendor ID:               AuthenticAMD
  Model name:            AMD Ryzen 5 7640HS w/ Radeon 760M Graphics
    CPU family:          25
    Model:               116
    Thread(s) per core:  2
    Core(s) per socket:  6
    Socket(s):           1
    Stepping:            1
    BogoMIPS:            8583.32
    Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse ss
                         e2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl tsc_reliable nons
                         top_tsc cpuid extd_apicid pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx
                          f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch o
                         svw topoext perfctr_core ssbd ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid
                          avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb avx512cd sha_ni avx512bw avx512vl
                         xsaveopt xsavec xgetbv1 xsaves avx512_bf16 clzero xsaveerptr arat npt nrip_save tsc_scale vmcb_
                         clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload avx512vbmi umip avx512_
                         vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq rdpid
Virtualization features:
  Virtualization:        AMD-V
  Hypervisor vendor:     Microsoft
  Virtualization type:   full
Caches (sum of all):
  L1d:                   192 KiB (6 instances)
  L1i:                   192 KiB (6 instances)
  L2:                    6 MiB (6 instances)
  L3:                    16 MiB (1 instance)
Vulnerabilities:
  Gather data sampling:  Not affected
  Itlb multihit:         Not affected
  L1tf:                  Not affected
  Mds:                   Not affected
  Meltdown:              Not affected
  Mmio stale data:       Not affected
  Retbleed:              Not affected
  Spec rstack overflow:  Mitigation; safe RET, no microcode
  Spec store bypass:     Mitigation; Speculative Store Bypass disabled via prctl and seccomp
  Spectre v1:            Mitigation; usercopy/swapgs barriers and __user pointer sanitization
  Spectre v2:            Mitigation; Retpolines, IBPB conditional, IBRS_FW, STIBP conditional, RSB filling, PBRSB-eIBRS
                         Not affected
  Srbds:                 Not affected
  Tsx async abort:       Not affected

Build command:

cmake .. -G "Ninja" -DCMAKE_CXX_COMPILER=clang-cl -DCMAKE_C_COMPILER=clang-cl -DCMAKE_Fortran_COMPILER=flang -DCMAKE_MT=mt -DBUILD_BFLOAT16=ON -DBUILD_WITHOUT_LAPACK=yes -DNOFORTRAN=1 -DDYNAMIC_ARCH=ON -DCMAKE_BUILD_TYPE=Release
cmake --build . --config Release
cmake --install . -v

[martin-frbg]

which version of LLVM (and MSVC) is this - can you try a different (newer) version ? I assume you will not want to build without the BF16 code...

[moderato]

which version of LLVM (and MSVC) is this - can you try a different (newer) version ? I assume you will not want to build without the BF16 code...

Thank you for the reply. My clang version is 18.1.1 and MSVC version is 2022 community. And you're right BF16 is all I need...

[martin-frbg]

Well, internal compiler error is just that, and I do not see anything around line 130 of tobf16.c that could be "expected" to break a compiler - just the declaration of an external function that appears in several other files as well. The only moderately odd thing in the file is that at the very end, in the call to this function blas_level1_thread(), there is a spurious cast to (void *). I guess you could try removing that cast...

[moderato]

Somehow I couldn't reproduce this bug on my machine... But I'll try your solution once it comes up again. Now I got a new bug here (for znrm2.c and a couple of other routines):

In file included from C:\Users\moderato\Documents\repos\OpenBLAS\build\windows\interface\CMakeFiles\znrm2.c:9:
In file included from C:/Users/moderato/Documents/repos/OpenBLAS/interface/nrm2.c:40:
C:\Users\moderato\Documents\repos\OpenBLAS\common.h(95,10): fatal error: 'sched.h' file not found
   95 | #include <sched.h>
      |          ^~~~~~~~~

[martin-frbg]

Strange if problems keep coming and going on that machine. sched.h should come with your libc I think, perhaps in a separate "dev" or "devel" package.

[moderato]

I'm a bit confused here. The original bug came up as I tried to build the OpenBLAS in a conda env that I already have. Then I realized I might need to try rebuilding it in a clean conda env from scratch. So I tried by following this page with a brand new conda env and MSVC 2022 pre-installed and it started to throw me bugs of sched.h and sys/mman.h. I searched a bit and some online answers said both these files are Linux files and shouldn't appear on Windows, but I'm not sure about this.

I might have missed some settings here that lead to the bug. Could you please try to follow the Windows installation instructions above and let me know if it goes through?

[martin-frbg]

The inclusion of "sched.h" by common.h only occurs when the operating system is assumed to be Linux, Android, MacOS or one of the BSD unix variants. This scenario would basically require that clang-cl -E ctest.c returns something other than "OS_WINNT" so that the contents of the generated config.h become misleading. (Or you somehow managed to reuse the config.h that was generated by the WSL build, e.g. by sharing the same source directory between WSL and Conda).
The build process for Windows should still work as described, and I may be able to verify this for LLVM18 (again) - what I cannot do at the moment is test Ryzen4 or Cooperlake hardware under Windows.

[moderato]

(Or you somehow managed to reuse the config.h that was generated by the WSL build, e.g. by sharing the same source directory between WSL and Conda).

GOOD CATCH! In fact this is exactly the cause as I did share the source repo and config.h and Makefile.conf don't get regenerated when I switch platforms. I delete them and rebuild OpenBLAS and it all goes through well. The original bug doesn't come up again.

(Here sbgemm_kernel.c is still built with generic 2x2 though, but we'll continue the discussion in the other issue.)

Thank you so much Martin!