CPU detection in distro build

[bmwiedemann]

Related to #667, I found that with pocl-1.4, there is again CPU detection happening on the build machine.

The build environment has several significant diffs:

+++ pocl-1.4/build/sleef_config_temp_avx.h    2019-11-19 10:17:59.409588738 +000
0
@@ -3,4 +3,3 @@
 #define SLEEF_DOUBLE_VEC_AVAILABLE
 #define SLEEF_VINT_IS_VLONG
 #define SLEEF_VEC_128_AVAILABLE
-#define SLEEF_VEC_256_AVAILABLE

+++ pocl-1.4/build/sleef_config_temp_avx_fma4.h       2019-11-19 10:21:44.401592562 +0000
@@ -3,8 +3,3 @@
 #define SLEEF_DOUBLE_VEC_AVAILABLE
 #define SLEEF_VINT_IS_VLONG
 #define SLEEF_VEC_128_AVAILABLE
-#define HAVE_FMA32_128
-#define HAVE_FMA64_128
-#define SLEEF_VEC_256_AVAILABLE
-#define HAVE_FMA32_256
-#define HAVE_FMA64_256

Our build call is around line 100 of https://build.opensuse.org/package/view_file/openSUSE:Factory/pocl/pocl.spec

This bug was found as part of my work on reproducible builds for openSUSE.

[franz]

I could not reproduce, but the those lines removed in the diff should definitely be there.

The checks for CPU features are done in lib/kernel/host/CMakeLists.txt around line 400,
by calling clang with arguments like

--target=x86_64-pc-linux-gnu -D_CL_DISABLE_HALF -march=haswell -emit-llvm -ffp-contract=off -DDORENAME -DVEC512

on the source file lib/kernel/sleef/test.c.

there is again CPU detection happening on the build machine.

If by "CPU detection happening" you mean CMake generating of sleef_config_* headers, that has been there since pocl 1.0 or so, and it's independent of the host CPU when building with -DKERNELLIB_HOST_CPU_VARIANTS=distro.

[bmwiedemann]

-#define HAVE_FMA32_128
those lines removed in the diff should definitely be there

even if I build on a machine/CPU like qemu64 that does not support the FMA instruction?

From what I saw in my test results, something changed there with 1.4 over 1.3

[franz]

even if I build on a machine/CPU like qemu64 that does not support the FMA instruction?

If you're building a distro build, then yes, wherever you build. As you can see the clang arguments:

-march=haswell

which means Clang should always compile for (and report features for) Intel Haswell. Unlike -mtune argument which compiles for some generic CPU with tuning for a specific CPU.

A pocl "distro" build precompiles several kernel libraries for different hardcoded CPU names with -march. The available features of those CPUs are detected like i described in previous comment.

something changed there with 1.4 over 1.3

It seems your Clang reports incorrect CPU extensions for some reason. Possibly a bug in Clang or something else, but doesn't seems to be a bug in pocl (so far).

For "avx" kernel library, we use -march=sandybridge, and for "avx_fma4", we use -march=bdver1.

Try this command in your build environment:

clang --target=x86_64-pc-linux-gnu -D_CL_DISABLE_HALF -march=sandybridge -emit-llvm -ffp-contract=off -DDORENAME -DVEC256 -c lib/kernel/sleef/test.c

If it prints an error, your Clang (or build environment) is broken.

[bmwiedemann]

I finally managed to test the above command and it does not print an error.

However, I also see

+++ /home/abuild/rpmbuild/BUILD/pocl-1.4/build/CMakeCache.txt	2035-04-29 21:36:36.660911810 +0000
@@ -412,7 +412,7 @@
 LINK_COMMAND:FILEPATH=/usr/bin/ld
 
 //The Host CPU to use with llc
-LLC_HOST_CPU:STRING=westmere
+LLC_HOST_CPU:STRING=pentium2
 
 //LLVM assembler
 LLVM_AS:FILEPATH=/usr/bin/llvm-as
@@ -563,7 +563,7 @@
 //Clang's available with 64bit math
 CLANG_HAS_64B_MATH:INTERNAL=ON
 //Clang option used to specify the target cpu
-CLANG_MARCH_FLAG:INTERNAL=-march=
+CLANG_MARCH_FLAG:INTERNAL=-mcpu=
 //Clang needs extra --rtlib flag for compiler-rt math
 CLANG_NEEDS_RTLIB:INTERNAL=OFF
 //Clang resource dir
@@ -849,7 +849,7 @@
 Hwloc_hwloc_PREFIX:INTERNAL=
 Hwloc_hwloc_VERSION:INTERNAL=
 //Autodetected CPU
-LLC_HOST_CPU_AUTO:INTERNAL=westmere
+LLC_HOST_CPU_AUTO:INTERNAL=pentium2
 //LLC_TRIPLE
 LLC_TRIPLE:INTERNAL=x86_64-unknown-linux-gnu
 //LLVM link test result

+++ /home/abuild/rpmbuild/BUILD/pocl-1.4/build/config.h	2035-04-29 21:36:30.340912151 +0000
@@ -166,7 +166,7 @@
 
 /* used in lib/CL/devices/basic */
 #define OCL_KERNEL_TARGET  "x86_64-unknown-linux-gnu"
-#define OCL_KERNEL_TARGET_CPU  "westmere"
+#define OCL_KERNEL_TARGET_CPU  "pentium2"
 
 #define PACKAGE_VERSION "1.4"

And in one of the two build logs I see 1592 instances of such

clang-9.0: warning: argument unused during compilation: '-mcpu=athlon64' [-Wunused-command-line-argument]

The other build.log has 0 of these.

It seems, CLANG_MARCH_FLAG is set incorrectly and that seems to come from
https://github.com/pocl/pocl/blob/release_1_4/cmake/LLVM.cmake#L698

[bmwiedemann]

And the root of the trouble is that clang -march=pentium2 throws an error. But that is the CPU you get autodetected with qemu-kvm -cpu kvm64

[bmwiedemann]

additionally, compiling the sleef/test.c with -march=penryn throws an error in test.c:40 : error: 256bit vectors unavailable

[franz]

Sounds like a bug in Clang cpu detection. Pentium 2 is a 32bit cpu, the first CPU with x86-64 was athlon64.

Compiling sleef/test.c with -DVEC256 -march=penryn should result in an error, penryn was core 2 which didn't have 256bit AVX.

[franz]

The simplest workaround could be to run cmake with -DCLANG_MARCH_FLAG:STRING=-march on x86 in your build scripts, and leave the detection for non-x86 architectures only. Please try this with release_1_5 branch, i have added a small necessary fix for this.

With distro builds, the LLC_HOST_CPU variable is not used in actual build commands, only some cmake checks, so unless you run into some more broken cmake checks, it shouldn't matter what is it set to.

[bmwiedemann]

Indeed, it helps to add -DCLANG_MARCH_FLAG:STRING=-march= with pocl-1.5