the workflow of "Tests (clang++-18, C++23, release, --enable-dpdk) fails due to dpdk build failure #2242
Comments
tchaikov
changed the title
|
Likely due to some translation units compiled with -march=native and some not. |
tchaikov
added a commit
to tchaikov/seastar
that referenced
this issue
May 16, 2024
when building dpdk on a building host which has AVX10.1 ISA support,
with Clang 18, we have following failure
```
/usr/lib/llvm-18/bin/clang -Ilib/net/libnet_crc_avx512_lib.a.p -Ilib/net -I../../../../../../dpdk/lib/net -I. -I../../../../../../dpdk -Iconfig -I../../../../../../dpdk/config -Ilib/eal/include -I../../../../../../dpdk/lib/eal/include -Ilib/eal/linux/include -I../../../../../../dpdk/lib/eal/linux/include -Ilib/eal/x86/include -I../../../../../../dpdk/lib/eal/x86/include -Ilib/eal/common -I../../../../../../dpdk/lib/eal/common -Ilib/eal -I../../../../../../dpdk/lib/eal -Ilib/kvargs -I../../../../../../dpdk/lib/kvargs -Ilib/metrics -I../../../../../../dpdk/lib/metrics -Ilib/telemetry -I../../../../../../dpdk/lib/telemetry -fcolor-diagnostics -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Wextra -O3 -include rte_config.h -Wcast-qual -Wdeprecated -Wformat -Wformat-nonliteral -Wformat-security -Wmissing-declarations -Wmissing-prototypes -Wnested-externs -Wold-style-definition -Wpointer-arith -Wsign-compare -Wstrict-prototypes -Wundef -Wwrite-strings -Wno-address-of-packed-member -Wno-missing-field-initializers -D_GNU_SOURCE -Wno-error -fPIC -march=native -DALLOW_EXPERIMENTAL_API -DALLOW_INTERNAL_API -Wno-format-truncation -DCC_X86_64_SSE42_PCLMULQDQ_SUPPORT -DCC_X86_64_AVX512_VPCLMULQDQ_SUPPORT -mavx512f -mavx512bw -mavx512dq -mavx512vl -mvpclmulqdq -mavx2 -mavx -MD -MQ lib/net/libnet_crc_avx512_lib.a.p/net_crc_avx512.c.o -MF lib/net/libnet_crc_avx512_lib.a.p/net_crc_avx512.c.o.d -o lib/net/libnet_crc_avx512_lib.a.p/net_crc_avx512.c.o -c ../../../../../../dpdk/lib/net/net_crc_avx512.c
Error: ../../../../../../dpdk/lib/net/net_crc_avx512.c:324:22: error: always_inline function '_mm512_broadcast_i32x4' requires target feature 'evex512', but would be inlined into function 'crc32_load_init_constants' that is compiled without support for 'evex512'
324 | crc32_eth.rk1_rk2 = _mm512_broadcast_i32x4(a);
| ^
```
according to https://gcc.gnu.org/onlinedocs/gcc/x86-Options.html
and
https://github.com/llvm/llvm-project/blob/release/18.x/clang/docs/UsersManual.rst#x86,
we should either use `-mevex512` or use `-mavx10.1-512` for accessing
these new vectorized instructions provided by AVX10.1. so in this
change, per the suggestion in Clang's document of
> Users should avoid using AVX512 features in function target attributes
> when developing code for AVX10.
instead of keeping the `-mavx512*` options, we replace them with
`-mavx10.1-512`. the patch is not merged into DPDK upstream yet,
so let's apply it when building dpdk.
Fixes scylladb#2242
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
tchaikov
added a commit
to tchaikov/seastar
that referenced
this issue
May 16, 2024
when building dpdk on a building host which has AVX10.1 ISA support,
with Clang 18, we have following failure
```
/usr/lib/llvm-18/bin/clang -Ilib/net/libnet_crc_avx512_lib.a.p -Ilib/net -I../../../../../../dpdk/lib/net -I. -I../../../../../../dpdk -Iconfig -I../../../../../../dpdk/config -Ilib/eal/include -I../../../../../../dpdk/lib/eal/include -Ilib/eal/linux/include -I../../../../../../dpdk/lib/eal/linux/include -Ilib/eal/x86/include -I../../../../../../dpdk/lib/eal/x86/include -Ilib/eal/common -I../../../../../../dpdk/lib/eal/common -Ilib/eal -I../../../../../../dpdk/lib/eal -Ilib/kvargs -I../../../../../../dpdk/lib/kvargs -Ilib/metrics -I../../../../../../dpdk/lib/metrics -Ilib/telemetry -I../../../../../../dpdk/lib/telemetry -fcolor-diagnostics -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Wextra -O3 -include rte_config.h -Wcast-qual -Wdeprecated -Wformat -Wformat-nonliteral -Wformat-security -Wmissing-declarations -Wmissing-prototypes -Wnested-externs -Wold-style-definition -Wpointer-arith -Wsign-compare -Wstrict-prototypes -Wundef -Wwrite-strings -Wno-address-of-packed-member -Wno-missing-field-initializers -D_GNU_SOURCE -Wno-error -fPIC -march=native -DALLOW_EXPERIMENTAL_API -DALLOW_INTERNAL_API -Wno-format-truncation -DCC_X86_64_SSE42_PCLMULQDQ_SUPPORT -DCC_X86_64_AVX512_VPCLMULQDQ_SUPPORT -mavx512f -mavx512bw -mavx512dq -mavx512vl -mvpclmulqdq -mavx2 -mavx -MD -MQ lib/net/libnet_crc_avx512_lib.a.p/net_crc_avx512.c.o -MF lib/net/libnet_crc_avx512_lib.a.p/net_crc_avx512.c.o.d -o lib/net/libnet_crc_avx512_lib.a.p/net_crc_avx512.c.o -c ../../../../../../dpdk/lib/net/net_crc_avx512.c
Error: ../../../../../../dpdk/lib/net/net_crc_avx512.c:324:22: error: always_inline function '_mm512_broadcast_i32x4' requires target feature 'evex512', but would be inlined into function 'crc32_load_init_constants' that is compiled without support for 'evex512'
324 | crc32_eth.rk1_rk2 = _mm512_broadcast_i32x4(a);
| ^
```
according to https://gcc.gnu.org/onlinedocs/gcc/x86-Options.html
and
https://github.com/llvm/llvm-project/blob/release/18.x/clang/docs/UsersManual.rst#x86,
we should either use `-mevex512` or use `-mavx10.1-512` for accessing
these new vectorized instructions provided by AVX10.1. so in this
change, per the suggestion in Clang's document of
> Users should avoid using AVX512 features in function target attributes
> when developing code for AVX10.
instead of keeping the `-mavx512*` options, we replace them with
`-mavx10.1-512`. the patch is not merged into DPDK upstream yet,
so let's apply it when building dpdk.
Fixes scylladb#2242
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
tchaikov
added a commit
to tchaikov/seastar
that referenced
this issue
May 16, 2024
when building dpdk on a building host which has AVX10.1 ISA support,
with Clang 18, we have following failure
```
/usr/lib/llvm-18/bin/clang -Ilib/net/libnet_crc_avx512_lib.a.p -Ilib/net -I../../../../../../dpdk/lib/net -I. -I../../../../../../dpdk -Iconfig -I../../../../../../dpdk/config -Ilib/eal/include -I../../../../../../dpdk/lib/eal/include -Ilib/eal/linux/include -I../../../../../../dpdk/lib/eal/linux/include -Ilib/eal/x86/include -I../../../../../../dpdk/lib/eal/x86/include -Ilib/eal/common -I../../../../../../dpdk/lib/eal/common -Ilib/eal -I../../../../../../dpdk/lib/eal -Ilib/kvargs -I../../../../../../dpdk/lib/kvargs -Ilib/metrics -I../../../../../../dpdk/lib/metrics -Ilib/telemetry -I../../../../../../dpdk/lib/telemetry -fcolor-diagnostics -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Wextra -O3 -include rte_config.h -Wcast-qual -Wdeprecated -Wformat -Wformat-nonliteral -Wformat-security -Wmissing-declarations -Wmissing-prototypes -Wnested-externs -Wold-style-definition -Wpointer-arith -Wsign-compare -Wstrict-prototypes -Wundef -Wwrite-strings -Wno-address-of-packed-member -Wno-missing-field-initializers -D_GNU_SOURCE -Wno-error -fPIC -march=native -DALLOW_EXPERIMENTAL_API -DALLOW_INTERNAL_API -Wno-format-truncation -DCC_X86_64_SSE42_PCLMULQDQ_SUPPORT -DCC_X86_64_AVX512_VPCLMULQDQ_SUPPORT -mavx512f -mavx512bw -mavx512dq -mavx512vl -mvpclmulqdq -mavx2 -mavx -MD -MQ lib/net/libnet_crc_avx512_lib.a.p/net_crc_avx512.c.o -MF lib/net/libnet_crc_avx512_lib.a.p/net_crc_avx512.c.o.d -o lib/net/libnet_crc_avx512_lib.a.p/net_crc_avx512.c.o -c ../../../../../../dpdk/lib/net/net_crc_avx512.c
Error: ../../../../../../dpdk/lib/net/net_crc_avx512.c:324:22: error: always_inline function '_mm512_broadcast_i32x4' requires target feature 'evex512', but would be inlined into function 'crc32_load_init_constants' that is compiled without support for 'evex512'
324 | crc32_eth.rk1_rk2 = _mm512_broadcast_i32x4(a);
| ^
```
according to https://gcc.gnu.org/onlinedocs/gcc/x86-Options.html
and
https://github.com/llvm/llvm-project/blob/release/18.x/clang/docs/UsersManual.rst#x86,
we should either use `-mevex512` or use `-mavx10.1-512` for accessing
these new vectorized instructions provided by AVX10.1. so in this
change, per the suggestion in Clang's document of
> Users should avoid using AVX512 features in function target attributes
> when developing code for AVX10.
instead of keeping the `-mavx512*` options, we replace them with
`-mavx10.1-512`. the patch is not merged into DPDK upstream yet,
so let's apply it when building dpdk.
Fixes scylladb#2242
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
tchaikov
added a commit
to tchaikov/seastar
that referenced
this issue
May 16, 2024
when building dpdk on a building host which has AVX10.1 ISA support,
with Clang 18, we have following failure
```
/usr/lib/llvm-18/bin/clang -Ilib/net/libnet_crc_avx512_lib.a.p -Ilib/net -I../../../../../../dpdk/lib/net -I. -I../../../../../../dpdk -Iconfig -I../../../../../../dpdk/config -Ilib/eal/include -I../../../../../../dpdk/lib/eal/include -Ilib/eal/linux/include -I../../../../../../dpdk/lib/eal/linux/include -Ilib/eal/x86/include -I../../../../../../dpdk/lib/eal/x86/include -Ilib/eal/common -I../../../../../../dpdk/lib/eal/common -Ilib/eal -I../../../../../../dpdk/lib/eal -Ilib/kvargs -I../../../../../../dpdk/lib/kvargs -Ilib/metrics -I../../../../../../dpdk/lib/metrics -Ilib/telemetry -I../../../../../../dpdk/lib/telemetry -fcolor-diagnostics -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Wextra -O3 -include rte_config.h -Wcast-qual -Wdeprecated -Wformat -Wformat-nonliteral -Wformat-security -Wmissing-declarations -Wmissing-prototypes -Wnested-externs -Wold-style-definition -Wpointer-arith -Wsign-compare -Wstrict-prototypes -Wundef -Wwrite-strings -Wno-address-of-packed-member -Wno-missing-field-initializers -D_GNU_SOURCE -Wno-error -fPIC -march=native -DALLOW_EXPERIMENTAL_API -DALLOW_INTERNAL_API -Wno-format-truncation -DCC_X86_64_SSE42_PCLMULQDQ_SUPPORT -DCC_X86_64_AVX512_VPCLMULQDQ_SUPPORT -mavx512f -mavx512bw -mavx512dq -mavx512vl -mvpclmulqdq -mavx2 -mavx -MD -MQ lib/net/libnet_crc_avx512_lib.a.p/net_crc_avx512.c.o -MF lib/net/libnet_crc_avx512_lib.a.p/net_crc_avx512.c.o.d -o lib/net/libnet_crc_avx512_lib.a.p/net_crc_avx512.c.o -c ../../../../../../dpdk/lib/net/net_crc_avx512.c
Error: ../../../../../../dpdk/lib/net/net_crc_avx512.c:324:22: error: always_inline function '_mm512_broadcast_i32x4' requires target feature 'evex512', but would be inlined into function 'crc32_load_init_constants' that is compiled without support for 'evex512'
324 | crc32_eth.rk1_rk2 = _mm512_broadcast_i32x4(a);
| ^
```
according to https://gcc.gnu.org/onlinedocs/gcc/x86-Options.html
and
https://github.com/llvm/llvm-project/blob/release/18.x/clang/docs/UsersManual.rst#x86,
we should either use `-mevex512` or use `-mavx10.1-512` for accessing
these new vectorized instructions provided by AVX10.1. so in this
change, per the suggestion in Clang's document of
> Users should avoid using AVX512 features in function target attributes
> when developing code for AVX10.
instead of keeping the `-mavx512*` options, we replace them with
`-mavx10.1-512`. the patch is not merged into DPDK upstream yet,
so let's apply it when building dpdk.
Fixes scylladb#2242
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
tchaikov
added a commit
to tchaikov/seastar
that referenced
this issue
May 16, 2024
when building dpdk on a building host which has AVX10.1 ISA support,
with Clang 18, we have following failure
```
/usr/lib/llvm-18/bin/clang -Ilib/net/libnet_crc_avx512_lib.a.p -Ilib/net -I../../../../../../dpdk/lib/net -I. -I../../../../../../dpdk -Iconfig -I../../../../../../dpdk/config -Ilib/eal/include -I../../../../../../dpdk/lib/eal/include -Ilib/eal/linux/include -I../../../../../../dpdk/lib/eal/linux/include -Ilib/eal/x86/include -I../../../../../../dpdk/lib/eal/x86/include -Ilib/eal/common -I../../../../../../dpdk/lib/eal/common -Ilib/eal -I../../../../../../dpdk/lib/eal -Ilib/kvargs -I../../../../../../dpdk/lib/kvargs -Ilib/metrics -I../../../../../../dpdk/lib/metrics -Ilib/telemetry -I../../../../../../dpdk/lib/telemetry -fcolor-diagnostics -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Wextra -O3 -include rte_config.h -Wcast-qual -Wdeprecated -Wformat -Wformat-nonliteral -Wformat-security -Wmissing-declarations -Wmissing-prototypes -Wnested-externs -Wold-style-definition -Wpointer-arith -Wsign-compare -Wstrict-prototypes -Wundef -Wwrite-strings -Wno-address-of-packed-member -Wno-missing-field-initializers -D_GNU_SOURCE -Wno-error -fPIC -march=native -DALLOW_EXPERIMENTAL_API -DALLOW_INTERNAL_API -Wno-format-truncation -DCC_X86_64_SSE42_PCLMULQDQ_SUPPORT -DCC_X86_64_AVX512_VPCLMULQDQ_SUPPORT -mavx512f -mavx512bw -mavx512dq -mavx512vl -mvpclmulqdq -mavx2 -mavx -MD -MQ lib/net/libnet_crc_avx512_lib.a.p/net_crc_avx512.c.o -MF lib/net/libnet_crc_avx512_lib.a.p/net_crc_avx512.c.o.d -o lib/net/libnet_crc_avx512_lib.a.p/net_crc_avx512.c.o -c ../../../../../../dpdk/lib/net/net_crc_avx512.c
Error: ../../../../../../dpdk/lib/net/net_crc_avx512.c:324:22: error: always_inline function '_mm512_broadcast_i32x4' requires target feature 'evex512', but would be inlined into function 'crc32_load_init_constants' that is compiled without support for 'evex512'
324 | crc32_eth.rk1_rk2 = _mm512_broadcast_i32x4(a);
| ^
```
according to https://gcc.gnu.org/onlinedocs/gcc/x86-Options.html
and
https://github.com/llvm/llvm-project/blob/release/18.x/clang/docs/UsersManual.rst#x86,
we should either use `-mevex512` or use `-mavx10.1-512` for accessing
these new vectorized instructions provided by AVX10.1. but we also
pass "-march=native" to the compiler. it turns out that at least some
github-hosted runners do not support AVX10.1 ISA, but the Clang 18
compiler does. so this combination breaks the build, as dpdk tries
to check if compiler supports AVX512, and if it does, dpdk builds
the AVX512 C files with this unsupported combination. that's why we
have build failures recently.
to address this issue, instead of using "native" as the value of
`Seastar_DPDK_MACHINE`, let's use a more conservative but still
a capable architecture supported by github-hosted runner: "haswell",
which supports the instruction set that is used by DPDK. so we can
still have a decent coverage for building with DPDK, and for testing
with the build.
Fixes scylladb#2242
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
tchaikov
added a commit
to tchaikov/seastar
that referenced
this issue
May 16, 2024
…runner
when building dpdk on a building host which has AVX10.1 ISA support,
with Clang 18, we have following failure
```
/usr/lib/llvm-18/bin/clang -Ilib/net/libnet_crc_avx512_lib.a.p -Ilib/net -I../../../../../../dpdk/lib/net -I. -I../../../../../../dpdk -Iconfig -I../../../../../../dpdk/config -Ilib/eal/include -I../../../../../../dpdk/lib/eal/include -Ilib/eal/linux/include -I../../../../../../dpdk/lib/eal/linux/include -Ilib/eal/x86/include -I../../../../../../dpdk/lib/eal/x86/include -Ilib/eal/common -I../../../../../../dpdk/lib/eal/common -Ilib/eal -I../../../../../../dpdk/lib/eal -Ilib/kvargs -I../../../../../../dpdk/lib/kvargs -Ilib/metrics -I../../../../../../dpdk/lib/metrics -Ilib/telemetry -I../../../../../../dpdk/lib/telemetry -fcolor-diagnostics -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Wextra -O3 -include rte_config.h -Wcast-qual -Wdeprecated -Wformat -Wformat-nonliteral -Wformat-security -Wmissing-declarations -Wmissing-prototypes -Wnested-externs -Wold-style-definition -Wpointer-arith -Wsign-compare -Wstrict-prototypes -Wundef -Wwrite-strings -Wno-address-of-packed-member -Wno-missing-field-initializers -D_GNU_SOURCE -Wno-error -fPIC -march=native -DALLOW_EXPERIMENTAL_API -DALLOW_INTERNAL_API -Wno-format-truncation -DCC_X86_64_SSE42_PCLMULQDQ_SUPPORT -DCC_X86_64_AVX512_VPCLMULQDQ_SUPPORT -mavx512f -mavx512bw -mavx512dq -mavx512vl -mvpclmulqdq -mavx2 -mavx -MD -MQ lib/net/libnet_crc_avx512_lib.a.p/net_crc_avx512.c.o -MF lib/net/libnet_crc_avx512_lib.a.p/net_crc_avx512.c.o.d -o lib/net/libnet_crc_avx512_lib.a.p/net_crc_avx512.c.o -c ../../../../../../dpdk/lib/net/net_crc_avx512.c
Error: ../../../../../../dpdk/lib/net/net_crc_avx512.c:324:22: error: always_inline function '_mm512_broadcast_i32x4' requires target feature 'evex512', but would be inlined into function 'crc32_load_init_constants' that is compiled without support for 'evex512'
324 | crc32_eth.rk1_rk2 = _mm512_broadcast_i32x4(a);
| ^
```
according to https://gcc.gnu.org/onlinedocs/gcc/x86-Options.html
and
https://github.com/llvm/llvm-project/blob/release/18.x/clang/docs/UsersManual.rst#x86,
we should either use `-mevex512` or use `-mavx10.1-512` for accessing
these new vectorized instructions provided by AVX10.1. but we also
pass "-march=native" to the compiler. it turns out that at least some
github-hosted runners do not support AVX10.1 ISA, but the Clang 18
compiler does. so this combination breaks the build, as dpdk tries
to check if compiler supports AVX512, and if it does, dpdk builds
the AVX512 C files with this unsupported combination. that's why we
have build failures recently.
to address this issue, instead of using "native" as the value of
`Seastar_DPDK_MACHINE`, let's use a more conservative but still
a capable architecture supported by github-hosted runner: "haswell",
which supports the instruction set that is used by DPDK. so we can
still have a decent coverage for building with DPDK, and for testing
with the build.
Fixes scylladb#2242
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
tchaikov
added a commit
to tchaikov/seastar
that referenced
this issue
May 16, 2024
…ted runner
when building dpdk on a building host which has AVX10.1 ISA support,
with Clang 18, we have following failure
```
/usr/lib/llvm-18/bin/clang -Ilib/net/libnet_crc_avx512_lib.a.p -Ilib/net -I../../../../../../dpdk/lib/net -I. -I../../../../../../dpdk -Iconfig -I../../../../../../dpdk/config -Ilib/eal/include -I../../../../../../dpdk/lib/eal/include -Ilib/eal/linux/include -I../../../../../../dpdk/lib/eal/linux/include -Ilib/eal/x86/include -I../../../../../../dpdk/lib/eal/x86/include -Ilib/eal/common -I../../../../../../dpdk/lib/eal/common -Ilib/eal -I../../../../../../dpdk/lib/eal -Ilib/kvargs -I../../../../../../dpdk/lib/kvargs -Ilib/metrics -I../../../../../../dpdk/lib/metrics -Ilib/telemetry -I../../../../../../dpdk/lib/telemetry -fcolor-diagnostics -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Wextra -O3 -include rte_config.h -Wcast-qual -Wdeprecated -Wformat -Wformat-nonliteral -Wformat-security -Wmissing-declarations -Wmissing-prototypes -Wnested-externs -Wold-style-definition -Wpointer-arith -Wsign-compare -Wstrict-prototypes -Wundef -Wwrite-strings -Wno-address-of-packed-member -Wno-missing-field-initializers -D_GNU_SOURCE -Wno-error -fPIC -march=native -DALLOW_EXPERIMENTAL_API -DALLOW_INTERNAL_API -Wno-format-truncation -DCC_X86_64_SSE42_PCLMULQDQ_SUPPORT -DCC_X86_64_AVX512_VPCLMULQDQ_SUPPORT -mavx512f -mavx512bw -mavx512dq -mavx512vl -mvpclmulqdq -mavx2 -mavx -MD -MQ lib/net/libnet_crc_avx512_lib.a.p/net_crc_avx512.c.o -MF lib/net/libnet_crc_avx512_lib.a.p/net_crc_avx512.c.o.d -o lib/net/libnet_crc_avx512_lib.a.p/net_crc_avx512.c.o -c ../../../../../../dpdk/lib/net/net_crc_avx512.c
Error: ../../../../../../dpdk/lib/net/net_crc_avx512.c:324:22: error: always_inline function '_mm512_broadcast_i32x4' requires target feature 'evex512', but would be inlined into function 'crc32_load_init_constants' that is compiled without support for 'evex512'
324 | crc32_eth.rk1_rk2 = _mm512_broadcast_i32x4(a);
| ^
```
according to https://gcc.gnu.org/onlinedocs/gcc/x86-Options.html
and
https://github.com/llvm/llvm-project/blob/release/18.x/clang/docs/UsersManual.rst#x86,
we should either use `-mevex512` or use `-mavx10.1-512` for accessing
these new vectorized instructions provided by AVX10.1. but we also
pass "-march=native" to the compiler. it turns out that at least some
github-hosted runners do not support AVX10.1 ISA, but the Clang 18
compiler does. so this combination breaks the build, as dpdk tries
to check if compiler supports AVX512, and if it does, dpdk builds
the AVX512 C files with this unsupported combination. that's why we
have build failures recently.
to address this issue, instead of using "native" as the value of
`Seastar_DPDK_MACHINE`, let's use a more conservative but still
capable architecture supported by github-hosted runner: "haswell",
which supports the instruction set that is used by quite a few
DPDK optimizations. so we can still have a decent coverage for
building with DPDK, and for testing the build. please note,
we don't test Seastar with the DPDK backend yet.
Fixes scylladb#2242
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
tchaikov
added a commit
to tchaikov/seastar
that referenced
this issue
May 16, 2024
…ted runner
when building dpdk on a building host which has AVX10.1 ISA support,
with Clang 18, we have following failure
```
/usr/lib/llvm-18/bin/clang -Ilib/net/libnet_crc_avx512_lib.a.p -Ilib/net -I../../../../../../dpdk/lib/net -I. -I../../../../../../dpdk -Iconfig -I../../../../../../dpdk/config -Ilib/eal/include -I../../../../../../dpdk/lib/eal/include -Ilib/eal/linux/include -I../../../../../../dpdk/lib/eal/linux/include -Ilib/eal/x86/include -I../../../../../../dpdk/lib/eal/x86/include -Ilib/eal/common -I../../../../../../dpdk/lib/eal/common -Ilib/eal -I../../../../../../dpdk/lib/eal -Ilib/kvargs -I../../../../../../dpdk/lib/kvargs -Ilib/metrics -I../../../../../../dpdk/lib/metrics -Ilib/telemetry -I../../../../../../dpdk/lib/telemetry -fcolor-diagnostics -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Wextra -O3 -include rte_config.h -Wcast-qual -Wdeprecated -Wformat -Wformat-nonliteral -Wformat-security -Wmissing-declarations -Wmissing-prototypes -Wnested-externs -Wold-style-definition -Wpointer-arith -Wsign-compare -Wstrict-prototypes -Wundef -Wwrite-strings -Wno-address-of-packed-member -Wno-missing-field-initializers -D_GNU_SOURCE -Wno-error -fPIC -march=native -DALLOW_EXPERIMENTAL_API -DALLOW_INTERNAL_API -Wno-format-truncation -DCC_X86_64_SSE42_PCLMULQDQ_SUPPORT -DCC_X86_64_AVX512_VPCLMULQDQ_SUPPORT -mavx512f -mavx512bw -mavx512dq -mavx512vl -mvpclmulqdq -mavx2 -mavx -MD -MQ lib/net/libnet_crc_avx512_lib.a.p/net_crc_avx512.c.o -MF lib/net/libnet_crc_avx512_lib.a.p/net_crc_avx512.c.o.d -o lib/net/libnet_crc_avx512_lib.a.p/net_crc_avx512.c.o -c ../../../../../../dpdk/lib/net/net_crc_avx512.c
Error: ../../../../../../dpdk/lib/net/net_crc_avx512.c:324:22: error: always_inline function '_mm512_broadcast_i32x4' requires target feature 'evex512', but would be inlined into function 'crc32_load_init_constants' that is compiled without support for 'evex512'
324 | crc32_eth.rk1_rk2 = _mm512_broadcast_i32x4(a);
| ^
```
according to https://gcc.gnu.org/onlinedocs/gcc/x86-Options.html
and
https://github.com/llvm/llvm-project/blob/release/18.x/clang/docs/UsersManual.rst#x86,
we should either use `-mevex512` or use `-mavx10.1-512` for accessing
these new vectorized instructions provided by AVX10.1. but we also
pass "-march=native" to the compiler. it turns out that at least some
github-hosted runners do not support AVX10.1 ISA, but the Clang 18
compiler does. so this combination breaks the build, as dpdk tries
to check if compiler supports AVX512, and if it does, dpdk builds
the AVX512 C files with this unsupported combination. that's why we
have build failures recently.
to address this issue, instead of using "native" as the value of
`Seastar_DPDK_MACHINE`, let's use a more conservative but still
capable architecture supported by github-hosted runner: "haswell",
which supports the instruction set that is used by quite a few
DPDK optimizations. so we can still have a decent coverage for
building with DPDK, and for testing the build. please note,
we don't test Seastar with the DPDK backend yet.
Fixes scylladb#2242
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
the building host is not able to execute AVX-10 instructions, so |
avikivity
added a commit
that referenced
this issue
May 16, 2024
…hub-hosted runner' from Kefu Chai
when building dpdk on a building host which has AVX10.1 ISA support,
with Clang 18, we have following failure
```
/usr/lib/llvm-18/bin/clang -Ilib/net/libnet_crc_avx512_lib.a.p -Ilib/net -I../../../../../../dpdk/lib/net -I. -I../../../../../../dpdk -Iconfig -I../../../../../../dpdk/config -Ilib/eal/include -I../../../../../../dpdk/lib/eal/include -Ilib/eal/linux/include -I../../../../../../dpdk/lib/eal/linux/include -Ilib/eal/x86/include -I../../../../../../dpdk/lib/eal/x86/include -Ilib/eal/common -I../../../../../../dpdk/lib/eal/common -Ilib/eal -I../../../../../../dpdk/lib/eal -Ilib/kvargs -I../../../../../../dpdk/lib/kvargs -Ilib/metrics -I../../../../../../dpdk/lib/metrics -Ilib/telemetry -I../../../../../../dpdk/lib/telemetry -fcolor-diagnostics -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Wextra -O3 -include rte_config.h -Wcast-qual -Wdeprecated -Wformat -Wformat-nonliteral -Wformat-security -Wmissing-declarations -Wmissing-prototypes -Wnested-externs -Wold-style-definition -Wpointer-arith -Wsign-compare -Wstrict-prototypes -Wundef -Wwrite-strings -Wno-address-of-packed-member -Wno-missing-field-initializers -D_GNU_SOURCE -Wno-error -fPIC -march=native -DALLOW_EXPERIMENTAL_API -DALLOW_INTERNAL_API -Wno-format-truncation -DCC_X86_64_SSE42_PCLMULQDQ_SUPPORT -DCC_X86_64_AVX512_VPCLMULQDQ_SUPPORT -mavx512f -mavx512bw -mavx512dq -mavx512vl -mvpclmulqdq -mavx2 -mavx -MD -MQ lib/net/libnet_crc_avx512_lib.a.p/net_crc_avx512.c.o -MF lib/net/libnet_crc_avx512_lib.a.p/net_crc_avx512.c.o.d -o lib/net/libnet_crc_avx512_lib.a.p/net_crc_avx512.c.o -c ../../../../../../dpdk/lib/net/net_crc_avx512.c
Error: ../../../../../../dpdk/lib/net/net_crc_avx512.c:324:22: error: always_inline function '_mm512_broadcast_i32x4' requires target feature 'evex512', but would be inlined into function 'crc32_load_init_constants' that is compiled without support for 'evex512'
324 | crc32_eth.rk1_rk2 = _mm512_broadcast_i32x4(a);
| ^
```
according to https://gcc.gnu.org/onlinedocs/gcc/x86-Options.html
and
https://github.com/llvm/llvm-project/blob/release/18.x/clang/docs/UsersManual.rst#x86,
we should either use `-mevex512` or use `-mavx10.1-512` for accessing these new vectorized instructions provided by AVX10.1.
we also pass `-march=native` to the compiler. and it turns out that at least some github-hosted runners do not support AVX10.1 ISA, but the Clang 18 compiler does. so this combination breaks the build, as dpdk tries to check if compiler supports AVX512, and if it does, dpdk builds the AVX512 C files with this unsupported combination. that's why we
have build failures recently.
to address this issue, instead of using "native" as the value of `Seastar_DPDK_MACHINE`, let's use a more conservative but still capable architecture supported by github-hosted runner: "haswell", which supports the instruction set that is used by quite a few DPDK optimizations. so we can still have a decent coverage for building with DPDK, and for testing the build. please note, we don't test Seastar with the DPDK backend yet.
to enable the workflow to pass extra options to `configure.py`, we change the "cooks" parameter in the matrix to "options", and append "--dpdk-machine haswell" to it, so that we use this option only when testing the dpdk build.
Fixes #2242
Closes #2243
* github.com:scylladb/seastar:
build: use DPDK_MACHINE=haswell when testing dpdk build on github-hosted runner
build: add --dpdk-machine option to configure.py
build: stop translating -march option to names recognized by DPDK
|
we can revert the fix of ba395a2 if we can use clang 18.1.6. see https://github.com/llvm/llvm-project/releases/tag/llvmorg-18.1.6
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
there are more of similar failures where vector extensions are used, and we are not compiling with
evex512arch enabled.The text was updated successfully, but these errors were encountered: