Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ARM Intrinsics don't build on clang #21

Closed
gliwka opened this issue Apr 28, 2021 · 13 comments
Closed

ARM Intrinsics don't build on clang #21

gliwka opened this issue Apr 28, 2021 · 13 comments

Comments

@gliwka
Copy link

gliwka commented Apr 28, 2021 •
edited
Loading

I'm trying to build vectorscan on an ARM machine using clang (arm64, macOS, M1 chip). Clang compilation works on x86_64, but fails on ARM. The build fails while compiling the NEON intrinsics:

Version (clang -v):

Apple clang version 12.0.5 (clang-1205.0.22.9)
Target: arm64-apple-darwin20.4.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

Compiler output:

Consolidate compiler generated dependencies of target hs_exec
[  1%] Building C object CMakeFiles/hs_exec.dir/src/runtime.c.o
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/runtime.c:45:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/nfa/nfa_rev_api.h:38:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/nfa/vermicelli.h:36:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/util/bitutils.h:51:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/util/arch/arm/bitutils.h:41:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/util/arch/common/bitutils.h:38:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/util/simd_utils.h:67:
/Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/util/arch/arm/simd_utils.h:98:19: error: argument to '__builtin_neon_vshlq_n_v' must be a constant integer
    return (m128) vshlq_n_s32((int64x2_t)a, b);
                  ^                         ~
/Library/Developer/CommandLineTools/usr/lib/clang/12.0.5/include/arm_neon.h:24760:23: note: expanded from macro 'vshlq_n_s32'
  __ret = (int32x4_t) __builtin_neon_vshlq_n_v((int8x16_t)__s0, __p1, 34); \
                      ^                                         ~~~~
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/runtime.c:45:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/nfa/nfa_rev_api.h:38:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/nfa/vermicelli.h:36:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/util/bitutils.h:51:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/util/arch/arm/bitutils.h:41:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/util/arch/common/bitutils.h:38:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/util/simd_utils.h:67:
/Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/util/arch/arm/simd_utils.h:103:19: error: argument to '__builtin_neon_vshrq_n_v' must be a constant integer
    return (m128) vshrq_n_s32((int64x2_t)a, b);
                  ^                         ~
/Library/Developer/CommandLineTools/usr/lib/clang/12.0.5/include/arm_neon.h:25168:23: note: expanded from macro 'vshrq_n_s32'
  __ret = (int32x4_t) __builtin_neon_vshrq_n_v((int8x16_t)__s0, __p1, 34); \
                      ^                                         ~~~~
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/runtime.c:45:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/nfa/nfa_rev_api.h:38:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/nfa/vermicelli.h:36:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/util/bitutils.h:51:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/util/arch/arm/bitutils.h:41:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/util/arch/common/bitutils.h:38:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/util/simd_utils.h:67:
/Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/util/arch/arm/simd_utils.h:108:19: error: argument to '__builtin_neon_vshlq_n_v' must be a constant integer
    return (m128) vshlq_n_s64((int64x2_t)a, b);
                  ^                         ~
/Library/Developer/CommandLineTools/usr/lib/clang/12.0.5/include/arm_neon.h:24778:23: note: expanded from macro 'vshlq_n_s64'
  __ret = (int64x2_t) __builtin_neon_vshlq_n_v((int8x16_t)__s0, __p1, 35); \
                      ^                                         ~~~~
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/runtime.c:45:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/nfa/nfa_rev_api.h:38:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/nfa/vermicelli.h:36:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/util/bitutils.h:51:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/util/arch/arm/bitutils.h:41:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/util/arch/common/bitutils.h:38:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/util/simd_utils.h:67:
/Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/util/arch/arm/simd_utils.h:113:19: error: argument to '__builtin_neon_vshrq_n_v' must be a constant integer
    return (m128) vshrq_n_s64((int64x2_t)a, b);
                  ^                         ~
/Library/Developer/CommandLineTools/usr/lib/clang/12.0.5/include/arm_neon.h:25186:23: note: expanded from macro 'vshrq_n_s64'
  __ret = (int64x2_t) __builtin_neon_vshrq_n_v((int8x16_t)__s0, __p1, 35); \
                      ^                                         ~~~~
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/runtime.c:45:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/nfa/nfa_rev_api.h:38:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/nfa/vermicelli.h:36:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/util/bitutils.h:51:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/util/arch/arm/bitutils.h:41:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/util/arch/common/bitutils.h:38:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/util/simd_utils.h:67:
/Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/util/arch/arm/simd_utils.h:166:12: error: argument to '__builtin_neon_vgetq_lane_i32' must be a constant integer
    return vgetq_lane_u32((uint32x4_t) in, imm);
           ^                               ~~~
/Library/Developer/CommandLineTools/usr/lib/clang/12.0.5/include/arm_neon.h:7511:22: note: expanded from macro 'vgetq_lane_u32'
  __ret = (uint32_t) __builtin_neon_vgetq_lane_i32((int32x4_t)__s0, __p1); \
                     ^                                              ~~~~
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/runtime.c:45:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/nfa/nfa_rev_api.h:38:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/nfa/vermicelli.h:36:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/util/bitutils.h:51:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/util/arch/arm/bitutils.h:41:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/util/arch/common/bitutils.h:38:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/util/simd_utils.h:67:
/Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/util/arch/arm/simd_utils.h:190:12: error: argument to '__builtin_neon_vgetq_lane_i64' must be a constant integer
    return vgetq_lane_u64((uint64x2_t) in, imm);
           ^                               ~~~
/Library/Developer/CommandLineTools/usr/lib/clang/12.0.5/include/arm_neon.h:7534:22: note: expanded from macro 'vgetq_lane_u64'
  __ret = (uint64_t) __builtin_neon_vgetq_lane_i64((int64x2_t)__s0, __p1); \
                     ^                                              ~~~~
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/runtime.c:45:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/nfa/nfa_rev_api.h:38:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/nfa/vermicelli.h:36:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/util/bitutils.h:51:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/util/arch/arm/bitutils.h:41:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/util/arch/common/bitutils.h:38:
In file included from /Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/util/simd_utils.h:67:
/Users/mg/projects/hyperscan-java-native/cppbuild/hyperscan-5.4.0/src/util/arch/arm/simd_utils.h:304:18: error: argument to '__builtin_neon_vextq_v' must be a constant integer
    return (m128)vextq_s8((int8x16_t)l, (int8x16_t)r, offset);
                 ^                                    ~~~~~~
/Library/Developer/CommandLineTools/usr/lib/clang/12.0.5/include/arm_neon.h:6896:23: note: expanded from macro 'vextq_s8'
  __ret = (int8x16_t) __builtin_neon_vextq_v((int8x16_t)__s0, (int8x16_t)__s1, __p2, 32); \
                      ^                                                        ~~~~
7 errors generated.
make[2]: *** [CMakeFiles/hs_exec.dir/src/runtime.c.o] Error 1
make[1]: *** [CMakeFiles/hs_exec.dir/all] Error 2
make: *** [all] Error 2

It seems like clang doesn't understand how to inline parameters it expects to be constant.

Maybe some of the abstraction functions for the intrinsics need to be converted to macros?
@gliwka
Copy link
Author

gliwka commented Apr 28, 2021 •
edited
Loading

I also had to comment those lines in CMakeLists.txt, because clang doesn't support the -march=native flag on M1 chips:

    #if (NOT CMAKE_C_FLAGS MATCHES .*march.* AND NOT CMAKE_C_FLAGS MATCHES .*mtune.*)
    #    set(ARCH_C_FLAGS "-march=native -mtune=${TUNE_FLAG}")
    #endif()

    #if (NOT CMAKE_CXX_FLAGS MATCHES .*march.* AND NOT CMAKE_CXX_FLAGS MATCHES .*mtune.*)
    #    set(ARCH_CXX_FLAGS "-march=native -mtune=${TUNE_FLAG}")
    #endif()

See https://stackoverflow.com/questions/65966969/why-does-march-native-not-work-on-apple-m1

@gliwka gliwka mentioned this issue Apr 28, 2021
Closed
@gliwka
Copy link
Author

gliwka commented Apr 30, 2021

I've started work on this and will submit a pull request

@gliwka
Copy link
Author

gliwka commented May 1, 2021

#22

@markos
Copy link

markos commented May 14, 2021

@gliwka I'm currently doing a rework of the SIMD support, I will tackle the clang issues there, thanks for the heads up! Work in progress, can be seen in branch templates-refactor.

@markos markos mentioned this issue Oct 26, 2021
Closed
@markos
Copy link

markos commented Dec 7, 2021

Clang support has been added in #81 for all platforms! :)

@markos markos closed this as completed Dec 7, 2021
NotAHero04 pushed a commit to NotAHero04/SVT-HEVC-NEON that referenced this issue Aug 13, 2023
Add support for other SIMDs via SIMDe
ee26625 
Known issue: Clang (Android NDK) won't compile with NEON (but GCC should), see VectorCamp/vectorscan#21
@yyctw yyctw mentioned this issue Nov 3, 2023
Closed
@RamaMalladiAWS
Copy link

Is this issue fixed in the recent releases of Clang? I see it to be an issue on Clang18.

$ clang+llvm-18.1.0-aarch64-linux-gnu/bin/clang++ -v
clang version 18.1.0rc

Build fail details:

intrins.hpp:789:14: error: argument to '__builtin_neon_vshrq_n_v' must be a constant integer
  789 |   x.val[0] = vshrq_n_u32(x.val[0], n);
      |              ^                     ~

Contents of intrins.hpp file (this call):

 788 INTRIN_INLINE uint32x4x2_t vshrq_n_u32_x2(uint32x4x2_t x, const int n) {
 789   x.val[0] = vshrq_n_u32(x.val[0], n);

Thank you.

@markos
Copy link

markos commented Mar 28, 2024

we haven't added clang 18 to our CI yet, but we will do so. Could you please add a new issue for clang 18?

@RamaMalladiAWS
Copy link

RamaMalladiAWS commented Mar 28, 2024 •
edited
Loading

Sure. I ran into the same issue w Clang15, Clang17. Is there a version of Clang which has this fixed? Thanks

@markos
Copy link

markos commented Mar 28, 2024

clang 15/17 are already tested on aarch64 linux and they should work fine, could you please paste your cmake cmd?

@RamaMalladiAWS
Copy link

RamaMalladiAWS commented Mar 28, 2024 •
edited
Loading

Ubuntu clang version 15.0.7

/usr/bin/clang++-15  -I/home/ubuntu/SIMD_kernels/src/include -O3 -mcpu=neoverse-n1 -mtune=neoverse-n1 -fwrapv -fno-omit-frame-pointer -fnon-call-exceptions -std=gnu++17 -MD -MT CMakeFiles/SIMD_kernels.dir/src/
unicode_ops.cpp.o -MF CMakeFiles/SIMD_kernels.dir/src/unicode_ops.cpp.o.d -o CMakeFiles/SIMD_kernels.dir/src/unicode_ops.cpp.o -c /home/ubuntu/SIMD_kernels/src/unicode_ops.cpp

@markos
Copy link

markos commented Mar 28, 2024

I was talking about cmake invocation

@markos
Copy link

markos commented Mar 28, 2024

@RamaMalladiAWS sorry, which project is that? there is no file src/unicode_ops.cpp in Vectorscan.

@RamaMalladiAWS
Copy link

Sorry for the mixup, this isn't Vectorscan project. I was trying to build another test and ran into this same Clang issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@markos @gliwka @RamaMalladiAWS