Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[OpenMP] asan doesn't work when offload to host #75651

Open
ye-luo opened this issue Dec 15, 2023 · 10 comments · Fixed by #75757
Open

[OpenMP] asan doesn't work when offload to host #75651

ye-luo opened this issue Dec 15, 2023 · 10 comments · Fixed by #75757
Labels
clang-tools-extra offload openmp:libomptarget OpenMP offload runtime

Comments

@ye-luo
Copy link
Contributor

ye-luo commented Dec 15, 2023

Currently app doesn't link when OpenMP offload to host and address sanitizer are both turned on.

$ clang++ -fopenmp -fopenmp-targets=x86_64-pc-linux-gnu -fsanitize=address main.cpp
/usr/bin/ld: /tmp/main-6e59bc-x86_64-pc-linux-gnu--d95d92.o: in function `asan.module_ctor':
main.cpp:(.text.asan.module_ctor[asan.module_ctor]+0x5): undefined reference to `__asan_init'
/usr/bin/ld: main.cpp:(.text.asan.module_ctor[asan.module_ctor]+0xa): undefined reference to `__asan_version_mismatch_check_v8'
/usr/bin/ld: main.cpp:(.text.asan.module_ctor[asan.module_ctor]+0x24): undefined reference to `__asan_register_elf_globals'
clang: error: linker command failed with exit code 1 (use -v to see invocation)
/scratch3/packages/llvm/master-nightly/bin/clang-linker-wrapper: error: 'clang' failed
clang++: error: linker command failed with exit code 1 (use -v to see invocation)

main.cpp

int main() {}
@ye-luo ye-luo added the clang:openmp OpenMP related changes to Clang label Dec 15, 2023
@ye-luo
Copy link
Contributor Author

ye-luo commented Dec 15, 2023

It works in the llvm shipped by rocm 6.0.

@jhuber6
Copy link
Contributor

jhuber6 commented Dec 15, 2023

Can you provide the output with -v? I'd assume that when targeting the CPU target it should forward the needed runtime for asan.

@ye-luo
Copy link
Contributor Author

ye-luo commented Dec 15, 2023 

yeluo@epyc-server:~/temp$ clang++ -fopenmp -fopenmp-targets=x86_64-pc-linux-gnu -fsanitize=address main.cpp -v
clang version 18.0.0 (https://github.com/llvm/llvm-project.git 954af75cebbdad6de747b1491bd83a204c8b0d6f)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /soft/llvm/master-nightly/bin
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/7
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/7.5.0
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/8
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/9
Selected GCC installation: /usr/lib/gcc/x86_64-linux-gnu/9
Candidate multilib: .;@m64
Candidate multilib: 32;@m32
Candidate multilib: x32;@mx32
Selected multilib: .;@m64
Found CUDA installation: /usr/local/cuda-12.3, version 
Found HIP installation: /opt/rocm-6.0.0, version 6.0.32830
 "/soft/llvm/main-20231101/bin/clang-18" -cc1 -triple x86_64-unknown-linux-gnu -emit-llvm-bc -emit-llvm-uselists -dumpdir a- -disable-free -clear-ast-before-backend -main-file-name main.cpp -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu x86-64 -tune-cpu generic -debugger-tuning=gdb -fdebug-compilation-dir=/home/yeluo/temp -v -fcoverage-compilation-dir=/home/yeluo/temp -resource-dir /soft/llvm/main-20231101/lib/clang/18 -internal-isystem /soft/llvm/main-20231101/lib/clang/18/include/llvm_libc_wrappers -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/x86_64-linux-gnu/c++/9 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/backward -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/x86_64-linux-gnu/c++/9 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/backward -internal-isystem /soft/llvm/main-20231101/lib/clang/18/include -internal-isystem /usr/local/include -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/9/../../../../x86_64-linux-gnu/include -internal-externc-isystem /usr/include/x86_64-linux-gnu -internal-externc-isystem /include -internal-externc-isystem /usr/include -internal-isystem /soft/llvm/main-20231101/lib/clang/18/include -internal-isystem /usr/local/include -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/9/../../../../x86_64-linux-gnu/include -internal-externc-isystem /usr/include/x86_64-linux-gnu -internal-externc-isystem /include -internal-externc-isystem /usr/include -fdeprecated-macro -ferror-limit 19 -fopenmp -fsanitize=address -fsanitize-system-ignorelist=/soft/llvm/main-20231101/lib/clang/18/share/asan_ignorelist.txt -fno-sanitize-memory-param-retval -fsanitize-address-use-after-scope -fsanitize-address-globals-dead-stripping -fno-assume-sane-operator-new -fgnuc-version=4.2.1 -fcxx-exceptions -fexceptions -fcolor-diagnostics -disable-llvm-passes -fopenmp-targets=x86_64-pc-linux-gnu -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o /tmp/main-1c9570.bc -x c++ main.cpp
clang -cc1 version 18.0.0 based upon LLVM 18.0.0git default target x86_64-unknown-linux-gnu
ignoring nonexistent directory "/usr/local/include"
ignoring nonexistent directory "/usr/lib/gcc/x86_64-linux-gnu/9/../../../../x86_64-linux-gnu/include"
ignoring nonexistent directory "/include"
ignoring nonexistent directory "/usr/local/include"
ignoring nonexistent directory "/usr/lib/gcc/x86_64-linux-gnu/9/../../../../x86_64-linux-gnu/include"
ignoring nonexistent directory "/include"
ignoring duplicate directory "/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9"
ignoring duplicate directory "/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/x86_64-linux-gnu/c++/9"
ignoring duplicate directory "/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/backward"
ignoring duplicate directory "/soft/llvm/main-20231101/lib/clang/18/include"
ignoring duplicate directory "/usr/include/x86_64-linux-gnu"
ignoring duplicate directory "/usr/include"
#include "..." search starts here:
#include <...> search starts here:
 /soft/llvm/main-20231101/lib/clang/18/include/llvm_libc_wrappers
 /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9
 /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/x86_64-linux-gnu/c++/9
 /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/backward
 /soft/llvm/main-20231101/lib/clang/18/include
 /usr/include/x86_64-linux-gnu
 /usr/include
End of search list.
 "/soft/llvm/main-20231101/bin/clang-18" -cc1 -triple x86_64-pc-linux-gnu -aux-triple x86_64-unknown-linux-gnu -emit-obj -mrelax-all -dumpdir a- -disable-free -clear-ast-before-backend -main-file-name main.cpp -mrelocation-model pic -pic-level 2 -fhalf-no-semantic-interposition -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu x86-64 -tune-cpu generic -debugger-tuning=gdb -fdebug-compilation-dir=/home/yeluo/temp -v -fcoverage-compilation-dir=/home/yeluo/temp -resource-dir /soft/llvm/main-20231101/lib/clang/18 -internal-isystem /soft/llvm/main-20231101/lib/clang/18/include/llvm_libc_wrappers -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/x86_64-linux-gnu/c++/9 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/backward -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/x86_64-linux-gnu/c++/9 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/backward -internal-isystem /soft/llvm/main-20231101/lib/clang/18/include -internal-isystem /usr/local/include -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/9/../../../../x86_64-linux-gnu/include -internal-externc-isystem /usr/include/x86_64-linux-gnu -internal-externc-isystem /include -internal-externc-isystem /usr/include -internal-isystem /soft/llvm/main-20231101/lib/clang/18/include -internal-isystem /usr/local/include -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/9/../../../../x86_64-linux-gnu/include -internal-externc-isystem /usr/include/x86_64-linux-gnu -internal-externc-isystem /include -internal-externc-isystem /usr/include -fdeprecated-macro -ferror-limit 19 -fvisibility=protected -fopenmp -fsanitize=address -fsanitize-system-ignorelist=/soft/llvm/main-20231101/lib/clang/18/share/asan_ignorelist.txt -fno-sanitize-memory-param-retval -fsanitize-address-use-after-scope -fsanitize-address-globals-dead-stripping -fno-assume-sane-operator-new -fgnuc-version=4.2.1 -fcxx-exceptions -fexceptions -fcolor-diagnostics -fopenmp-is-target-device -fopenmp-host-ir-file-path /tmp/main-1c9570.bc -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o /tmp/main-577f50.o -x c++ main.cpp
clang -cc1 version 18.0.0 based upon LLVM 18.0.0git default target x86_64-unknown-linux-gnu
ignoring nonexistent directory "/usr/local/include"
ignoring nonexistent directory "/usr/lib/gcc/x86_64-linux-gnu/9/../../../../x86_64-linux-gnu/include"
ignoring nonexistent directory "/include"
ignoring nonexistent directory "/usr/local/include"
ignoring nonexistent directory "/usr/lib/gcc/x86_64-linux-gnu/9/../../../../x86_64-linux-gnu/include"
ignoring nonexistent directory "/include"
ignoring duplicate directory "/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9"
ignoring duplicate directory "/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/x86_64-linux-gnu/c++/9"
ignoring duplicate directory "/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/backward"
ignoring duplicate directory "/soft/llvm/main-20231101/lib/clang/18/include"
ignoring duplicate directory "/usr/include/x86_64-linux-gnu"
ignoring duplicate directory "/usr/include"
#include "..." search starts here:
#include <...> search starts here:
 /soft/llvm/main-20231101/lib/clang/18/include/llvm_libc_wrappers
 /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9
 /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/x86_64-linux-gnu/c++/9
 /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/backward
 /soft/llvm/main-20231101/lib/clang/18/include
 /usr/include/x86_64-linux-gnu
 /usr/include
End of search list.
 "/soft/llvm/master-nightly/bin/clang-offload-packager" -o /tmp/main-495293.out --image=file=/tmp/main-577f50.o,triple=x86_64-pc-linux-gnu,arch=,kind=openmp
 "/soft/llvm/main-20231101/bin/clang-18" -cc1 -triple x86_64-unknown-linux-gnu -emit-obj -mrelax-all -dumpdir a- -disable-free -clear-ast-before-backend -main-file-name main.cpp -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu x86-64 -tune-cpu generic -debugger-tuning=gdb -fdebug-compilation-dir=/home/yeluo/temp -v -fcoverage-compilation-dir=/home/yeluo/temp -resource-dir /soft/llvm/main-20231101/lib/clang/18 -ferror-limit 19 -fopenmp -fsanitize=address -fsanitize-system-ignorelist=/soft/llvm/main-20231101/lib/clang/18/share/asan_ignorelist.txt -fno-sanitize-memory-param-retval -fsanitize-address-use-after-scope -fsanitize-address-globals-dead-stripping -fno-assume-sane-operator-new -fgnuc-version=4.2.1 -fcolor-diagnostics -fembed-offload-object=/tmp/main-495293.out -fopenmp-targets=x86_64-pc-linux-gnu -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o /tmp/main-305e74.o -x ir /tmp/main-1c9570.bc
clang -cc1 version 18.0.0 based upon LLVM 18.0.0git default target x86_64-unknown-linux-gnu
 "/soft/llvm/master-nightly/bin/clang-linker-wrapper" --host-triple=x86_64-unknown-linux-gnu --wrapper-verbose --linker-path=/usr/bin/ld -- -pie -z relro --hash-style=gnu --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o a.out /usr/lib/x86_64-linux-gnu/Scrt1.o /usr/lib/x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/9/crtbeginS.o -L/soft/llvm/main-20231101/bin/../lib/x86_64-unknown-linux-gnu -L/soft/llvm/main-20231101/lib/clang/18/lib/x86_64-unknown-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/9 -L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu -L/lib -L/usr/lib --whole-archive /soft/llvm/main-20231101/lib/clang/18/lib/x86_64-unknown-linux-gnu/libclang_rt.asan_static.a --no-whole-archive --whole-archive /soft/llvm/main-20231101/lib/clang/18/lib/x86_64-unknown-linux-gnu/libclang_rt.asan.a --no-whole-archive --dynamic-list=/soft/llvm/main-20231101/lib/clang/18/lib/x86_64-unknown-linux-gnu/libclang_rt.asan.a.syms --whole-archive /soft/llvm/main-20231101/lib/clang/18/lib/x86_64-unknown-linux-gnu/libclang_rt.asan_cxx.a --no-whole-archive --dynamic-list=/soft/llvm/main-20231101/lib/clang/18/lib/x86_64-unknown-linux-gnu/libclang_rt.asan_cxx.a.syms /tmp/main-305e74.o -lstdc++ -lm --no-as-needed -lpthread -lrt -lm -ldl -lresolv -lomp -lomptarget -lomptarget.devicertl -L/soft/llvm/main-20231101/lib -lgcc_s -lgcc -lpthread -lc -lgcc_s -lgcc /usr/lib/gcc/x86_64-linux-gnu/9/crtendS.o /usr/lib/x86_64-linux-gnu/crtn.o
 "/soft/llvm/main-20231101/bin/clang" -o /tmp/a.out.x86_64.native-ed6b27.img --target=x86_64-pc-linux-gnu -march=native -O2 -Wl,--no-undefined /tmp/main-305e74-x86_64-pc-linux-gnu--bb29fd.o -Wl,-Bsymbolic -shared -L /soft/llvm/main-20231101/bin/../lib/x86_64-unknown-linux-gnu -L /soft/llvm/main-20231101/lib/clang/18/lib/x86_64-unknown-linux-gnu -L /usr/lib/gcc/x86_64-linux-gnu/9 -L /lib/x86_64-linux-gnu -L /lib/../lib64 -L /usr/lib/x86_64-linux-gnu -L /lib -L /usr/lib -l stdc++ -l m -l pthread -l rt -l m -l dl -l resolv -l omp -l omptarget -l omptarget.devicertl -L /soft/llvm/main-20231101/lib -l gcc_s -l gcc -l pthread -l c -l gcc_s -l gcc -v
clang version 18.0.0 (https://github.com/llvm/llvm-project.git 954af75cebbdad6de747b1491bd83a204c8b0d6f)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /soft/llvm/main-20231101/bin
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/7
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/7.5.0
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/8
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/9
Selected GCC installation: /usr/lib/gcc/x86_64-linux-gnu/9
Candidate multilib: .;@m64
Candidate multilib: 32;@m32
Candidate multilib: x32;@mx32
Selected multilib: .;@m64
Found CUDA installation: /usr/local/cuda-12.3, version 
Found HIP installation: /opt/rocm-6.0.0, version 6.0.32830
 "/usr/bin/ld" -z relro --hash-style=gnu --eh-frame-hdr -m elf_x86_64 -shared -o /tmp/a.out.x86_64.native-ed6b27.img /usr/lib/x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/9/crtbeginS.o -L/soft/llvm/main-20231101/bin/../lib/x86_64-unknown-linux-gnu -L/soft/llvm/main-20231101/lib/clang/18/lib/x86_64-unknown-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/9 -L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu -L/lib -L/usr/lib -L/soft/llvm/main-20231101/lib -L/usr/lib/gcc/x86_64-linux-gnu/9 -L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu -L/lib -L/usr/lib --no-undefined /tmp/main-305e74-x86_64-pc-linux-gnu--bb29fd.o -Bsymbolic -lstdc++ -lm -lpthread -lrt -lm -ldl -lresolv -lomp -lomptarget -lomptarget.devicertl -lgcc_s -lgcc -lpthread -lc -lgcc_s -lgcc -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/lib/gcc/x86_64-linux-gnu/9/crtendS.o /usr/lib/x86_64-linux-gnu/crtn.o
/usr/bin/ld: /tmp/main-305e74-x86_64-pc-linux-gnu--bb29fd.o: in function `asan.module_ctor':
main.cpp:(.text.asan.module_ctor[asan.module_ctor]+0x5): undefined reference to `__asan_init'
/usr/bin/ld: main.cpp:(.text.asan.module_ctor[asan.module_ctor]+0xa): undefined reference to `__asan_version_mismatch_check_v8'
/usr/bin/ld: main.cpp:(.text.asan.module_ctor[asan.module_ctor]+0x24): undefined reference to `__asan_register_elf_globals'
clang: error: linker command failed with exit code 1 (use -v to see invocation)
/soft/llvm/master-nightly/bin/clang-linker-wrapper: error: 'clang' failed
clang++: error: linker command failed with exit code 1 (use -v to see invocation)

@jdoerfert jdoerfert added openmp:libomptarget OpenMP offload runtime offload labels Dec 15, 2023
@jdoerfert
Copy link
Member

You build compiler-rt, I assume?
We probably need to compare the cc1 command (-v) for offloading to X86 with the one for native compilation, might just be a missed library.

@jhuber6
Copy link
Contributor

jhuber6 commented Dec 15, 2023

Probably a bug in the LinkerWrapper. There's --whole-archive /home/jhuber/Documents/llvm/clang/lib/clang/18/lib/x86_64-unknown-linux-gnu/libclang_rt.asan_static.a in the clang-linker-wrapper invocation but not in the actual link job.

@ye-luo
Copy link
Contributor Author

ye-luo commented Dec 16, 2023 •
edited

yes. I always enable compiler-rt. Works without offload.

@jhuber6 jhuber6 mentioned this issue Dec 18, 2023
Merged
jhuber6 added a commit to jhuber6/llvm-project that referenced this issue Dec 18, 2023
@jhuber6
[LinkerWrapper] Forward more arguments to the CPU offloading linker
0ba0fa0 
Summary:
The CPU target currently inherits all the libraries from the normal link
job to ensure that it has access to the same envrionment that the host
does. However, this previously was not respecting argument libraries
that are passed by name rather than `-l` as well as the whole archive
flags. This patch fixes this to allow the CPU linker to correctly pick
up the libraries associated with things like address sanitizers.

Fixes: llvm#75651
@EugeneZelenko EugeneZelenko added clang-tools-extra and removed clang:openmp OpenMP related changes to Clang labels Dec 18, 2023
@ye-luo
Copy link
Contributor Author

ye-luo commented Dec 18, 2023

Still doesn't fullly work on my side. Now I got.

yeluo@epyc-server:~/temp$ clang++ -fopenmp -fopenmp-targets=x86_64-pc-linux-gnu -fsanitize=address main.cpp 
/usr/bin/ld: /soft/llvm/main-20231218/lib/clang/18/lib/x86_64-unknown-linux-gnu/libclang_rt.asan.a(asan_preinit.cpp.o): .preinit_array section is not allowed in DSO
/usr/bin/ld: failed to set dynamic section sizes: nonrepresentable section on output
clang: error: linker command failed with exit code 1 (use -v to see invocation)
/soft/llvm/master-nightly/bin/clang-linker-wrapper: error: 'clang' failed
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
yeluo@epyc-server:~/temp$ clang++ -fopenmp -fopenmp-targets=x86_64-pc-linux-gnu -fsanitize=address -shared-libsan main.cpp 
/usr/bin/ld: /soft/llvm/main-20231218/lib/clang/18/lib/x86_64-unknown-linux-gnu/libclang_rt.asan-preinit.a(asan_preinit.cpp.o): .preinit_array section is not allowed in DSO
/usr/bin/ld: failed to set dynamic section sizes: nonrepresentable section on output
clang: error: linker command failed with exit code 1 (use -v to see invocation)
/soft/llvm/master-nightly/bin/clang-linker-wrapper: error: 'clang' failed
clang++: error: linker command failed with exit code 1 (use -v to see invocation)

Any idea?

@ye-luo ye-luo reopened this Dec 18, 2023
@jhuber6
Copy link
Contributor

jhuber6 commented Dec 18, 2023

It worked when I tried compiling, but I ended up with some weird bug about cuInit running out of resources. Can you run w/ -v again?

@ye-luo
Copy link
Contributor Author

ye-luo commented Dec 19, 2023

Current output.

$ clang++ -fopenmp -fopenmp-targets=x86_64-pc-linux-gnu -fsanitize=address main.cpp -v
clang version 18.0.0git (https://github.com/llvm/llvm-project.git 8e2cc19e2e60d348859c0967a2707bcc8cb9936a)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /soft/llvm/master-nightly/bin
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/7
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/7.5.0
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/8
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/9
Selected GCC installation: /usr/lib/gcc/x86_64-linux-gnu/9
Candidate multilib: .;@m64
Candidate multilib: 32;@m32
Candidate multilib: x32;@mx32
Selected multilib: .;@m64
Found CUDA installation: /usr/local/cuda-12.3, version 12.3
Found HIP installation: /opt/rocm-6.0.0, version 6.0.32830
 "/soft/llvm/main-20231218/bin/clang-18" -cc1 -triple x86_64-unknown-linux-gnu -emit-llvm-bc -emit-llvm-uselists -dumpdir a- -disable-free -clear-ast-before-backend -main-file-name main.cpp -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu x86-64 -tune-cpu generic -debugger-tuning=gdb -fdebug-compilation-dir=/home/yeluo/temp -v -fcoverage-compilation-dir=/home/yeluo/temp -resource-dir /soft/llvm/main-20231218/lib/clang/18 -internal-isystem /soft/llvm/main-20231218/lib/clang/18/include/llvm_libc_wrappers -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/x86_64-linux-gnu/c++/9 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/backward -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/x86_64-linux-gnu/c++/9 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/backward -internal-isystem /soft/llvm/main-20231218/lib/clang/18/include -internal-isystem /usr/local/include -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/9/../../../../x86_64-linux-gnu/include -internal-externc-isystem /usr/include/x86_64-linux-gnu -internal-externc-isystem /include -internal-externc-isystem /usr/include -internal-isystem /soft/llvm/main-20231218/lib/clang/18/include -internal-isystem /usr/local/include -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/9/../../../../x86_64-linux-gnu/include -internal-externc-isystem /usr/include/x86_64-linux-gnu -internal-externc-isystem /include -internal-externc-isystem /usr/include -fdeprecated-macro -ferror-limit 19 -fopenmp -fsanitize=address -fsanitize-system-ignorelist=/soft/llvm/main-20231218/lib/clang/18/share/asan_ignorelist.txt -fno-sanitize-memory-param-retval -fsanitize-address-use-after-scope -fsanitize-address-globals-dead-stripping -fno-assume-sane-operator-new -fgnuc-version=4.2.1 -fcxx-exceptions -fexceptions -fcolor-diagnostics -disable-llvm-passes -fopenmp-targets=x86_64-pc-linux-gnu -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o /tmp/main-82c5d7.bc -x c++ main.cpp
clang -cc1 version 18.0.0git based upon LLVM 18.0.0git default target x86_64-unknown-linux-gnu
ignoring nonexistent directory "/usr/local/include"
ignoring nonexistent directory "/usr/lib/gcc/x86_64-linux-gnu/9/../../../../x86_64-linux-gnu/include"
ignoring nonexistent directory "/include"
ignoring nonexistent directory "/usr/local/include"
ignoring nonexistent directory "/usr/lib/gcc/x86_64-linux-gnu/9/../../../../x86_64-linux-gnu/include"
ignoring nonexistent directory "/include"
ignoring duplicate directory "/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9"
ignoring duplicate directory "/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/x86_64-linux-gnu/c++/9"
ignoring duplicate directory "/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/backward"
ignoring duplicate directory "/soft/llvm/main-20231218/lib/clang/18/include"
ignoring duplicate directory "/usr/include/x86_64-linux-gnu"
ignoring duplicate directory "/usr/include"
#include "..." search starts here:
#include <...> search starts here:
 /soft/llvm/main-20231218/lib/clang/18/include/llvm_libc_wrappers
 /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9
 /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/x86_64-linux-gnu/c++/9
 /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/backward
 /soft/llvm/main-20231218/lib/clang/18/include
 /usr/include/x86_64-linux-gnu
 /usr/include
End of search list.
 "/soft/llvm/main-20231218/bin/clang-18" -cc1 -triple x86_64-pc-linux-gnu -aux-triple x86_64-unknown-linux-gnu -emit-obj -mrelax-all -dumpdir a- -disable-free -clear-ast-before-backend -main-file-name main.cpp -mrelocation-model pic -pic-level 2 -fhalf-no-semantic-interposition -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu x86-64 -tune-cpu generic -debugger-tuning=gdb -fdebug-compilation-dir=/home/yeluo/temp -v -fcoverage-compilation-dir=/home/yeluo/temp -resource-dir /soft/llvm/main-20231218/lib/clang/18 -internal-isystem /soft/llvm/main-20231218/lib/clang/18/include/llvm_libc_wrappers -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/x86_64-linux-gnu/c++/9 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/backward -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/x86_64-linux-gnu/c++/9 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/backward -internal-isystem /soft/llvm/main-20231218/lib/clang/18/include -internal-isystem /usr/local/include -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/9/../../../../x86_64-linux-gnu/include -internal-externc-isystem /usr/include/x86_64-linux-gnu -internal-externc-isystem /include -internal-externc-isystem /usr/include -internal-isystem /soft/llvm/main-20231218/lib/clang/18/include -internal-isystem /usr/local/include -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/9/../../../../x86_64-linux-gnu/include -internal-externc-isystem /usr/include/x86_64-linux-gnu -internal-externc-isystem /include -internal-externc-isystem /usr/include -fdeprecated-macro -ferror-limit 19 -fvisibility=protected -fopenmp -fsanitize=address -fsanitize-system-ignorelist=/soft/llvm/main-20231218/lib/clang/18/share/asan_ignorelist.txt -fno-sanitize-memory-param-retval -fsanitize-address-use-after-scope -fsanitize-address-globals-dead-stripping -fno-assume-sane-operator-new -fgnuc-version=4.2.1 -fcxx-exceptions -fexceptions -fcolor-diagnostics -fopenmp-is-target-device -fopenmp-host-ir-file-path /tmp/main-82c5d7.bc -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o /tmp/main-257bd7.o -x c++ main.cpp
clang -cc1 version 18.0.0git based upon LLVM 18.0.0git default target x86_64-unknown-linux-gnu
ignoring nonexistent directory "/usr/local/include"
ignoring nonexistent directory "/usr/lib/gcc/x86_64-linux-gnu/9/../../../../x86_64-linux-gnu/include"
ignoring nonexistent directory "/include"
ignoring nonexistent directory "/usr/local/include"
ignoring nonexistent directory "/usr/lib/gcc/x86_64-linux-gnu/9/../../../../x86_64-linux-gnu/include"
ignoring nonexistent directory "/include"
ignoring duplicate directory "/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9"
ignoring duplicate directory "/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/x86_64-linux-gnu/c++/9"
ignoring duplicate directory "/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/backward"
ignoring duplicate directory "/soft/llvm/main-20231218/lib/clang/18/include"
ignoring duplicate directory "/usr/include/x86_64-linux-gnu"
ignoring duplicate directory "/usr/include"
#include "..." search starts here:
#include <...> search starts here:
 /soft/llvm/main-20231218/lib/clang/18/include/llvm_libc_wrappers
 /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9
 /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/x86_64-linux-gnu/c++/9
 /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/backward
 /soft/llvm/main-20231218/lib/clang/18/include
 /usr/include/x86_64-linux-gnu
 /usr/include
End of search list.
 "/soft/llvm/master-nightly/bin/clang-offload-packager" -o /tmp/main-19f3a1.out --image=file=/tmp/main-257bd7.o,triple=x86_64-pc-linux-gnu,arch=,kind=openmp
 "/soft/llvm/main-20231218/bin/clang-18" -cc1 -triple x86_64-unknown-linux-gnu -emit-obj -mrelax-all -dumpdir a- -disable-free -clear-ast-before-backend -main-file-name main.cpp -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu x86-64 -tune-cpu generic -debugger-tuning=gdb -fdebug-compilation-dir=/home/yeluo/temp -v -fcoverage-compilation-dir=/home/yeluo/temp -resource-dir /soft/llvm/main-20231218/lib/clang/18 -ferror-limit 19 -fopenmp -fsanitize=address -fsanitize-system-ignorelist=/soft/llvm/main-20231218/lib/clang/18/share/asan_ignorelist.txt -fno-sanitize-memory-param-retval -fsanitize-address-use-after-scope -fsanitize-address-globals-dead-stripping -fno-assume-sane-operator-new -fgnuc-version=4.2.1 -fcolor-diagnostics -fembed-offload-object=/tmp/main-19f3a1.out -fopenmp-targets=x86_64-pc-linux-gnu -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o /tmp/main-5164cd.o -x ir /tmp/main-82c5d7.bc
clang -cc1 version 18.0.0git based upon LLVM 18.0.0git default target x86_64-unknown-linux-gnu
 "/soft/llvm/master-nightly/bin/clang-linker-wrapper" --host-triple=x86_64-unknown-linux-gnu --wrapper-verbose --linker-path=/usr/bin/ld -- -z relro --hash-style=gnu --eh-frame-hdr -m elf_x86_64 -pie -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o a.out /usr/lib/x86_64-linux-gnu/Scrt1.o /usr/lib/x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/9/crtbeginS.o -L/soft/llvm/main-20231218/bin/../lib/x86_64-unknown-linux-gnu -L/soft/llvm/main-20231218/lib/clang/18/lib/x86_64-unknown-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/9 -L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu -L/lib -L/usr/lib --whole-archive /soft/llvm/main-20231218/lib/clang/18/lib/x86_64-unknown-linux-gnu/libclang_rt.asan_static.a --no-whole-archive --whole-archive /soft/llvm/main-20231218/lib/clang/18/lib/x86_64-unknown-linux-gnu/libclang_rt.asan.a --no-whole-archive --dynamic-list=/soft/llvm/main-20231218/lib/clang/18/lib/x86_64-unknown-linux-gnu/libclang_rt.asan.a.syms --whole-archive /soft/llvm/main-20231218/lib/clang/18/lib/x86_64-unknown-linux-gnu/libclang_rt.asan_cxx.a --no-whole-archive --dynamic-list=/soft/llvm/main-20231218/lib/clang/18/lib/x86_64-unknown-linux-gnu/libclang_rt.asan_cxx.a.syms /tmp/main-5164cd.o -lstdc++ -lm --no-as-needed -lpthread -lrt -lm -ldl -lresolv -lomp -lomptarget -lomptarget.devicertl -L/soft/llvm/main-20231218/lib -lgcc_s -lgcc -lpthread -lc -lgcc_s -lgcc /usr/lib/gcc/x86_64-linux-gnu/9/crtendS.o /usr/lib/x86_64-linux-gnu/crtn.o
 "/soft/llvm/main-20231218/bin/clang" -o /tmp/a.out.x86_64.native-5bb49f.img --target=x86_64-pc-linux-gnu -march=native -O2 -Wl,--no-undefined /tmp/main-5164cd-x86_64-pc-linux-gnu--499105.o -Wl,-Bsymbolic -shared /lib64/ld-linux-x86-64.so.2 -L /soft/llvm/main-20231218/bin/../lib/x86_64-unknown-linux-gnu -L /soft/llvm/main-20231218/lib/clang/18/lib/x86_64-unknown-linux-gnu -L /usr/lib/gcc/x86_64-linux-gnu/9 -L /lib/x86_64-linux-gnu -L /lib/../lib64 -L /usr/lib/x86_64-linux-gnu -L /lib -L /usr/lib -Wl,--whole-archive /soft/llvm/main-20231218/lib/clang/18/lib/x86_64-unknown-linux-gnu/libclang_rt.asan_static.a -Wl,--no-whole-archive -Wl,--whole-archive /soft/llvm/main-20231218/lib/clang/18/lib/x86_64-unknown-linux-gnu/libclang_rt.asan.a -Wl,--no-whole-archive -Wl,--whole-archive /soft/llvm/main-20231218/lib/clang/18/lib/x86_64-unknown-linux-gnu/libclang_rt.asan_cxx.a -Wl,--no-whole-archive -l stdc++ -l m -l pthread -l rt -l m -l dl -l resolv -l omp -l omptarget -l omptarget.devicertl -L /soft/llvm/main-20231218/lib -l gcc_s -l gcc -l pthread -l c -l gcc_s -l gcc -v
clang version 18.0.0git (https://github.com/llvm/llvm-project.git 8e2cc19e2e60d348859c0967a2707bcc8cb9936a)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /soft/llvm/main-20231218/bin
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/7
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/7.5.0
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/8
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/9
Selected GCC installation: /usr/lib/gcc/x86_64-linux-gnu/9
Candidate multilib: .;@m64
Candidate multilib: 32;@m32
Candidate multilib: x32;@mx32
Selected multilib: .;@m64
Found CUDA installation: /usr/local/cuda-12.3, version 12.3
Found HIP installation: /opt/rocm-6.0.0, version 6.0.32830
 "/usr/bin/ld" -z relro --hash-style=gnu --eh-frame-hdr -m elf_x86_64 -shared -o /tmp/a.out.x86_64.native-5bb49f.img /usr/lib/x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/9/crtbeginS.o -L/soft/llvm/main-20231218/bin/../lib/x86_64-unknown-linux-gnu -L/soft/llvm/main-20231218/lib/clang/18/lib/x86_64-unknown-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/9 -L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu -L/lib -L/usr/lib -L/soft/llvm/main-20231218/lib -L/usr/lib/gcc/x86_64-linux-gnu/9 -L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu -L/lib -L/usr/lib --no-undefined /tmp/main-5164cd-x86_64-pc-linux-gnu--499105.o -Bsymbolic /lib64/ld-linux-x86-64.so.2 --whole-archive /soft/llvm/main-20231218/lib/clang/18/lib/x86_64-unknown-linux-gnu/libclang_rt.asan_static.a --no-whole-archive --whole-archive /soft/llvm/main-20231218/lib/clang/18/lib/x86_64-unknown-linux-gnu/libclang_rt.asan.a --no-whole-archive --whole-archive /soft/llvm/main-20231218/lib/clang/18/lib/x86_64-unknown-linux-gnu/libclang_rt.asan_cxx.a --no-whole-archive -lstdc++ -lm -lpthread -lrt -lm -ldl -lresolv -lomp -lomptarget -lomptarget.devicertl -lgcc_s -lgcc -lpthread -lc -lgcc_s -lgcc -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/lib/gcc/x86_64-linux-gnu/9/crtendS.o /usr/lib/x86_64-linux-gnu/crtn.o
/usr/bin/ld: /soft/llvm/main-20231218/lib/clang/18/lib/x86_64-unknown-linux-gnu/libclang_rt.asan.a(asan_preinit.cpp.o): .preinit_array section is not allowed in DSO
/usr/bin/ld: failed to set dynamic section sizes: nonrepresentable section on output
clang: error: linker command failed with exit code 1 (use -v to see invocation)
/soft/llvm/master-nightly/bin/clang-linker-wrapper: error: 'clang' failed
clang++: error: linker command failed with exit code 1 (use -v to see invocation)

@EugeneZelenko
Copy link
Contributor

Not merged yet.

@EugeneZelenko EugeneZelenko reopened this Dec 20, 2023
qihangkong pushed a commit to rvgpu/rvgpu-llvm that referenced this issue Apr 23, 2024
@jhuber6 @wangxiang090105
[LinkerWrapper] Forward more arguments to the CPU offloading linker (…
c421831 
…#75757)

Summary:
The CPU target currently inherits all the libraries from the normal link
job to ensure that it has access to the same envrionment that the host
does. However, this previously was not respecting argument libraries
that are passed by name rather than `-l` as well as the whole archive
flags. This patch fixes this to allow the CPU linker to correctly pick
up the libraries associated with things like address sanitizers.

Fixes: llvm/llvm-project#75651
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
clang-tools-extra offload openmp:libomptarget OpenMP offload runtime
Projects
LLVM/Offload - Development
Status: Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[LinkerWrapper] Forward more arguments to the CPU offloading linker jhuber6/llvm-project
4 participants
@ye-luo @jdoerfert @EugeneZelenko @jhuber6