Skip to content

Conversation

azat
Copy link
Contributor

@azat azat commented Jul 5, 2024

Right now if you have runtime libraries under
lib/x86_64-unknown-linux-gnu you should use --target x86_64-unknown-linux-gnu, x86_64-pc-linux-gnu will not work.

Treat the interchangeable so that you can use any.

The initial reason for this patch is that debian packages uses x86_64-pc-linux-gnu, and after they enabled
LLVM_ENABLE_PER_TARGET_RUNTIME_DIR 1, clang cannot find runtime libraries for sanitizers.

Fixes: #95792

@llvmbot llvmbot added clang Clang issues not falling into any other category clang:driver 'clang' and 'clang++' user-facing binaries. Not 'clang-cl' labels Jul 5, 2024
@llvmbot
Copy link
Member

llvmbot commented Jul 5, 2024

@llvm/pr-subscribers-clang

@llvm/pr-subscribers-clang-driver

Author: Azat Khuzhin (azat)

Changes

Right now if you have runtime libraries under
lib/x86_64-unknown-linux-gnu you should use --target x86_64-unknown-linux-gnu, x86_64-pc-linux-gnu will not work.

Treat the interchangeable so that you can use any.

The initial reason for this patch is that debian packages uses x86_64-pc-linux-gnu, and after they enabled
LLVM_ENABLE_PER_TARGET_RUNTIME_DIR 1, clang cannot find runtime libraries for sanitizers.

Fixes: #95792


Full diff: https://github.com/llvm/llvm-project/pull/97802.diff

2 Files Affected:

  • (modified) clang/lib/Driver/ToolChain.cpp (+21)
  • (added) clang/test/Driver/pc-unknown-toolchain.c (+4)
diff --git a/clang/lib/Driver/ToolChain.cpp b/clang/lib/Driver/ToolChain.cpp
index 977e08390800d7..716cbfb43232b6 100644
--- a/clang/lib/Driver/ToolChain.cpp
+++ b/clang/lib/Driver/ToolChain.cpp
@@ -765,6 +765,27 @@ ToolChain::getTargetSubDirPath(StringRef BaseDir) const {
   if (auto Path = getPathForTriple(getTriple()))
     return *Path;
 
+  // Treat "pc" and "unknown" vendors interchangeable
+  switch (getTriple().getVendor())
+  {
+    case Triple::UnknownVendor: {
+      llvm::Triple TripleFallback = Triple;
+      TripleFallback.setVendor(Triple::PC);
+      if (auto Path = getPathForTriple(TripleFallback))
+        return *Path;
+      break;
+    }
+    case Triple::PC: {
+      llvm::Triple TripleFallback = Triple;
+      TripleFallback.setVendor(Triple::UnknownVendor);
+      if (auto Path = getPathForTriple(TripleFallback))
+        return *Path;
+      break;
+    }
+    default:
+      break;
+  }
+
   // When building with per target runtime directories, various ways of naming
   // the Arm architecture may have been normalised to simply "arm".
   // For example "armv8l" (Armv8 AArch32 little endian) is replaced with "arm".
diff --git a/clang/test/Driver/pc-unknown-toolchain.c b/clang/test/Driver/pc-unknown-toolchain.c
new file mode 100644
index 00000000000000..a7e9bb80b0f2d6
--- /dev/null
+++ b/clang/test/Driver/pc-unknown-toolchain.c
@@ -0,0 +1,4 @@
+// RUN: %clang -### %s -fsanitize=address --target=x86_64-pc-linux-gnu 2>&1 | FileCheck -check-prefix=CHECK-ASAN-PC %s
+// CHECK-ASAN-PC: x86_64-unknown-linux-gnu/libclang_rt.asan_static.a
+// RUN: %clang -### %s -fsanitize=address --target=x86_64-unknown-linux-gnu 2>&1 | FileCheck -check-prefix=CHECK-ASAN-UNKNOWN %s
+// CHECK-ASAN-UNKNOWN: x86_64-unknown-linux-gnu/libclang_rt.asan_static.a

Copy link

github-actions bot commented Jul 5, 2024

✅ With the latest revision this PR passed the C/C++ code formatter.

@azat azat force-pushed the pc-unknown-toolchain branch from dd290dd to c2d4c46 Compare July 5, 2024 09:34
Right now if you have runtime libraries under
lib/x86_64-unknown-linux-gnu you should use --target
x86_64-unknown-linux-gnu, x86_64-pc-linux-gnu will not work.

Treat the interchangeable so that you can use any.

The initial reason for this patch is that debian packages uses
x86_64-pc-linux-gnu, and after they enabled
LLVM_ENABLE_PER_TARGET_RUNTIME_DIR [1], clang cannot find runtime
libraries for sanitizers.

  [1]: https://salsa.debian.org/pkg-llvm-team/llvm-toolchain/-/commit/9ca35f30383d89e4fdd45d15e0eb82c832df4b8c
@azat azat force-pushed the pc-unknown-toolchain branch from c2d4c46 to 1d78d83 Compare July 5, 2024 10:43
@azat
Copy link
Contributor Author

azat commented Jul 5, 2024

I've excluded new test for windows (https://buildkite.com/llvm-project/github-pull-requests/builds/78458)

@MaskRay
Copy link
Member

MaskRay commented Jul 5, 2024

How is Rust built? If the system compiler-rt libraries are installed to /usr/lib/llvm-18/lib/clang/18/lib/x86_64-pc-linux-gnu, which means that LLVM_DEFAULT_TARGET_TRIPLE=x86_64-pc-linux-gnu, the Rust build should also use x86_64-pc-linux-gnu.

If we don't consider Rust, Clang itself can find the library whether or not "pc" or "unknown" is used.

Copy link
Member

@MaskRay MaskRay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.

@azat
Copy link
Contributor Author

azat commented Jul 5, 2024

How is Rust built? If the system compiler-rt libraries are installed to /usr/lib/llvm-18/lib/clang/18/lib/x86_64-pc-linux-gnu, which means that LLVM_DEFAULT_TARGET_TRIPLE=x86_64-pc-linux-gnu, the Rust build should also use x86_64-pc-linux-gnu.

The problem is not only with Rust, it is any invocation with non-default --target (--target != LLVM_DEFAULT_TARGET_TRIPLE), and I was fixing a C++ issue not a Rust issue, but I guess it will be covered

Here is an example of a problem (assuming LLVM_DEFAULT_TARGET_TRIPLE is x86_64-pc-linux-gnu (in packages from apt.llvm.org)):

  • without explicit -target - everything is correct, it uses /usr/lib/llvm-18/lib/clang/18/lib/x86_64-pc-linux-gnu/libclang_rt.asan.a
$ docker run --rm -it clickhouse/binary-builder:9e5aa3a43749 clang-18 -### -xc - -fsanitize=address
Ubuntu clang version 18.1.8 (++20240615103753+3b5b5c1ec4a3-1~exp1~20240615223858.136)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
 "/usr/lib/llvm-18/bin/clang" "-cc1" "-triple" "x86_64-pc-linux-gnu" "-emit-obj" "-mrelax-all" "-dumpdir" "a-" "-disable-free" "-clear-ast-before-backend" "-disable-llvm-verifier" "-discard-value-names" "-main-file-name" "-" "-mrelocation-model" "pic" "-pic-level" "2" "-pic-is-pie" "-mframe-pointer=all" "-fmath-errno" "-ffp-contract=on" "-fno-rounding-math" "-mconstructor-aliases" "-funwind-tables=2" "-target-cpu" "x86-64" "-tune-cpu" "generic" "-debugger-tuning=gdb" "-fdebug-compilation-dir=/workdir" "-fcoverage-compilation-dir=/workdir" "-resource-dir" "/usr/lib/llvm-18/lib/clang/18" "-internal-isystem" "/usr/lib/llvm-18/lib/clang/18/include" "-internal-isystem" "/usr/local/include" "-internal-isystem" "/usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../x86_64-linux-gnu/include" "-internal-externc-isystem" "/usr/include/x86_64-linux-gnu" "-internal-externc-isystem" "/include" "-internal-externc-isystem" "/usr/include" "-ferror-limit" "19" "-fmessage-length=295" "-fsanitize=address" "-fsanitize-system-ignorelist=/usr/lib/llvm-18/lib/clang/18/share/asan_ignorelist.txt" "-fno-sanitize-memory-param-retval" "-fsanitize-address-use-after-scope" "-fsanitize-address-globals-dead-stripping" "-fno-assume-sane-operator-new" "-fgnuc-version=4.2.1" "-fskip-odr-check-in-gmf" "-fcolor-diagnostics" "-faddrsig" "-D__GCC_HAVE_DWARF2_CFI_ASM=1" "-o" "/tmp/--ed21dd.o" "-x" "c" "-"
 "/usr/bin/ld" "-z" "relro" "--hash-style=gnu" "--build-id" "--eh-frame-hdr" "-m" "elf_x86_64" "-pie" "-dynamic-linker" "/lib64/ld-linux-x86-64.so.2" "-o" "a.out" "/lib/x86_64-linux-gnu/Scrt1.o" "/lib/x86_64-linux-gnu/crti.o" "/usr/bin/../lib/gcc/x86_64-linux-gnu/11/crtbeginS.o" "-L/usr/lib/llvm-18/lib/clang/18/lib/x86_64-pc-linux-gnu" "-L/usr/bin/../lib/gcc/x86_64-linux-gnu/11" "-L/usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../lib64" "-L/lib/x86_64-linux-gnu" "-L/lib/../lib64" "-L/usr/lib/x86_64-linux-gnu" "-L/usr/lib/../lib64" "-L/lib" "-L/usr/lib" "--whole-archive" "/usr/lib/llvm-18/lib/clang/18/lib/x86_64-pc-linux-gnu/libclang_rt.asan_static.a" "--no-whole-archive" "--whole-archive" "/usr/lib/llvm-18/lib/clang/18/lib/x86_64-pc-linux-gnu/libclang_rt.asan.a" "--no-whole-archive" "--dynamic-list=/usr/lib/llvm-18/lib/clang/18/lib/x86_64-pc-linux-gnu/libclang_rt.asan.a.syms" "/tmp/--ed21dd.o" "--no-as-needed" "-lpthread" "-lrt" "-lm" "-ldl" "-lresolv" "-lgcc" "--as-needed" "-lgcc_s" "--no-as-needed" "-lc" "-lgcc" "--as-needed" "-lgcc_s" "--no-as-needed" "/usr/bin/../lib/gcc/x86_64-linux-gnu/11/crtendS.o" "/lib/x86_64-linux-gnu/crtn.o"
  • but with explicit -target=x86_64-unknown-linux-gnu, it try to use /usr/lib/llvm-18/lib/clang/18/lib/linux/libclang_rt.asan-x86_64.a (because there is no such file /usr/lib/llvm-18/lib/clang/18/lib/x86_64-unknown-linux-gnu/libclang_rt.asan.a), which obviously does not exists, and it will fail eventually
$ docker run --rm -it clickhouse/binary-builder:9e5aa3a43749 clang-18 -target x86_64-unknown-linux-gnu -### -xc - -fsanitize=address
Ubuntu clang version 18.1.8 (++20240615103753+3b5b5c1ec4a3-1~exp1~20240615223858.136)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
 "/usr/lib/llvm-18/bin/clang" "-cc1" "-triple" "x86_64-unknown-linux-gnu" "-emit-obj" "-mrelax-all" "-dumpdir" "a-" "-disable-free" "-clear-ast-before-backend" "-disable-llvm-verifier" "-discard-value-names" "-main-file-name" "-" "-mrelocation-model" "pic" "-pic-level" "2" "-pic-is-pie" "-mframe-pointer=all" "-fmath-errno" "-ffp-contract=on" "-fno-rounding-math" "-mconstructor-aliases" "-funwind-tables=2" "-target-cpu" "x86-64" "-tune-cpu" "generic" "-debugger-tuning=gdb" "-fdebug-compilation-dir=/workdir" "-fcoverage-compilation-dir=/workdir" "-resource-dir" "/usr/lib/llvm-18/lib/clang/18" "-internal-isystem" "/usr/lib/llvm-18/lib/clang/18/include" "-internal-isystem" "/usr/local/include" "-internal-isystem" "/usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../x86_64-linux-gnu/include" "-internal-externc-isystem" "/usr/include/x86_64-linux-gnu" "-internal-externc-isystem" "/include" "-internal-externc-isystem" "/usr/include" "-ferror-limit" "19" "-fmessage-length=295" "-fsanitize=address" "-fsanitize-system-ignorelist=/usr/lib/llvm-18/lib/clang/18/share/asan_ignorelist.txt" "-fno-sanitize-memory-param-retval" "-fsanitize-address-use-after-scope" "-fsanitize-address-globals-dead-stripping" "-fno-assume-sane-operator-new" "-fgnuc-version=4.2.1" "-fskip-odr-check-in-gmf" "-fcolor-diagnostics" "-faddrsig" "-D__GCC_HAVE_DWARF2_CFI_ASM=1" "-o" "/tmp/--85ef6b.o" "-x" "c" "-"
 "/usr/bin/ld" "-z" "relro" "--hash-style=gnu" "--build-id" "--eh-frame-hdr" "-m" "elf_x86_64" "-pie" "-dynamic-linker" "/lib64/ld-linux-x86-64.so.2" "-o" "a.out" "/lib/x86_64-linux-gnu/Scrt1.o" "/lib/x86_64-linux-gnu/crti.o" "/usr/bin/../lib/gcc/x86_64-linux-gnu/11/crtbeginS.o" "-L/usr/bin/../lib/gcc/x86_64-linux-gnu/11" "-L/usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../lib64" "-L/lib/x86_64-linux-gnu" "-L/lib/../lib64" "-L/usr/lib/x86_64-linux-gnu" "-L/usr/lib/../lib64" "-L/lib" "-L/usr/lib" "--whole-archive" "/usr/lib/llvm-18/lib/clang/18/lib/linux/libclang_rt.asan_static-x86_64.a" "--no-whole-archive" "--whole-archive" "/usr/lib/llvm-18/lib/clang/18/lib/linux/libclang_rt.asan-x86_64.a" "--no-whole-archive" "--export-dynamic" "/tmp/--85ef6b.o" "--no-as-needed" "-lpthread" "-lrt" "-lm" "-ldl" "-lresolv" "-lgcc" "--as-needed" "-lgcc_s" "--no-as-needed" "-lc" "-lgcc" "--as-needed" "-lgcc_s" "--no-as-needed" "/usr/bin/../lib/gcc/x86_64-linux-gnu/11/crtendS.o" "/lib/x86_64-linux-gnu/crtn.o"

Maybe you have better/proper idea on how this should be addressed?

@azat
Copy link
Contributor Author

azat commented Jul 9, 2024

@MaskRay you still think that it does not worth to make this two targets (x86_64-**pc**-linux-gnu and x86_64-**unknown**-linux-gnu) interchangeable?

@MaskRay
Copy link
Member

MaskRay commented Jul 10, 2024

The problem is not only with Rust, it is any invocation with non-default --target (--target != LLVM_DEFAULT_TARGET_TRIPLE), and I was fixing a C++ issue not a Rust issue, but I guess it will be covered

If you specify the wrong --target= (avoid -target , deprecated since Clang 3.4), clangDriver will not find the compiler-rt library.
This works as intended.

If you mix target triples for LTO you will get warnings:

% clang --target=x86_64-pc-linux-gnu -flto -c a.c && clang --target=x86_64-unkwown-linux-gnu -flto -c b.c && clang -flto -fuse-ld=lld a.o b.o
ld.lld: warning: Linking two modules of different target triples: 'b.o' is 'x86_64-unkwown-linux-gnu' whereas 'ld-temp.o' is 'x86_64-pc-linux-gnu'

@MaskRay you still think that it does not worth to make this two targets (x86_64-pc-linux-gnu and x86_64-unknown-linux-gnu) interchangeable?

Try fixing the build system issue instead:) I hope that we don't make Clang less strict.

@azat
Copy link
Contributor Author

azat commented Jul 10, 2024

If you specify the wrong --target= (avoid -target , deprecated since Clang 3.4), clangDriver will not find the compiler-rt library.
This works as intended.

But isn't this odd, that you depends on LLVM_DEFAULT_TARGET_TRIPLE?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
clang:driver 'clang' and 'clang++' user-facing binaries. Not 'clang-cl' clang Clang issues not falling into any other category
Projects
None yet
Development

Successfully merging this pull request may close these issues.

LLVM 18.1.8 breaks paths for sanitizers in connection with Rust
4 participants