[clang] clang driver fails to find headers/libraries when specified --gcc-install-dir or --gcc-toolchain or --sysroot to a cross-compiler directory #114210
Description
I am trying to use clang 18 to cross compile a program for Arm 32-bit target.
The clang 18 was installed from the official LLVM Ubuntu repository following the instructions here.
The GCC toolchain I was using is from bootlin.
The rough GCC toolchain directory layout is:
/path/to/bootlin/toolchain
├── arm-buildroot-linux-gnueabihf
│ ├── bin
│ ├── include
│ │ └── c++
│ ├── lib
│ └── sysroot
│ ├── bin
│ ├── dev
│ ├── etc
│ ├── lib
│ ├── lib32 -> lib
│ ├── media
│ ├── mnt
│ ├── opt
│ ├── proc
│ ├── root
│ ├── run
│ ├── sbin
│ ├── sys
│ ├── tmp
│ ├── usr
│ └── var
├── bin
├── etc
├── include
├── lib
│ ├── gcc
│ │ └── arm-buildroot-linux-gnueabihf
...
├── lib64 -> lib
├── libexec
└── share
Compiling a simple program using:
$ clang-18 --target=arm-eabi -mcpu=cortex-a7 -mthumb -mfpu=neon -mfloat-abi=hard -Wall --gcc-toolchain=/path/to/bootlin/toolchain -c test.c -o test.o
clang-18: warning: argument unused during compilation: '--gcc-toolchain=/path/to/bootlin/toolchain' [-Wunused-command-line-argument]
test.c:1:10: fatal error: 'stdio.h' file not found
1 | #include <stdio.h>
| ^~~~~~~~~
1 error generated.
clang-18 complains --gcc-toolchain is unused. Changing to use --gcc-install-dir does not make it better. The same unused argument warning was thrown. By looking at the documentation, the doc does not mention how exactly these 2 options affects what behavior of clang.
Changing to use--sysroot failed too. The sysroot value was obtained from the result of /path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-gcc --print-sysroot.
Then I found a workaround to help locate the header file by creating a symbolic link include to point to usr/include in the sysroot directory when using
/path/to/bootlin/toolchain/arm-buildroot-linux-gnueabihf/sysroot$ ln -s usr/include include
Then I removed -c to see if I can compile and link the program correctly, I got:
$ clang-18 --target=arm-eabi -mcpu=cortex-a7 -mthumb -mfpu=neon -mfloat-abi=hard -Wall --sysroot=/path/to/bootlin/toolchain/arm-buildroot-linux-gnueabihf/sysroot test.c -o test.o
ld.lld: error: unable to find library -lc
ld.lld: error: unable to find library -lm
ld.lld: error: unable to find library -lgcc
clang-18: error: ld.lld command failed with exit code 1 (use -v to see invocation)
I realized the new clang compiler driver defaults to use ld.lld, so I added -fuse-ld=/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld in the command line, which made clang to successfully locate libc and libm libraries, but not libgcc.
$ clang-18 --target=arm-eabi -mcpu=cortex-a7 -mthumb -mfpu=neon -mfloat-abi=hard -Wall --sysroot=/path/to/bootlin/toolchain/arm-buildroot-linux-gnueabihf/sysroot -fuse-ld=/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld test.c -o test.o
/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld: warning: library search path "/usr/lib/llvm-18/lib/clang/18/lib/baremetal" is unsafe for cross-compilation
/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld: cannot find -lgcc: No such file or directory
clang-18: error: ld.lld command failed with exit code 1 (use -v to see invocation)
The libgcc.a is located in the bootlin gcc toolchain directorty here:
$ /path/to/bootlin/toolchain$ find . -name libgcc.a
./lib/gcc/arm-buildroot-linux-gnueabihf/12.3.0/libgcc.a
So I added -L/path/to/bootlin/toolchain/lib/gcc/arm-buildroot-linux-gnueabihf/12.3.0 in the command line, then libgcc was located by clang, but the final linking procedure still complained:
$ clang-18 --target=arm-eabi -mcpu=cortex-a7 -mthumb -mfpu=neon -mfloat-abi=hard -Wall --sysroot=/path/to/bootlin/toolchain/arm-buildroot-linux-gnueabihf/sysroot -fuse-ld=/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld test.c -L/path/to/bootlin/toolchain/lib/gcc/arm-buildroot-linux-gnueabihf/12.3.0 -o test.o
/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld: warning: library search path "/usr/lib/llvm-18/lib/clang/18/lib/baremetal" is unsafe for cross-compilation
/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld: warning: cannot find entry symbol _start; defaulting to 00010140
/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld: /path/to/bootlin/toolchain/bin/../arm-buildroot-linux-gnueabihf/sysroot/usr/lib/libc.a(printf.o):(.ARM.exidx+0x0): undefined reference to `__aeabi_unwind_cpp_pr1'
/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld: /path/to/bootlin/toolchain/bin/../arm-buildroot-linux-gnueabihf/sysroot/usr/lib/libc.a(getsysstats.o):(.ARM.exidx+0x0): undefined reference to `__aeabi_unwind_cpp_pr0'
/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld: /path/to/bootlin/toolchain/bin/../arm-buildroot-linux-gnueabihf/sysroot/usr/lib/libc.a(getsysstats.o):(.ARM.exidx+0x8): undefined reference to `__aeabi_unwind_cpp_pr1'
/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld: /path/to/bootlin/toolchain/bin/../arm-buildroot-linux-gnueabihf/sysroot/usr/lib/libc.a(madvise.o):(.ARM.exidx+0x0): undefined reference to `__aeabi_unwind_cpp_pr0'
...
I gave up here. So what is the expected way of using clang to cross-compile a program and using libraries from a gcc cross-toolchain?