-Clinker-plugin-lto doesn't work without extra manual work #60059
Comments
Centril
added
A-linkage
|
FWIW, you can pass an argument to |
|
Indeed |
|
Nominating for (hopefully brief) discussion in the @rust-lang/compiler meeting -- this is holding up Firefox's efforts to use ThinLTO to eliminate glue code. It would be good to get it fixed or, at minimum, to specify what the correct behavior is and figure out who to ping about it (I'm not that familiar with these options). |
|
FWIW, we have a workaround for Firefox, which is that rust is not involved in linking at all. This is more about the general case than Firefox itself. |
|
It seems that this issue would be mostly resolved by documenting which cases and linker/flag combinations we know to work. We could then see if there are any changes to the CLI that could be made to improve the experience here. |
|
I recently ran into this issue and wanted to mention that if you just want to have lto'd libraries that you are using to link an executable with clang, but can't build the rust code you need because crates that build binaries fail to link due to this error on mac OS, you can sidestep the issue by using the following script on the .rlib files generated by cargo build: https://github.com/kframework/llvm-backend/blob/6ec0694fe63f5c552ea3b64022ff8ba197d1252d/bin/llvm-kompile-rust-lto It will not work if you want to link those libraries with rustc though (although it could be made to work with a small additional effort) |
Should it though? I'm unclear as to what our policy is here? As a user it would make me a bit uneasy if a program like |
|
Just wanted to mention that it's still quite difficult and unobvious to get a linker LTO plugin working with rustc. I had expected to just pass In case someone else is trying to get this working (including my future self), here's what I had to do (on Linux). I couldn't see a way to get rustbuild to install all of the necessary llvm-related tools (namely After building and installing llvm, put its Then I built a stage 1 rust with this in I added the resulting toolchain to rustup, and built a rust project like this: (Note that It took me a few hours to figure all of this out, so I'm quite keen to either make this easier, or at least document better how to do it. Any thoughts? |
|
I've just learned that I should also have used a recent branch on rust's fork of LLVM, but the above instructions are otherwise still valid. |
|
It looks like this was agreed on to be a bug for the MacOS case. We're now bumping into this in Chromium for any target where Rust drives the linking (anything with We want to use ld64.lld, but it doesn't handle the -plugin-opt arguments: I don't see a workaround for this issue above, nor any fixes for it, did I miss it? |
Tools built to be used during the build process don't need PGO and other official build things. This works around a bug on Mac for chrome.exe where Rust gives the linker -plugin-opt flags it doesn't understand: rust-lang/rust#60059 However the bug will be unresolved for other build targets that are link-driven by Rust. R=brucedawson@chromium.org Bug: 1386212 Change-Id: I21824ad484048f0bc283454579f2f603acf3fa99 Cq-Include-Trybots: luci.chromium.try:win-rust-x64-rel,win-rust-x64-dbg,linux-rust-x64-rel,linux-rust-x64-dbg,android-rust-arm64-rel,android-rust-arm64-dbg,android-rust-arm32-rel Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4545355 Commit-Queue: danakj <danakj@chromium.org> Reviewed-by: Bruce Dawson <brucedawson@chromium.org> Auto-Submit: danakj <danakj@chromium.org> Commit-Queue: Bruce Dawson <brucedawson@chromium.org> Cr-Commit-Position: refs/heads/main@{#1146226}
|
Bump - how does on enable LTO in a mixed binary that contains Rust and C++ and is linked by Rust on MacOS? |
|
In case it's of help since the Zulip seems to suggest that things work if you use clang as your linker, we are using It seems that rustc needs to not pass |
When linker plugin LTO is enabled on Apple, rustc passes invalid command line arguments to the linker. They are valid elsewhere but not known by the Apple linker. Fortunately, on Apple we have a python script that is used to invoke the linker. So we drop them from the command line in linker_driver.py for now. Upstream issue: rust-lang/rust#60059 R=hans@chromium.org Bug: 1446796 Change-Id: Idc051457cff194616ac9b0063e3fd255fdd261cc Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4659997 Reviewed-by: Adrian Taylor <adetaylor@chromium.org> Commit-Queue: danakj <danakj@chromium.org> Reviewed-by: Hans Wennborg <hans@chromium.org> Cr-Commit-Position: refs/heads/main@{#1164710}
Tools built to be used during the build process don't need PGO and other official build things. This works around a bug on Mac for chrome.exe where Rust gives the linker -plugin-opt flags it doesn't understand: rust-lang/rust#60059 However the bug will be unresolved for other build targets that are link-driven by Rust. R=brucedawson@chromium.org Bug: 1386212 Change-Id: I21824ad484048f0bc283454579f2f603acf3fa99 Cq-Include-Trybots: luci.chromium.try:win-rust-x64-rel,win-rust-x64-dbg,linux-rust-x64-rel,linux-rust-x64-dbg,android-rust-arm64-rel,android-rust-arm64-dbg,android-rust-arm32-rel Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4545355 Commit-Queue: danakj <danakj@chromium.org> Reviewed-by: Bruce Dawson <brucedawson@chromium.org> Auto-Submit: danakj <danakj@chromium.org> Commit-Queue: Bruce Dawson <brucedawson@chromium.org> Cr-Commit-Position: refs/heads/main@{#1146226} NOKEYCHECK=True GitOrigin-RevId: df7d81e33033a594ff56d5ab8387aa1f13cd1c39
When linker plugin LTO is enabled on Apple, rustc passes invalid command line arguments to the linker. They are valid elsewhere but not known by the Apple linker. Fortunately, on Apple we have a python script that is used to invoke the linker. So we drop them from the command line in linker_driver.py for now. Upstream issue: rust-lang/rust#60059 R=hans@chromium.org Bug: 1446796 Change-Id: Idc051457cff194616ac9b0063e3fd255fdd261cc Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4659997 Reviewed-by: Adrian Taylor <adetaylor@chromium.org> Commit-Queue: danakj <danakj@chromium.org> Reviewed-by: Hans Wennborg <hans@chromium.org> Cr-Commit-Position: refs/heads/main@{#1164710} NOKEYCHECK=True GitOrigin-RevId: dae22dc704135119ae08dabc84deaf39b74d8c6c
|
I have looked into this. Here's what I turned up: Workarounds
Actions to take
Where this problem comes from
ld64's LTO flagsHere are the docs for `ld64` relating to lto, from the manpage:
How cargo handles lto = "thin" in Cargo.toml
Given that the linker is being passed LLVM bitcode objects and is, in fact, doing LTO, you may wonder: how??
|
It doesn't pass it to actually enable linker plugin LTO in that case, but to create an rlib which only contains bitcode rather than the default object code + bitcode hybrid rustc produces by default when LTO is enabled.
That will male rustc do the LTO instead in which case you won't get cross-language LTO at all. You need By the way the LLVM shipped with XCode is almost always too old to load rustc produced bitcode files, so if it seems to work without using the linker plugin bundled with rustc, you aren't actually doing linker plugin LTO. |
I don't mean omitting it for Also, correction: # or your libLTO of choice. -v makes it verbose
( echo "#!/bin/sh";
echo 'exec cc -Wl,-v -Wl,-lto_library,/nix/store/l1lgmvwm5vd08cs127r8s7y9xycsdh0i-llvm-16.0.6-lib/lib/libLTO.dylib "$@"';
) > cc-wrapper.sh
chmod +x cc-wrapper.sh
export RUSTFLAGS="-Clinker=$PWD/cc-wrapper.sh"
export RUSTC_LOG="rustc_codegen_ssa::back::link=info" # to get the linker's -v output on your screen
cargo build --release
# it spits out
@(#)PROGRAM:ld PROJECT:dyld-1015.7
BUILD 18:48:48 Aug 22 2023
configured to support archs: ...
LTO support using: LLVM version 16.0.6 (static support for 29, runtime is 29)
TAPI support using: Apple TAPI version 15.0.0 (tapi-1500.0.12.3)
...
FWIW, I just built rust-analyzer using rustc 1.74 + Cargo.toml Also, rustc does not ship a libLTO.dylib. |
If you didn't pass |
|
Ohhh, you mean rustc will pick up the .rlibs, and optimise them internally with its own llvm. Hence a billion |
…o-wasm, r=petrochenkov Perform LTO optimisations with wasm-ld + -Clinker-plugin-lto Fixes (partially) rust-lang#60059. Technically, `--target wasm32-unknown-unknown -Clinker-plugin-lto` would complete without errors before, but it was not producing optimized code. At least, it may have been but it was probably not the opt-level people intended. Similarly to rust-lang#118377, this could benefit from a warning about using an explicit libLTO path with LLD, which will ignore it and use its internal LLVM. Especially given we always use lld on wasm targets. I left the code open to that possibility rather than making it perfectly neat.
Rollup merge of rust-lang#118378 - cormacrelf:bugfix/linker-plugin-lto-wasm, r=petrochenkov Perform LTO optimisations with wasm-ld + -Clinker-plugin-lto Fixes (partially) rust-lang#60059. Technically, `--target wasm32-unknown-unknown -Clinker-plugin-lto` would complete without errors before, but it was not producing optimized code. At least, it may have been but it was probably not the opt-level people intended. Similarly to rust-lang#118377, this could benefit from a warning about using an explicit libLTO path with LLD, which will ignore it and use its internal LLVM. Especially given we always use lld on wasm targets. I left the code open to that possibility rather than making it perfectly neat.
|
According to LLVM@17 ld64.lld --help: So, for newer lld we can ignore these flags and convert the Temporary workaround: macos-linker.sh #!/bin/sh
# this is a wrapper to adapt ld64 to gnu style arguments
declare -a args=()
for arg in "$@"
do
# options for linker
if [[ $arg == "-Wl,"* ]]; then
IFS=',' read -r -a options <<< "${arg#-Wl,}"
for option in "${options[@]}"
do
if [[ $option == "-plugin="* ]] || [[ $option == "-plugin-opt=mcpu="* ]]; then
# ignore -lto_library and -plugin-opt=mcpu
:
elif [[ $option == "-plugin-opt=O"* ]]; then
# convert -plugin-opt=O* to --lto-CGO*
args[${#args[@]}]="-Wl,--lto-CGO${option#-plugin-opt=O}"
else
# pass through other arguments
args[${#args[@]}]="-Wl,$option"
fi
done
else
# pass through other arguments
args[${#args[@]}]="$arg"
fi
done
# use clang to call ld64
exec ${CC} -v "${args[@]}"
Build with the following ENVs: CC=${HOMEBREW_PREFIX}/opt/llvm/bin/clang \
CXX=${HOMEBREW_PREFIX}/opt/llvm/bin/clang++ \
AR=${HOMEBREW_PREFIX}/opt/llvm/bin/llvm-ar \
CFLAGS="-flto=thin -O3" \
CXXFLAGS="-flto=thin -O3" \
RUSTFLAGS="-Clinker-plugin-lto -Clinker=$PWD/macos-linker.sh -Clink-arg=-fuse-ld=${HOMEBREW_PREFIX}/opt/llvm/bin/ld64.lld" \
cargo build --release |
Yields the following on mac:
On mac, the default linker is ld64. rustc really invokes cc, which invokes ld64 with the flags passed with
-Wland some others depending on the other command line arguments. ld64 does support the LLVM plugin... but doesn't support the -plugin-opt option to pass arguments to it. I know some things can be passed with -Wl,-mllvm,... but I don't know if that includes things that rust is trying to pass here.On Linux, it's funnier:
Because
ccis gcc, this just plain doesn't work, for no obvious reason. At the very least, it seems rust should try to use clang instead of cc in that case.But that also fails:
And here, the reason is essentially the same: the underlying linker doesn't support the
-plugin-optflag... except it does, but not when it's not passed-plugin, which happens when the compiler passes it, which happens when-fltowas on its command line:Using lld works too, because it doesn't need an explicit
-plugin:The text was updated successfully, but these errors were encountered: