Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[lld] enable fixup chains by default #79894

Merged
merged 1 commit into from Jan 31, 2024
Merged

[lld] enable fixup chains by default #79894

merged 1 commit into from Jan 31, 2024

Conversation

rmaz
Copy link
Contributor

@rmaz rmaz commented Jan 29, 2024

Enable chained fixups in lld when all platform and version criteria are met. This is an attempt at simplifying the logic used in ld 907:
https://github.com/apple-oss-distributions/ld64/blob/main/src/ld/Options.cpp#L5458-L5549

Some changes were made to simplify the logic:

  • only enable chained fixups for macOS from 13.0 to avoid the arch check
  • only enable chained fixups for iphonesimulator from 16.0 to avoid the arch check
  • don't enable chained fixups for not specifically listed platforms
  • don't enable chained fixups for arm64_32

@llvmbot
Copy link
Collaborator

llvmbot commented Jan 29, 2024

@llvm/pr-subscribers-lld

@llvm/pr-subscribers-lld-macho

Author: Richard Howell (rmaz)

Changes

Enable chained fixups in lld when all platform and version criteria are met. This aligns with behavior in ld64:
https://github.com/apple-opensource/ld64/blob/master/src/ld/Options.cpp#L5217-L5222


Full diff: https://github.com/llvm/llvm-project/pull/79894.diff

7 Files Affected:

  • (modified) lld/MachO/Driver.cpp (+1-2)
  • (modified) lld/test/MachO/arm64-objc-stubs-dyn.s (+3-3)
  • (modified) lld/test/MachO/arm64-relocs.s (+1-1)
  • (modified) lld/test/MachO/arm64-stubs.s (+1-1)
  • (modified) lld/test/MachO/dyld-stub-binder.s (+7-7)
  • (modified) lld/test/MachO/icf-safe.ll (+4-4)
  • (modified) lld/test/MachO/objc-selrefs.s (+4-3)
diff --git a/lld/MachO/Driver.cpp b/lld/MachO/Driver.cpp
index 411fbcfcf233eb8..796e9e0b351ca79 100644
--- a/lld/MachO/Driver.cpp
+++ b/lld/MachO/Driver.cpp
@@ -1075,8 +1075,7 @@ static bool shouldEmitChainedFixups(const InputArgList &args) {
     return false;
   }
 
-  // TODO: Enable by default once stable.
-  return isRequested;
+  return true;
 }
 
 void SymbolPatterns::clear() {
diff --git a/lld/test/MachO/arm64-objc-stubs-dyn.s b/lld/test/MachO/arm64-objc-stubs-dyn.s
index 9358fc5b31c2ba5..14dd6dd22b9d6f5 100644
--- a/lld/test/MachO/arm64-objc-stubs-dyn.s
+++ b/lld/test/MachO/arm64-objc-stubs-dyn.s
@@ -41,18 +41,18 @@
 # CHECK-EMPTY:
 
 # STUB: Contents of (__TEXT,__stubs) section
-# STUB-NEXT:  adrp    x16, 8 ; 0x100008000
+# STUB-NEXT:  adrp    x16, 4 ; 0x100004000
 # STUB-NEXT:  ldr     x16, [x16]
 # STUB-NEXT:  br      x16
 
 # SMALL: Contents of (__TEXT,__objc_stubs) section
 # SMALL-NEXT: _objc_msgSend$foo:
 # SMALL-NEXT: adrp    x1, 8 ; 0x100008000
-# SMALL-NEXT: ldr     x1, [x1, #0x18]
+# SMALL-NEXT: ldr     x1, [x1, #0x10]
 # SMALL-NEXT: b
 # SMALL-NEXT: _objc_msgSend$length:
 # SMALL-NEXT: adrp    x1, 8 ; 0x100008000
-# SMALL-NEXT: ldr     x1, [x1, #0x20]
+# SMALL-NEXT: ldr     x1, [x1, #0x18]
 # SMALL-NEXT: b
 
 .section  __TEXT,__objc_methname,cstring_literals
diff --git a/lld/test/MachO/arm64-relocs.s b/lld/test/MachO/arm64-relocs.s
index 4bd0f6b8a0452fe..c090596992065e9 100644
--- a/lld/test/MachO/arm64-relocs.s
+++ b/lld/test/MachO/arm64-relocs.s
@@ -22,7 +22,7 @@
 # CHECK-NEXT:  ret
 
 # CHECK-LABEL: Contents of (__DATA_CONST,__const) section
-# CHECK:       [[#PTR_1]]  {{0*}}[[#BAZ]]     00000000 00000000 00000000
+# CHECK:       [[#PTR_1]]  {{0*}}[[#BAZ]]     00200000 00000000 00000000
 # CHECK:       [[#PTR_2]]  {{0*}}[[#BAZ+123]] 00000000 00000000 00000000
 
 .text
diff --git a/lld/test/MachO/arm64-stubs.s b/lld/test/MachO/arm64-stubs.s
index f714e2008489533..6fd94661d32e275 100644
--- a/lld/test/MachO/arm64-stubs.s
+++ b/lld/test/MachO/arm64-stubs.s
@@ -5,7 +5,7 @@
 # RUN: llvm-mc -filetype=obj -triple=arm64-apple-darwin %t/test.s -o %t/test.o
 # RUN: %lld -arch arm64 -dylib -install_name @executable_path/libfoo.dylib %t/foo.o -o %t/libfoo.dylib
 # RUN: %lld -arch arm64 -dylib -install_name @executable_path/libbar.dylib %t/bar.o -o %t/libbar.dylib
-# RUN: %lld -arch arm64 -lSystem %t/libfoo.dylib %t/libbar.dylib %t/test.o -o %t/test
+# RUN: %lld -arch arm64 -lSystem %t/libfoo.dylib %t/libbar.dylib %t/test.o -o %t/test -no_fixup_chains
 
 # RUN: llvm-objdump --no-print-imm-hex --macho -d --no-show-raw-insn --section="__TEXT,__stubs" --section="__TEXT,__stub_helper" %t/test | FileCheck %s
 
diff --git a/lld/test/MachO/dyld-stub-binder.s b/lld/test/MachO/dyld-stub-binder.s
index 1d2e556607c79c6..170fe8abd89bd44 100644
--- a/lld/test/MachO/dyld-stub-binder.s
+++ b/lld/test/MachO/dyld-stub-binder.s
@@ -14,27 +14,27 @@
 # RUN: %lld -arch arm64 -lSystem -dylib %t/foo.o -o %t/libfoo.dylib
 # RUN: llvm-nm -m %t/libfoo.dylib | FileCheck --check-prefix=STUB %s
 
-
 ## Dylibs that do lazy dynamic calls do need dyld_stub_binder.
 # RUN: not %no-lsystem-lld -arch arm64 -dylib %t/bar.o %t/libfoo.dylib \
-# RUN:     -o %t/libbar.dylib 2>&1 | FileCheck --check-prefix=MISSINGSTUB %s
+# RUN:     -o %t/libbar.dylib -no_fixup_chains 2>&1 | \
+# RUN:     FileCheck --check-prefix=MISSINGSTUB %s
 # RUN: %lld -arch arm64 -lSystem -dylib %t/bar.o  %t/libfoo.dylib \
-# RUN:     -o %t/libbar.dylib
+# RUN:     -o %t/libbar.dylib -no_fixup_chains
 # RUN: llvm-nm -m %t/libbar.dylib | FileCheck --check-prefix=STUB %s
 
 ## As do executables.
 # RUN: not %no-lsystem-lld -arch arm64 %t/libfoo.dylib %t/libbar.dylib %t/test.o \
-# RUN:     -o %t/test 2>&1 | FileCheck --check-prefix=MISSINGSTUB %s
+# RUN:     -o %t/test -no_fixup_chains 2>&1 | FileCheck --check-prefix=MISSINGSTUB %s
 # RUN: %lld -arch arm64 -lSystem %t/libfoo.dylib %t/libbar.dylib %t/test.o \
-# RUN:     -o %t/test
+# RUN:     -o %t/test -no_fixup_chains
 # RUN: llvm-nm -m %t/test | FileCheck --check-prefix=STUB %s
 
 ## Test dynamic lookup of dyld_stub_binder.
 # RUN: %no-lsystem-lld -arch arm64 %t/libfoo.dylib %t/libbar.dylib %t/test.o \
-# RUN:     -o %t/test -undefined dynamic_lookup
+# RUN:     -o %t/test -undefined dynamic_lookup -no_fixup_chains
 # RUN: llvm-nm -m %t/test | FileCheck --check-prefix=DYNSTUB %s
 # RUN: %no-lsystem-lld -arch arm64 %t/libfoo.dylib %t/libbar.dylib %t/test.o \
-# RUN:     -o %t/test -U dyld_stub_binder
+# RUN:     -o %t/test -U dyld_stub_binder -no_fixup_chains
 # RUN: llvm-nm -m %t/test | FileCheck --check-prefix=DYNSTUB %s
 
 # MISSINGSTUB:      error: undefined symbol: dyld_stub_binder
diff --git a/lld/test/MachO/icf-safe.ll b/lld/test/MachO/icf-safe.ll
index 71c6f9f7ddac874..7f342e12e5c2a69 100644
--- a/lld/test/MachO/icf-safe.ll
+++ b/lld/test/MachO/icf-safe.ll
@@ -3,14 +3,14 @@
 ; RUN: rm -rf %t; mkdir %t
 
 ; RUN: llc -filetype=obj %s -O3 -o %t/icf-obj.o -enable-machine-outliner=never -mtriple arm64-apple-macos -addrsig
-; RUN: %lld -arch arm64 -lSystem --icf=safe -dylib -o %t/icf-safe.dylib %t/icf-obj.o
-; RUN: %lld -arch arm64 -lSystem --icf=all  -dylib -o %t/icf-all.dylib  %t/icf-obj.o
+; RUN: %lld -arch arm64 -lSystem --icf=safe -dylib -o %t/icf-safe.dylib %t/icf-obj.o -no_fixup_chains
+; RUN: %lld -arch arm64 -lSystem --icf=all  -dylib -o %t/icf-all.dylib  %t/icf-obj.o -no_fixup_chains
 ; RUN: llvm-objdump %t/icf-safe.dylib -d --macho | FileCheck %s --check-prefix=ICFSAFE
 ; RUN: llvm-objdump %t/icf-all.dylib  -d --macho | FileCheck %s --check-prefix=ICFALL
 
 ; RUN: llvm-as %s -o %t/icf-bitcode.o
-; RUN: %lld -arch arm64 -lSystem --icf=safe -dylib -o %t/icf-safe-bitcode.dylib %t/icf-bitcode.o
-; RUN: %lld -arch arm64 -lSystem --icf=all  -dylib -o %t/icf-all-bitcode.dylib %t/icf-bitcode.o
+; RUN: %lld -arch arm64 -lSystem --icf=safe -dylib -o %t/icf-safe-bitcode.dylib %t/icf-bitcode.o -no_fixup_chains
+; RUN: %lld -arch arm64 -lSystem --icf=all  -dylib -o %t/icf-all-bitcode.dylib %t/icf-bitcode.o -no_fixup_chains
 ; RUN: llvm-objdump %t/icf-safe-bitcode.dylib -d --macho | FileCheck %s --check-prefix=ICFSAFE
 ; RUN: llvm-objdump %t/icf-all-bitcode.dylib  -d --macho | FileCheck %s --check-prefix=ICFALL
 
diff --git a/lld/test/MachO/objc-selrefs.s b/lld/test/MachO/objc-selrefs.s
index 9dff440727ac34b..a906978cc2cf728 100644
--- a/lld/test/MachO/objc-selrefs.s
+++ b/lld/test/MachO/objc-selrefs.s
@@ -6,14 +6,14 @@
 # RUN: llvm-mc -filetype=obj -triple=arm64-apple-darwin %t/implicit-selrefs.s -o %t/implicit-selrefs.o
 
 # RUN: %lld -dylib -arch arm64 -lSystem -o %t/explicit-only-no-icf \
-# RUN:   %t/explicit-selrefs-1.o %t/explicit-selrefs-2.o
+# RUN:   %t/explicit-selrefs-1.o %t/explicit-selrefs-2.o -no_fixup_chains
 # RUN: llvm-otool -vs __DATA __objc_selrefs %t/explicit-only-no-icf | \
 # RUN:   FileCheck %s --check-prefix=EXPLICIT-NO-ICF
 
 ## NOTE: ld64 always dedups the selrefs unconditionally, but we only do it when
 ## ICF is enabled.
 # RUN: %lld -dylib -arch arm64 -lSystem -o %t/explicit-only-with-icf \
-# RUN:   %t/explicit-selrefs-1.o %t/explicit-selrefs-2.o
+# RUN:   %t/explicit-selrefs-1.o %t/explicit-selrefs-2.o -no_fixup_chains
 # RUN: llvm-otool -vs __DATA __objc_selrefs %t/explicit-only-with-icf \
 # RUN:   | FileCheck %s --check-prefix=EXPLICIT-WITH-ICF
 
@@ -26,7 +26,8 @@
 
 ## We don't yet support dedup'ing implicitly-defined selrefs.
 # RUN: %lld -dylib -arch arm64 -lSystem --icf=all -o %t/explicit-and-implicit \
-# RUN:   %t/explicit-selrefs-1.o %t/explicit-selrefs-2.o %t/implicit-selrefs.o
+# RUN:   %t/explicit-selrefs-1.o %t/explicit-selrefs-2.o %t/implicit-selrefs.o \
+# RUN:   -no_fixup_chains
 # RUN: llvm-otool -vs __DATA __objc_selrefs %t/explicit-and-implicit \
 # RUN:   | FileCheck %s --check-prefix=EXPLICIT-AND-IMPLICIT
 

@thevinster
Copy link
Contributor

@BertalanD I believe you've done extensive testing when implementing this. Do you know if this is stable?

Copy link
Contributor

@alx32 alx32 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@BertalanD
Copy link
Member

LGTM!

Yes, this feature should be ready to go.

IIRC, the Chromium folks are already using it in production for their iOS builds, but @nico should be able to confirm that.

Copy link

github-actions bot commented Jan 30, 2024

✅ With the latest revision this PR passed the C/C++ code formatter.

@rmaz rmaz requested a review from drodriguez January 30, 2024 23:06
@rmaz rmaz force-pushed the cfixupsdefault branch 2 times, most recently from f6748d2 to d22c878 Compare January 31, 2024 18:02
This commit enables chained fixups by default for supported platforms.
This matches the logic used in ld64 907.
@rmaz rmaz merged commit 775c285 into llvm:main Jan 31, 2024
4 checks passed
@rmaz rmaz deleted the cfixupsdefault branch January 31, 2024 18:30
@zmodem
Copy link
Collaborator

zmodem commented Feb 14, 2024

We're hitting linker crashes/asserts after this. See https://issues.chromium.org/u/1/issues/325133695#comment8 for a reproducer.

Do you think this can be fixed quickly, or could we revert it in the meantime?

@alx32
Copy link
Contributor

alx32 commented Feb 14, 2024

Do you think this can be fixed quickly, or could we revert it in the meantime?

While we look at this - passing "-no_fixup_chains" to the linker will make the crash go away. Would that work as a temp work-around ?

@BertalanD
Copy link
Member

BertalanD commented Feb 14, 2024

Original author of our fixup-chains support here. I had a quick (sorry, school et al.) look at it.

It looks like we are trying to find the address of __RNvCsiLjxBhyzEAX_4curl9INIT_CTOR (curl[da8bc63371d5233d]::INIT_CTOR). This is actually a symbol defined in the __mod_init_func section. The problem is that this section is not emitted when chained fixups are used, but instead, __init_offsets is synthetized from it (389e0a8). So, the data this symbol points to simply will not exist in the final binary.

Now, obviously people shouldn't be defining exported symbols inside __mod_init_func (which is just an array of function pointers to static initializers) willy-nilly. I assume the value does not matter, it's just that they want it to be a root for the section GC (which is an earlier pass than where we're crashing), we could get away with simply omitting this symbol from the exports trie and giving it a garbage value.

The relevant "offending" code is here:

https://github.com/alexcrichton/curl-rust/blob/dbf8d87ecb93c0f3069742e351d35de120189760/src/lib.rs#L123-L151

@BertalanD
Copy link
Member

cc @nico

@zmodem
Copy link
Collaborator

zmodem commented Feb 14, 2024

While we look at this - passing "-no_fixup_chains" to the linker will make the crash go away. Would that work as a temp work-around ?

Yes, that should work. Thanks!

@zmodem
Copy link
Collaborator

zmodem commented Feb 16, 2024

We've hit another issue with fixup chains. In this case, lld doesn't crash but appears to create a corrupt binary. See https://crbug.com/325410295 for a reproducer.

To prevent others from hitting the same problem, I'll go ahead and revert this now.

zmodem added a commit that referenced this pull request Feb 16, 2024
This caused links to fail with:

  lld/MachO/Symbols.cpp:97:
  virtual uint64_t lld::macho::Defined::getVA() const:
  Assertion `target->usesThunks()' failed.

or crash when asserts are disabled. See comment on
#79894

> Enable chained fixups in lld when all platform and version criteria are
> met. This is an attempt at simplifying the logic used in ld 907:
>
> https://github.com/apple-oss-distributions/ld64/blob/93d74eafc37c0558b4ffb88a8bc15c17bed44a20/src/ld/Options.cpp#L5458-L5549
>
> Some changes were made to simplify the logic:
> - only enable chained fixups for macOS from 13.0 to avoid the arch check
> - only enable chained fixups for iphonesimulator from 16.0 to avoid the
> arch check
> - don't enable chained fixups for not specifically listed platforms
> - don't enable chained fixups for arm64_32

This reverts commit 775c285.
@drodriguez
Copy link
Contributor

@zmodem: it seems, from https://issues.chromium.org/issues/325410295, that the issue was with strip from Apple cctools, and seems that also happens with ld, while llvm-strip seems to work fine. Do you think it will be fine to re-revert since the problem was not with the implementation of LLD or any other tool in LLVM?

@zmodem
Copy link
Collaborator

zmodem commented Feb 21, 2024

@zmodem: it seems, from https://issues.chromium.org/issues/325410295, that the issue was with strip from Apple cctools, and seems that also happens with ld, while llvm-strip seems to work fine.

No, the issue did not reproduce with ld. Linking with ld and then running strip worked fine.

Even if the root cause is that strip does something wrong, I think lld needs to be compatible with that (or wait for strip to change). It does not seem reasonable for lld to emit binaries which break when stripped by the standard tool.

There is also the separate issue of crashing when linking rust code (https://crbug.com/325133695).

@drodriguez
Copy link
Contributor

Sorry, misread one phrase of the other bug, and I thought ld was also affected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants