Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[llvm] Support IFuncs on Darwin platforms #73686

Merged

Conversation

jroelofs
Copy link
Contributor

... by lowering them as lazy resolve-on-first-use symbol resolvers. Note that
this is subtly different timing than on ELF platforms, where ifunc resolution
happens at load time.

Since ld64 and ld-prime don't support all the cases we need for these, there is
a manual fallback lowering.

Created using spr 1.3.4
@llvmbot
Copy link
Collaborator

llvmbot commented Nov 28, 2023

@llvm/pr-subscribers-llvm-globalisel
@llvm/pr-subscribers-llvm-ir

@llvm/pr-subscribers-backend-x86

Author: Jon Roelofs (jroelofs)

Changes

... by lowering them as lazy resolve-on-first-use symbol resolvers. Note that
this is subtly different timing than on ELF platforms, where ifunc resolution
happens at load time.

Since ld64 and ld-prime don't support all the cases we need for these, there is
a manual fallback lowering.


Patch is 29.31 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/73686.diff

12 Files Affected:

  • (modified) llvm/docs/LangRef.rst (+4-3)
  • (modified) llvm/include/llvm/CodeGen/AsmPrinter.h (+5-1)
  • (modified) llvm/lib/CodeGen/GlobalISel/CallLowering.cpp (+6-1)
  • (modified) llvm/lib/IR/Verifier.cpp (+5-7)
  • (modified) llvm/lib/Target/AArch64/AArch64AsmPrinter.cpp (+308)
  • (modified) llvm/lib/Target/X86/X86AsmPrinter.cpp (+28)
  • (modified) llvm/lib/Target/X86/X86AsmPrinter.h (+1)
  • (added) llvm/test/CodeGen/AArch64/GlobalISel/call-lowering-ifunc.ll (+37)
  • (modified) llvm/test/CodeGen/AArch64/addrsig-macho.ll (+2-2)
  • (added) llvm/test/CodeGen/AArch64/ifunc-asm.ll (+82)
  • (modified) llvm/test/CodeGen/X86/ifunc-asm.ll (+19-9)
  • (added) llvm/test/Verifier/ifunc-macho.ll (+42)
diff --git a/llvm/docs/LangRef.rst b/llvm/docs/LangRef.rst
index e448c5ed5c5d947..cb222e979db29d4 100644
--- a/llvm/docs/LangRef.rst
+++ b/llvm/docs/LangRef.rst
@@ -934,10 +934,11 @@ IFuncs
 -------
 
 IFuncs, like as aliases, don't create any new data or func. They are just a new
-symbol that dynamic linker resolves at runtime by calling a resolver function.
+symbol that is resolved at runtime by calling a resolver function.
 
-IFuncs have a name and a resolver that is a function called by dynamic linker
-that returns address of another function associated with the name.
+On ELF platforms, IFuncs are resolved by the dynamic linker at load time. On
+MachO platforms, they are lowered in terms of ``.symbol_resolver``s, which
+lazily resolve the callee the first time they are called.
 
 IFunc may have an optional :ref:`linkage type <linkage>` and an optional
 :ref:`visibility style <visibility>`.
diff --git a/llvm/include/llvm/CodeGen/AsmPrinter.h b/llvm/include/llvm/CodeGen/AsmPrinter.h
index 2731ef452c79cbb..48fa6c478464c73 100644
--- a/llvm/include/llvm/CodeGen/AsmPrinter.h
+++ b/llvm/include/llvm/CodeGen/AsmPrinter.h
@@ -882,7 +882,11 @@ class AsmPrinter : public MachineFunctionPass {
 
   GCMetadataPrinter *getOrCreateGCPrinter(GCStrategy &S);
   void emitGlobalAlias(Module &M, const GlobalAlias &GA);
-  void emitGlobalIFunc(Module &M, const GlobalIFunc &GI);
+
+protected:
+  virtual void emitGlobalIFunc(Module &M, const GlobalIFunc &GI);
+
+private:
 
   /// This method decides whether the specified basic block requires a label.
   bool shouldEmitLabelForBasicBlock(const MachineBasicBlock &MBB) const;
diff --git a/llvm/lib/CodeGen/GlobalISel/CallLowering.cpp b/llvm/lib/CodeGen/GlobalISel/CallLowering.cpp
index 2527b1431289677..e0080b145d4f995 100644
--- a/llvm/lib/CodeGen/GlobalISel/CallLowering.cpp
+++ b/llvm/lib/CodeGen/GlobalISel/CallLowering.cpp
@@ -144,7 +144,12 @@ bool CallLowering::lowerCall(MachineIRBuilder &MIRBuilder, const CallBase &CB,
   // Try looking through a bitcast from one function type to another.
   // Commonly happens with calls to objc_msgSend().
   const Value *CalleeV = CB.getCalledOperand()->stripPointerCasts();
-  if (const Function *F = dyn_cast<Function>(CalleeV))
+  if (const GlobalIFunc *IF = dyn_cast<GlobalIFunc>(CalleeV);
+      IF && MF.getTarget().getTargetTriple().isOSBinFormatMachO()) {
+    // ld64 requires that .symbol_resolvers to be called via a stub, so these
+    // must always be a diret call.
+    Info.Callee = MachineOperand::CreateGA(IF, 0);
+  } else if (const Function *F = dyn_cast<Function>(CalleeV))
     Info.Callee = MachineOperand::CreateGA(F, 0);
   else
     Info.Callee = MachineOperand::CreateReg(GetCalleeReg(), false);
diff --git a/llvm/lib/IR/Verifier.cpp b/llvm/lib/IR/Verifier.cpp
index 5560c037aa3ee6b..94e76a43bf38d6d 100644
--- a/llvm/lib/IR/Verifier.cpp
+++ b/llvm/lib/IR/Verifier.cpp
@@ -959,6 +959,7 @@ void Verifier::visitGlobalIFunc(const GlobalIFunc &GI) {
       GlobalIFunc::getResolverFunctionType(GI.getValueType());
   Check(ResolverTy == ResolverFuncTy->getPointerTo(GI.getAddressSpace()),
         "IFunc resolver has incorrect type", &GI);
+
 }
 
 void Verifier::visitNamedMDNode(const NamedMDNode &NMD) {
@@ -2239,13 +2240,10 @@ void Verifier::verifyFunctionAttrs(FunctionType *FT, AttributeList Attrs,
   }
 
   // Check EVEX512 feature.
-  if (MaxParameterWidth >= 512 && Attrs.hasFnAttr("target-features")) {
-    Triple T(M.getTargetTriple());
-    if (T.isX86()) {
-      StringRef TF = Attrs.getFnAttr("target-features").getValueAsString();
-      Check(!TF.contains("+avx512f") || !TF.contains("-evex512"),
-            "512-bit vector arguments require 'evex512' for AVX512", V);
-    }
+  if (MaxParameterWidth >= 512 && Attrs.hasFnAttr("target-features") && TT.isX86()) {
+    StringRef TF = Attrs.getFnAttr("target-features").getValueAsString();
+    Check(!TF.contains("+avx512f") || !TF.contains("-evex512"),
+          "512-bit vector arguments require 'evex512' for AVX512", V);
   }
 
   checkUnsignedBaseTenFuncAttr(Attrs, "patchable-function-prefix", V);
diff --git a/llvm/lib/Target/AArch64/AArch64AsmPrinter.cpp b/llvm/lib/Target/AArch64/AArch64AsmPrinter.cpp
index f4d3a85f34c88da..2dab8e126c9abd0 100644
--- a/llvm/lib/Target/AArch64/AArch64AsmPrinter.cpp
+++ b/llvm/lib/Target/AArch64/AArch64AsmPrinter.cpp
@@ -30,6 +30,7 @@
 #include "llvm/ADT/Twine.h"
 #include "llvm/BinaryFormat/COFF.h"
 #include "llvm/BinaryFormat/ELF.h"
+#include "llvm/BinaryFormat/MachO.h"
 #include "llvm/CodeGen/AsmPrinter.h"
 #include "llvm/CodeGen/FaultMaps.h"
 #include "llvm/CodeGen/MachineBasicBlock.h"
@@ -47,10 +48,12 @@
 #include "llvm/MC/MCInst.h"
 #include "llvm/MC/MCInstBuilder.h"
 #include "llvm/MC/MCSectionELF.h"
+#include "llvm/MC/MCSectionMachO.h"
 #include "llvm/MC/MCStreamer.h"
 #include "llvm/MC/MCSymbol.h"
 #include "llvm/MC/TargetRegistry.h"
 #include "llvm/Support/Casting.h"
+#include "llvm/Support/CommandLine.h"
 #include "llvm/Support/ErrorHandling.h"
 #include "llvm/Support/raw_ostream.h"
 #include "llvm/Target/TargetMachine.h"
@@ -68,6 +71,22 @@ using namespace llvm;
 
 namespace {
 
+enum class IFuncLowering { SymbolResolverIfSupported, SymbolResolverAlways, SymbolResolverNever };
+
+static cl::opt<IFuncLowering> PreferredIFuncLowering(
+    "arm64-darwin-ifunc-symbol_resolver", cl::init(IFuncLowering::SymbolResolverNever),
+    cl::desc("Pick the lowering for ifuncs on darwin platforms"), cl::Hidden,
+    cl::values(
+        clEnumValN(
+            IFuncLowering::SymbolResolverIfSupported, "if_supported",
+            "Use .symbol_resolver's when known to be supported by the linker."),
+        clEnumValN(IFuncLowering::SymbolResolverAlways, "always",
+                   "Always use .symbol_resolvers. NOTE: this might not be "
+                   "supported by the linker in all cases."),
+        clEnumValN(IFuncLowering::SymbolResolverNever, "never",
+                   "Use a manual lowering, doing what the linker would have "
+                   "done, but in the compiler.")));
+
 class AArch64AsmPrinter : public AsmPrinter {
   AArch64MCInstLower MCInstLowering;
   FaultMaps FM;
@@ -198,6 +217,11 @@ class AArch64AsmPrinter : public AsmPrinter {
   bool shouldEmitWeakSwiftAsyncExtendedFramePointerFlags() const override {
     return ShouldEmitWeakSwiftAsyncExtendedFramePointerFlags;
   }
+
+  void emitGlobalIFunc(Module &M, const GlobalIFunc &GI) override;
+
+  void emitLinkerSymbolResolver(Module &M, const GlobalIFunc &GI);
+  void emitManualSymbolResolver(Module &M, const GlobalIFunc &GI);
 };
 
 } // end anonymous namespace
@@ -1809,6 +1833,290 @@ void AArch64AsmPrinter::emitInstruction(const MachineInstr *MI) {
   EmitToStreamer(*OutStreamer, TmpInst);
 }
 
+void AArch64AsmPrinter::emitLinkerSymbolResolver(Module &M,
+                                                 const GlobalIFunc &GI) {
+  OutStreamer->switchSection(OutContext.getObjectFileInfo()->getTextSection());
+
+  MCSymbol *Name = getSymbol(&GI);
+
+  // NOTE: non-global .symbol_resolvers are not yet supported by Darwin linkers
+
+  if (GI.hasExternalLinkage() || !MAI->getWeakRefDirective())
+    OutStreamer->emitSymbolAttribute(Name, MCSA_Global);
+  else if (GI.hasWeakLinkage() || GI.hasLinkOnceLinkage())
+    OutStreamer->emitSymbolAttribute(Name, MCSA_WeakReference);
+  else
+    assert(GI.hasLocalLinkage() && "Invalid ifunc linkage");
+
+  OutStreamer->emitCodeAlignment(Align(4), STI);
+  OutStreamer->emitLabel(Name);
+  OutStreamer->emitSymbolAttribute(Name, MCSA_SymbolResolver);
+  emitVisibility(Name, GI.getVisibility());
+
+  // ld-prime does not seem to support aliases of symbol resolvers, so we have to
+  // tail call the resolver manually.
+  OutStreamer->emitInstruction(
+      MCInstBuilder(AArch64::B)
+          .addOperand(MCOperand::createExpr(lowerConstant(GI.getResolver()))),
+      *STI);
+}
+
+/// \brief Emit a manually-constructed .symbol_resolver that implements the
+/// symbol resolution duties of the IFunc.
+///
+/// Normally, this would be handled by linker magic, but unfortunately there are
+/// a few limitations in ld64 and ld-prime's implementation of .symbol_resolver
+/// that mean we can't always use them:
+///
+///    *  resolvers cannot be the target of an alias
+///    *  resolvers cannot have private linkage
+///    *  resolvers cannot have linkonce linkage
+///    *  resolvers cannot appear in executables
+///    *  resolvers cannot appear in bundles
+///
+/// This works around that by emitting a close approximation of what the linker
+/// would have done.
+void AArch64AsmPrinter::emitManualSymbolResolver(Module &M,
+                                                 const GlobalIFunc &GI) {
+  auto EmitLinkage = [&](MCSymbol *Sym) {
+    if (GI.hasExternalLinkage() || !MAI->getWeakRefDirective())
+      OutStreamer->emitSymbolAttribute(Sym, MCSA_Global);
+    else if (GI.hasWeakLinkage() || GI.hasLinkOnceLinkage())
+      OutStreamer->emitSymbolAttribute(Sym, MCSA_WeakReference);
+    else
+      assert(GI.hasLocalLinkage() && "Invalid ifunc linkage");
+  };
+
+  MCSymbol *LazyPointer = TM.getObjFileLowering()->getContext().getOrCreateSymbol(
+      "_" + GI.getName() + ".lazy_pointer");
+  MCSymbol *StubHelper =
+      TM.getObjFileLowering()->getContext().getOrCreateSymbol(
+          "_" + GI.getName() + ".stub_helper");
+
+  OutStreamer->switchSection(OutContext.getObjectFileInfo()->getDataSection());
+
+  EmitLinkage(LazyPointer);
+  OutStreamer->emitLabel(LazyPointer);
+  emitVisibility(LazyPointer, GI.getVisibility());
+  OutStreamer->emitValue(MCSymbolRefExpr::create(StubHelper, OutContext), 8);
+
+  OutStreamer->switchSection(OutContext.getObjectFileInfo()->getTextSection());
+
+  MCSymbol *Stub = getSymbol(&GI);
+
+  EmitLinkage(Stub);
+  OutStreamer->emitCodeAlignment(Align(4), STI);
+  OutStreamer->emitLabel(Stub);
+  emitVisibility(Stub, GI.getVisibility());
+
+  // adrp	x16, lazy_pointer@GOTPAGE
+  // ldr	x16, [x16, lazy_pointer@GOTPAGEOFF]
+  // ldr	x16, [x16]
+  // br	x16
+
+  {
+    MCInst Adrp;
+    Adrp.setOpcode(AArch64::ADRP);
+    Adrp.addOperand(MCOperand::createReg(AArch64::X16));
+    MCOperand SymPage;
+    MCInstLowering.lowerOperand(
+        MachineOperand::CreateES(LazyPointer->getName().data() + 1,
+                                 AArch64II::MO_GOT | AArch64II::MO_PAGE),
+        SymPage);
+    Adrp.addOperand(SymPage);
+    OutStreamer->emitInstruction(Adrp, *STI);
+  }
+
+  {
+    MCInst Ldr;
+    Ldr.setOpcode(AArch64::LDRXui);
+    Ldr.addOperand(MCOperand::createReg(AArch64::X16));
+    Ldr.addOperand(MCOperand::createReg(AArch64::X16));
+    MCOperand SymPageOff;
+    MCInstLowering.lowerOperand(
+        MachineOperand::CreateES(LazyPointer->getName().data() + 1,
+                                 AArch64II::MO_GOT | AArch64II::MO_PAGEOFF),
+        SymPageOff);
+    Ldr.addOperand(SymPageOff);
+    Ldr.addOperand(MCOperand::createImm(0));
+    OutStreamer->emitInstruction(Ldr, *STI);
+  }
+
+  OutStreamer->emitInstruction(MCInstBuilder(AArch64::LDRXui)
+    .addReg(AArch64::X16)
+    .addReg(AArch64::X16)
+    .addImm(0), *STI);
+
+  OutStreamer->emitInstruction(MCInstBuilder(TM.getTargetTriple().isArm64e()
+                                                 ? AArch64::BRAAZ
+                                                 : AArch64::BR)
+                                   .addReg(AArch64::X16),
+                               *STI);
+
+  EmitLinkage(StubHelper);
+  OutStreamer->emitCodeAlignment(Align(4), STI);
+  OutStreamer->emitLabel(StubHelper);
+  emitVisibility(StubHelper, GI.getVisibility());
+
+  // stp	fp, lr, [sp, #-16]!
+  // mov	fp, sp
+  // stp	x1, x0, [sp, #-16]!
+  // stp	x3, x2, [sp, #-16]!
+  // stp	x5, x4, [sp, #-16]!
+  // stp	x7, x6, [sp, #-16]!
+  // stp	d1, d0, [sp, #-16]!
+  // stp	d3, d2, [sp, #-16]!
+  // stp	d5, d4, [sp, #-16]!
+  // stp	d7, d6, [sp, #-16]!
+  // bl	_resolver
+  // adrp	x16, lazy_pointer@GOTPAGE
+  // ldr	x16, [x16, lazy_pointer@GOTPAGEOFF]
+  // str	x0, [x16]
+  // mov	x16, x0
+  // ldp	d7, d6, [sp], #16
+  // ldp	d5, d4, [sp], #16
+  // ldp	d3, d2, [sp], #16
+  // ldp	d1, d0, [sp], #16
+  // ldp	x7, x6, [sp], #16
+  // ldp	x5, x4, [sp], #16
+  // ldp	x3, x2, [sp], #16
+  // ldp	x1, x0, [sp], #16
+  // ldp	fp, lr, [sp], #16
+  // br	x16
+
+  OutStreamer->emitInstruction(MCInstBuilder(AArch64::STPXpre)
+                                   .addReg(AArch64::SP)
+                                   .addReg(AArch64::FP)
+                                   .addReg(AArch64::LR)
+                                   .addReg(AArch64::SP)
+                                   .addImm(-2),
+                               *STI);
+
+  OutStreamer->emitInstruction(MCInstBuilder(AArch64::ADDXri)
+                                   .addReg(AArch64::FP)
+                                   .addReg(AArch64::SP)
+                                   .addImm(0)
+                                   .addImm(0),
+                               *STI);
+
+  for (int I = 0; I != 4; ++I)
+    OutStreamer->emitInstruction(MCInstBuilder(AArch64::STPXpre)
+                                     .addReg(AArch64::SP)
+                                     .addReg(AArch64::X1 + 2 * I)
+                                     .addReg(AArch64::X0 + 2 * I)
+                                     .addReg(AArch64::SP)
+                                     .addImm(-2),
+                                 *STI);
+
+  for (int I = 0; I != 4; ++I)
+    OutStreamer->emitInstruction(MCInstBuilder(AArch64::STPDpre)
+                                     .addReg(AArch64::SP)
+                                     .addReg(AArch64::D1 + 2 * I)
+                                     .addReg(AArch64::D0 + 2 * I)
+                                     .addReg(AArch64::SP)
+                                     .addImm(-2),
+                                 *STI);
+
+  OutStreamer->emitInstruction(
+      MCInstBuilder(AArch64::BL)
+          .addOperand(MCOperand::createExpr(lowerConstant(GI.getResolver()))),
+      *STI);
+
+  {
+    MCInst Adrp;
+    Adrp.setOpcode(AArch64::ADRP);
+    Adrp.addOperand(MCOperand::createReg(AArch64::X16));
+    MCOperand SymPage;
+    MCInstLowering.lowerOperand(
+        MachineOperand::CreateES(LazyPointer->getName().data() + 1,
+                                 AArch64II::MO_GOT | AArch64II::MO_PAGE),
+        SymPage);
+    Adrp.addOperand(SymPage);
+    OutStreamer->emitInstruction(Adrp, *STI);
+  }
+
+  {
+    MCInst Ldr;
+    Ldr.setOpcode(AArch64::LDRXui);
+    Ldr.addOperand(MCOperand::createReg(AArch64::X16));
+    Ldr.addOperand(MCOperand::createReg(AArch64::X16));
+    MCOperand SymPageOff;
+    MCInstLowering.lowerOperand(
+        MachineOperand::CreateES(LazyPointer->getName().data() + 1,
+                                 AArch64II::MO_GOT | AArch64II::MO_PAGEOFF),
+        SymPageOff);
+    Ldr.addOperand(SymPageOff);
+    Ldr.addOperand(MCOperand::createImm(0));
+    OutStreamer->emitInstruction(Ldr, *STI);
+  }
+
+  OutStreamer->emitInstruction(MCInstBuilder(AArch64::STRXui)
+                                   .addReg(AArch64::X0)
+                                   .addReg(AArch64::X16)
+                                   .addImm(0),
+                               *STI);
+
+  OutStreamer->emitInstruction(MCInstBuilder(AArch64::ADDXri)
+                                   .addReg(AArch64::X16)
+                                   .addReg(AArch64::X0)
+                                   .addImm(0)
+                                   .addImm(0),
+                               *STI);
+
+  for (int I = 3; I != -1; --I)
+    OutStreamer->emitInstruction(MCInstBuilder(AArch64::LDPDpost)
+                                     .addReg(AArch64::SP)
+                                     .addReg(AArch64::D1 + 2 * I)
+                                     .addReg(AArch64::D0 + 2 * I)
+                                     .addReg(AArch64::SP)
+                                     .addImm(2),
+                                 *STI);
+
+  for (int I = 3; I != -1; --I)
+    OutStreamer->emitInstruction(MCInstBuilder(AArch64::LDPXpost)
+                                     .addReg(AArch64::SP)
+                                     .addReg(AArch64::X1 + 2 * I)
+                                     .addReg(AArch64::X0 + 2 * I)
+                                     .addReg(AArch64::SP)
+                                     .addImm(2),
+                                 *STI);
+
+  OutStreamer->emitInstruction(MCInstBuilder(AArch64::LDPXpost)
+                                   .addReg(AArch64::SP)
+                                   .addReg(AArch64::FP)
+                                   .addReg(AArch64::LR)
+                                   .addReg(AArch64::SP)
+                                   .addImm(2),
+                               *STI);
+
+  OutStreamer->emitInstruction(MCInstBuilder(TM.getTargetTriple().isArm64e()
+                                                 ? AArch64::BRAAZ
+                                                 : AArch64::BR)
+                                   .addReg(AArch64::X16),
+                               *STI);
+}
+
+void AArch64AsmPrinter::emitGlobalIFunc(Module &M, const GlobalIFunc &GI) {
+  if (!TM.getTargetTriple().isOSBinFormatMachO())
+    return AsmPrinter::emitGlobalIFunc(M, GI);
+
+  switch (PreferredIFuncLowering) {
+  case IFuncLowering::SymbolResolverAlways:
+    return emitLinkerSymbolResolver(M, GI);
+  case IFuncLowering::SymbolResolverNever:
+    return emitManualSymbolResolver(M, GI);
+  case IFuncLowering::SymbolResolverIfSupported:
+    if (GI.hasExternalLinkage() || !MAI->getWeakRefDirective())
+      return emitLinkerSymbolResolver(M, GI);
+    else if (GI.hasWeakLinkage() || GI.hasLinkOnceLinkage())
+      // NOTE: non-global .symbol_resolvers are not yet supported by Darwin
+      // linkers
+      return emitManualSymbolResolver(M, GI);
+    else
+      assert(GI.hasLocalLinkage() && "Invalid ifunc linkage");
+  }
+}
+
 // Force static initialization.
 extern "C" LLVM_EXTERNAL_VISIBILITY void LLVMInitializeAArch64AsmPrinter() {
   RegisterAsmPrinter<AArch64AsmPrinter> X(getTheAArch64leTarget());
diff --git a/llvm/lib/Target/X86/X86AsmPrinter.cpp b/llvm/lib/Target/X86/X86AsmPrinter.cpp
index 73c7450620966cd..5241aa6e1c0eade 100644
--- a/llvm/lib/Target/X86/X86AsmPrinter.cpp
+++ b/llvm/lib/Target/X86/X86AsmPrinter.cpp
@@ -530,6 +530,34 @@ void X86AsmPrinter::PrintIntelMemReference(const MachineInstr *MI,
   O << ']';
 }
 
+void X86AsmPrinter::emitGlobalIFunc(Module &M, const GlobalIFunc &GI) {
+  if (!TM.getTargetTriple().isOSBinFormatMachO())
+    return AsmPrinter::emitGlobalIFunc(M, GI);
+
+  OutStreamer->switchSection(OutContext.getObjectFileInfo()->getTextSection());
+
+  MCSymbol *Name = getSymbol(&GI);
+
+  if (GI.hasExternalLinkage() || !MAI->getWeakRefDirective())
+    OutStreamer->emitSymbolAttribute(Name, MCSA_Global);
+  else if (GI.hasWeakLinkage() || GI.hasLinkOnceLinkage())
+    OutStreamer->emitSymbolAttribute(Name, MCSA_WeakReference);
+  else
+    assert(GI.hasLocalLinkage() && "Invalid ifunc linkage");
+
+  OutStreamer->emitCodeAlignment(Align(16), Subtarget);
+  OutStreamer->emitLabel(Name);
+  OutStreamer->emitSymbolAttribute(Name, MCSA_SymbolResolver);
+  emitVisibility(Name, GI.getVisibility());
+
+  // ld64 does not seem to support aliases of symbol resolvers, so we have to
+  // tail call the resolver manually.
+  MCInst JMP;
+  JMP.setOpcode(X86::JMP_4);
+  JMP.addOperand(MCOperand::createExpr(lowerConstant(GI.getResolver())));
+  OutStreamer->emitInstruction(JMP, *Subtarget);
+}
+
 static bool printAsmMRegister(const X86AsmPrinter &P, const MachineOperand &MO,
                               char Mode, raw_ostream &O) {
   Register Reg = MO.getReg();
diff --git a/llvm/lib/Target/X86/X86AsmPrinter.h b/llvm/lib/Target/X86/X86AsmPrinter.h
index c81651cf7f2f0e6..47f3b7c00c99ddb 100644
--- a/llvm/lib/Target/X86/X86AsmPrinter.h
+++ b/llvm/lib/Target/X86/X86AsmPrinter.h
@@ -120,6 +120,7 @@ class LLVM_LIBRARY_VISIBILITY X86AsmPrinter : public AsmPrinter {
                          const char *Modifier);
   void PrintIntelMemReference(const MachineInstr *MI, unsigned OpNo,
                               raw_ostream &O, const char *Modifier);
+  void emitGlobalIFunc(Module &M, const GlobalIFunc &GI) override;
 
 public:
   X86AsmPrinter(TargetMachine &TM, std::unique_ptr<MCStreamer> Streamer);
...
[truncated]

Copy link

github-actions bot commented Nov 28, 2023

:white_check_mark: With the latest revision this PR passed the C/C++ code formatter.

StringRef TF = Attrs.getFnAttr("target-features").getValueAsString();
Check(!TF.contains("+avx512f") || !TF.contains("-evex512"),
"512-bit vector arguments require 'evex512' for AVX512", V);
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

drive-by fix. We don't need to create a Triple, since we already have one.

@erichkeane
Copy link
Collaborator

So I'm not a good reviewer here, my knowledge of ifuncs comes exclusively from using them to implement multiversioning.

Created using spr 1.3.4
Copy link
Contributor

@aemerson aemerson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few comments so far, I'll probably need to revisit this later.

llvm/lib/CodeGen/GlobalISel/CallLowering.cpp Outdated Show resolved Hide resolved
Comment on lines 1968 to 1977
// stp fp, lr, [sp, #-16]!
// mov fp, sp
// stp x1, x0, [sp, #-16]!
// stp x3, x2, [sp, #-16]!
// stp x5, x4, [sp, #-16]!
// stp x7, x6, [sp, #-16]!
// stp d1, d0, [sp, #-16]!
// stp d3, d2, [sp, #-16]!
// stp d5, d4, [sp, #-16]!
// stp d7, d6, [sp, #-16]!
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it doesn't matter so much here, but I think it's generally better to adjust SP once and the use immediate offsets vs pre-indexed, since it won't create register dependencies between each store.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. This pretty closely matches what ld64 does, but if we can do better, we should.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On second thought, these are only ever called once. The slight size savings is probably worth more than the tiny perf bump.

llvm/test/CodeGen/AArch64/ifunc-asm.ll Outdated Show resolved Hide resolved
Created using spr 1.3.4
Created using spr 1.3.4
Created using spr 1.3.4
Created using spr 1.3.4
Created using spr 1.3.4
Created using spr 1.3.4
Created using spr 1.3.4
Created using spr 1.3.4
Created using spr 1.3.4
Created using spr 1.3.4
Created using spr 1.3.4
Created using spr 1.3.4
Created using spr 1.3.4
Created using spr 1.3.4
@jroelofs
Copy link
Contributor Author

ping

@jroelofs jroelofs requested review from aemerson and ahmedbougacha and removed request for erichkeane December 12, 2023 19:40
Copy link
Member

@ahmedbougacha ahmedbougacha left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only a couple minor comments, LG otherwise, thanks! The Subtarget weirdness is unfortunate but I can't say I have a better alternative

llvm/include/llvm/CodeGen/AsmPrinter.h Show resolved Hide resolved
llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp Outdated Show resolved Hide resolved
Created using spr 1.3.4
Created using spr 1.3.4
Copy link
Member

@ahmedbougacha ahmedbougacha left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neat, ty!

Created using spr 1.3.4
@jroelofs jroelofs merged commit 640c1d3 into main Dec 14, 2023
4 of 5 checks passed
@jroelofs jroelofs deleted the users/jroelofs/spr/llvm-support-ifuncs-on-darwin-platforms branch December 14, 2023 21:40
jroelofs added a commit to apple/llvm-project that referenced this pull request Jan 10, 2024
... by lowering them as lazy resolve-on-first-use symbol resolvers. Note that this is subtly different timing than on ELF platforms, where ifunc resolution happens at load time.

Since ld64 and ld-prime don't support all the cases we need for these, we lower them manually in the AsmPrinter.
jroelofs added a commit to apple/llvm-project that referenced this pull request Jan 10, 2024
... by lowering them as lazy resolve-on-first-use symbol resolvers. Note that this is subtly different timing than on ELF platforms, where ifunc resolution happens at load time.

Since ld64 and ld-prime don't support all the cases we need for these, we lower them manually in the AsmPrinter.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants