[X86] Don't emit ENDBR for asm goto branch targets

[samitolvanen]

Similarly to #141562, which disabled BTI generation for ARM asm goto branch targets, drop unnecessary ENDBRs from IsInlineAsmBrIndirectTarget machine basic blocks.

[llvmbot]

@llvm/pr-subscribers-backend-x86

Author: Sami Tolvanen (samitolvanen)

Changes

Similarly to #141562, which disabled BTI generation for ARM asm goto branch targets, drop unnecessary ENDBRs from IsInlineAsmBrIndirectTarget machine basic blocks.

---

Full diff: https://github.com/llvm/llvm-project/pull/143439.diff

2 Files Affected:

• (modified) llvm/lib/Target/X86/X86IndirectBranchTracking.cpp (+1-1)
• (added) llvm/test/CodeGen/X86/callbr-asm-endbr.ll (+35)

diff --git a/llvm/lib/Target/X86/X86IndirectBranchTracking.cpp b/llvm/lib/Target/X86/X86IndirectBranchTracking.cpp
index 7740a174af4f3..52be14228e555 100644
--- a/llvm/lib/Target/X86/X86IndirectBranchTracking.cpp
+++ b/llvm/lib/Target/X86/X86IndirectBranchTracking.cpp
@@ -147,7 +147,7 @@ bool X86IndirectBranchTrackingPass::runOnMachineFunction(MachineFunction &MF) {
   for (auto &MBB : MF) {
     // Find all basic blocks that their address was taken (for example
     // in the case of indirect jump) and add ENDBR instruction.
-    if (MBB.hasAddressTaken())
+    if (MBB.isMachineBlockAddressTaken() || MBB.isIRBlockAddressTaken())
       Changed |= addENDBR(MBB, MBB.begin());
 
     for (MachineBasicBlock::iterator I = MBB.begin(); I != MBB.end(); ++I) {
diff --git a/llvm/test/CodeGen/X86/callbr-asm-endbr.ll b/llvm/test/CodeGen/X86/callbr-asm-endbr.ll
new file mode 100644
index 0000000000000..748d7717fdc0c
--- /dev/null
+++ b/llvm/test/CodeGen/X86/callbr-asm-endbr.ll
@@ -0,0 +1,35 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
+; RUN: llc < %s -mtriple=x86_64-unknown-unknown | FileCheck %s
+
+define i32 @test1(i32 %a) {
+; CHECK-LABEL: test1:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    endbr64
+; CHECK-NEXT:    addl $4, %edi
+; CHECK-NEXT:    #APP
+; CHECK-NEXT:    xorl %edi, %edi
+; CHECK-NEXT:    jmp .LBB0_2
+; CHECK-NEXT:    #NO_APP
+; CHECK-NEXT:  # %bb.1: # %normal
+; CHECK-NEXT:    xorl %eax, %eax
+; CHECK-NEXT:    retq
+; CHECK-NEXT:  .LBB0_2: # Inline asm indirect target
+; CHECK-NEXT:    # %fail
+; CHECK-NEXT:    # Label of block must be emitted
+; CHECK-NEXT:    movl $1, %eax
+; CHECK-NEXT:    retq
+entry:
+  %0 = add i32 %a, 4
+  callbr void asm "xorl $0, $0; jmp ${1:l}", "r,!i,~{dirflag},~{fpsr},~{flags}"(i32 %0) to label %normal [label %fail]
+
+normal:
+  ret i32 0
+
+fail:
+  ret i32 1
+}
+
+!llvm.module.flags = !{!0}
+
+!0 = !{i32 8, !"cf-protection-branch", i32 1}
+

[smithp35]

This looks fine to me, and reflects existing x86 GCC behaviour. I've added some x86 backend maintainers. As an Arm employee I'd best not appove an exclusively x86 change.

[RKSimon]

LGTM