Skip to content

Commit bf95cf4

Browse files
committed
[x86][seses] Introduce SESES pass for LVI
This is an implementation of Speculative Execution Side Effect Suppression which is intended as a last resort mitigation against Load Value Injection, LVI, a newly disclosed speculative execution side channel vulnerability. One pager: https://software.intel.com/security-software-guidance/software-guidance/load-value-injection Deep dive: https://software.intel.com/security-software-guidance/insights/deep-dive-load-value-injection The mitigation consists of a compiler pass that inserts an LFENCE before each memory read instruction, memory write instruction, and the first branch instruction in a group of terminators at the end of a basic block. The goal is to prevent speculative execution, potentially based on misspeculated conditions and/or containing secret data, from leaking that data via side channels embedded in such instructions. This is something of a last-resort mitigation: it is expected to have extreme performance implications and it may not be a complete mitigation due to trying to enumerate side channels. In addition to the full version of the mitigation, this patch implements three flags to turn off part of the mitigation. These flags are disabled by default. The flags are not intended to result in a secure variant of the mitigation. The flags are intended to be used by users who would like to experiment with improving the performance of the mitigation. I ran benchmarks with each of these flags enabled in order to find if there was any room for further optimization of LFENCE placement with respect to LVI. Performance Testing Results When applying this mitigation to BoringSSL, we see the following results. These are a summary/aggregation of the performance changes when this mitigation is applied versus when no mitigation is applied. Fully Mitigated vs Baseline Geometric mean 0.071 (Note: This can be read as the ops/s of the mitigated program was 7.1% of the ops/s of the unmitigated program.) Minimum 0.041 Quartile 1 0.060 Median 0.063 Quartile 3 0.077 Maximum 0.230 Reviewed By: george.burgess.iv Differential Revision: https://reviews.llvm.org/D75939
1 parent 6ed61a2 commit bf95cf4

File tree

7 files changed

+473
-0
lines changed

7 files changed

+473
-0
lines changed

llvm/lib/Target/X86/CMakeLists.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,7 @@ set(sources
6464
X86SelectionDAGInfo.cpp
6565
X86ShuffleDecodeConstantPool.cpp
6666
X86SpeculativeLoadHardening.cpp
67+
X86SpeculativeExecutionSideEffectSuppression.cpp
6768
X86Subtarget.cpp
6869
X86TargetMachine.cpp
6970
X86TargetObjectFile.cpp

llvm/lib/Target/X86/X86.h

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -142,6 +142,7 @@ InstructionSelector *createX86InstructionSelector(const X86TargetMachine &TM,
142142

143143
FunctionPass *createX86LoadValueInjectionRetHardeningPass();
144144
FunctionPass *createX86SpeculativeLoadHardeningPass();
145+
FunctionPass *createX86SpeculativeExecutionSideEffectSuppression();
145146

146147
void initializeEvexToVexInstPassPass(PassRegistry &);
147148
void initializeFixupBWInstPassPass(PassRegistry &);
@@ -162,6 +163,7 @@ void initializeX86LoadValueInjectionRetHardeningPassPass(PassRegistry &);
162163
void initializeX86OptimizeLEAPassPass(PassRegistry &);
163164
void initializeX86PartialReductionPass(PassRegistry &);
164165
void initializeX86SpeculativeLoadHardeningPassPass(PassRegistry &);
166+
void initializeX86SpeculativeExecutionSideEffectSuppressionPass(PassRegistry &);
165167

166168
namespace X86AS {
167169
enum : unsigned {
Lines changed: 157 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,157 @@
1+
//===-- X86SpeculativeExecutionSideEffectSuppression.cpp ------------------===//
2+
//
3+
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
4+
// See https://llvm.org/LICENSE.txt for license information.
5+
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
6+
//
7+
//===----------------------------------------------------------------------===//
8+
/// \file
9+
///
10+
/// This file contains the X86 implementation of the speculative execution side
11+
/// effect suppression mitigation.
12+
///
13+
/// This must be used with the -mlvi-cfi flag in order to mitigate indirect
14+
/// branches and returns.
15+
//===----------------------------------------------------------------------===//
16+
17+
#include "X86.h"
18+
#include "X86InstrInfo.h"
19+
#include "X86Subtarget.h"
20+
#include "llvm/ADT/Statistic.h"
21+
#include "llvm/CodeGen/MachineFunction.h"
22+
#include "llvm/CodeGen/MachineFunctionPass.h"
23+
#include "llvm/CodeGen/MachineInstrBuilder.h"
24+
#include "llvm/Pass.h"
25+
using namespace llvm;
26+
27+
#define DEBUG_TYPE "x86-seses"
28+
29+
STATISTIC(NumLFENCEsInserted, "Number of lfence instructions inserted");
30+
31+
static cl::opt<bool> EnableSpeculativeExecutionSideEffectSuppression(
32+
"x86-seses-enable",
33+
cl::desc("Force enable speculative execution side effect suppresion. "
34+
"(Note: User must pass -mlvi-cfi in order to mitigate indirect "
35+
"branches and returns.)"),
36+
cl::init(false), cl::Hidden);
37+
38+
static cl::opt<bool> OneLFENCEPerBasicBlock(
39+
"x86-seses-one-lfence-per-bb",
40+
cl::desc(
41+
"Omit all lfences other than the first to be placed in a basic block."),
42+
cl::init(false), cl::Hidden);
43+
44+
static cl::opt<bool> OnlyLFENCENonConst(
45+
"x86-seses-only-lfence-non-const",
46+
cl::desc("Only lfence before groups of terminators where at least one "
47+
"branch instruction has an input to the addressing mode that is a "
48+
"register other than %rip."),
49+
cl::init(false), cl::Hidden);
50+
51+
static cl::opt<bool>
52+
OmitBranchLFENCEs("x86-seses-omit-branch-lfences",
53+
cl::desc("Omit all lfences before branch instructions."),
54+
cl::init(false), cl::Hidden);
55+
56+
namespace {
57+
58+
class X86SpeculativeExecutionSideEffectSuppression
59+
: public MachineFunctionPass {
60+
public:
61+
X86SpeculativeExecutionSideEffectSuppression() : MachineFunctionPass(ID) {}
62+
63+
static char ID;
64+
StringRef getPassName() const override {
65+
return "X86 Speculative Execution Side Effect Suppression";
66+
}
67+
68+
bool runOnMachineFunction(MachineFunction &MF) override;
69+
};
70+
} // namespace
71+
72+
char X86SpeculativeExecutionSideEffectSuppression::ID = 0;
73+
74+
// This function returns whether the passed instruction uses a memory addressing
75+
// mode that is constant. We treat all memory addressing modes that read
76+
// from a register that is not %rip as non-constant. Note that the use
77+
// of the EFLAGS register results in an addressing mode being considered
78+
// non-constant, therefore all JCC instructions will return false from this
79+
// function since one of their operands will always be the EFLAGS register.
80+
static bool hasConstantAddressingMode(const MachineInstr &MI) {
81+
for (const MachineOperand &MO : MI.uses())
82+
if (MO.isReg() && X86::RIP != MO.getReg())
83+
return false;
84+
return true;
85+
}
86+
87+
bool X86SpeculativeExecutionSideEffectSuppression::runOnMachineFunction(
88+
MachineFunction &MF) {
89+
if (!EnableSpeculativeExecutionSideEffectSuppression)
90+
return false;
91+
92+
LLVM_DEBUG(dbgs() << "********** " << getPassName() << " : " << MF.getName()
93+
<< " **********\n");
94+
bool Modified = false;
95+
const X86Subtarget &Subtarget = MF.getSubtarget<X86Subtarget>();
96+
const X86InstrInfo *TII = Subtarget.getInstrInfo();
97+
for (MachineBasicBlock &MBB : MF) {
98+
MachineInstr *FirstTerminator = nullptr;
99+
100+
for (auto &MI : MBB) {
101+
// We want to put an LFENCE before any instruction that
102+
// may load or store. This LFENCE is intended to avoid leaking any secret
103+
// data due to a given load or store. This results in closing the cache
104+
// and memory timing side channels. We will treat terminators that load
105+
// or store separately.
106+
if (MI.mayLoadOrStore() && !MI.isTerminator()) {
107+
BuildMI(MBB, MI, DebugLoc(), TII->get(X86::LFENCE));
108+
NumLFENCEsInserted++;
109+
Modified = true;
110+
if (OneLFENCEPerBasicBlock)
111+
break;
112+
}
113+
// The following section will be LFENCEing before groups of terminators
114+
// that include branches. This will close the branch prediction side
115+
// channels since we will prevent code executing after misspeculation as
116+
// a result of the LFENCEs placed with this logic.
117+
118+
// Keep track of the first terminator in a basic block since if we need
119+
// to LFENCE the terminators in this basic block we must add the
120+
// instruction before the first terminator in the basic block (as
121+
// opposed to before the terminator that indicates an LFENCE is
122+
// required). An example of why this is necessary is that the
123+
// X86InstrInfo::analyzeBranch method assumes all terminators are grouped
124+
// together and terminates it's analysis once the first non-termintor
125+
// instruction is found.
126+
if (MI.isTerminator() && FirstTerminator == nullptr)
127+
FirstTerminator = &MI;
128+
129+
// Look for branch instructions that will require an LFENCE to be put
130+
// before this basic block's terminators.
131+
if (!MI.isBranch() || OmitBranchLFENCEs)
132+
// This isn't a branch or we're not putting LFENCEs before branches.
133+
continue;
134+
135+
if (OnlyLFENCENonConst && hasConstantAddressingMode(MI))
136+
// This is a branch, but it only has constant addressing mode and we're
137+
// not adding LFENCEs before such branches.
138+
continue;
139+
140+
// This branch requires adding an LFENCE.
141+
BuildMI(MBB, FirstTerminator, DebugLoc(), TII->get(X86::LFENCE));
142+
NumLFENCEsInserted++;
143+
Modified = true;
144+
break;
145+
}
146+
}
147+
148+
return Modified;
149+
}
150+
151+
FunctionPass *llvm::createX86SpeculativeExecutionSideEffectSuppression() {
152+
return new X86SpeculativeExecutionSideEffectSuppression();
153+
}
154+
155+
INITIALIZE_PASS(X86SpeculativeExecutionSideEffectSuppression, "x86-seses",
156+
"X86 Speculative Execution Side Effect Suppresion", false,
157+
false)

llvm/lib/Target/X86/X86TargetMachine.cpp

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -82,6 +82,7 @@ extern "C" LLVM_EXTERNAL_VISIBILITY void LLVMInitializeX86Target() {
8282
initializeX86AvoidSFBPassPass(PR);
8383
initializeX86AvoidTrailingCallPassPass(PR);
8484
initializeX86SpeculativeLoadHardeningPassPass(PR);
85+
initializeX86SpeculativeExecutionSideEffectSuppressionPass(PR);
8586
initializeX86FlagsCopyLoweringPassPass(PR);
8687
initializeX86CondBrFoldingPassPass(PR);
8788
initializeX86LoadValueInjectionRetHardeningPassPass(PR);
@@ -525,6 +526,16 @@ void X86PassConfig::addPreEmitPass2() {
525526
const Triple &TT = TM->getTargetTriple();
526527
const MCAsmInfo *MAI = TM->getMCAsmInfo();
527528

529+
// The X86 Speculative Execution Pass must run after all control
530+
// flow graph modifying passes. As a result it was listed to run right before
531+
// the X86 Retpoline Thunks pass. The reason it must run after control flow
532+
// graph modifications is that the model of LFENCE in LLVM has to be updated
533+
// (FIXME: https://bugs.llvm.org/show_bug.cgi?id=45167). Currently the
534+
// placement of this pass was hand checked to ensure that the subsequent
535+
// passes don't move the code around the LFENCEs in a way that will hurt the
536+
// correctness of this pass. This placement has been shown to work based on
537+
// hand inspection of the codegen output.
538+
addPass(createX86SpeculativeExecutionSideEffectSuppression());
528539
addPass(createX86IndirectThunksPass());
529540

530541
// Insert extra int3 instructions after trailing call instructions to avoid

llvm/test/CodeGen/X86/O0-pipeline.ll

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,7 @@
7373
; CHECK-NEXT: Contiguously Lay Out Funclets
7474
; CHECK-NEXT: StackMap Liveness Analysis
7575
; CHECK-NEXT: Live DEBUG_VALUE analysis
76+
; CHECK-NEXT: X86 Speculative Execution Side Effect Suppression
7677
; CHECK-NEXT: X86 Indirect Thunks
7778
; CHECK-NEXT: Check CFA info and insert CFI instructions if needed
7879
; CHECK-NEXT: X86 Load Value Injection (LVI) Ret-Hardening

llvm/test/CodeGen/X86/O3-pipeline.ll

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -186,6 +186,7 @@
186186
; CHECK-NEXT: Contiguously Lay Out Funclets
187187
; CHECK-NEXT: StackMap Liveness Analysis
188188
; CHECK-NEXT: Live DEBUG_VALUE analysis
189+
; CHECK-NEXT: X86 Speculative Execution Side Effect Suppression
189190
; CHECK-NEXT: X86 Indirect Thunks
190191
; CHECK-NEXT: Check CFA info and insert CFI instructions if needed
191192
; CHECK-NEXT: X86 Load Value Injection (LVI) Ret-Hardening

0 commit comments

Comments
 (0)