Skip to content

Commit

Permalink
Add straight-line strength reduction to LLVM
Browse files Browse the repository at this point in the history
Summary:
Straight-line strength reduction (SLSR) is implemented in GCC but not yet in
LLVM. It has proven to effectively simplify statements derived from an unrolled
loop, and can potentially benefit many other cases too. For example,

LLVM unrolls

  #pragma unroll
  foo (int i = 0; i < 3; ++i) {
    sum += foo((b + i) * s);
  }

into

  sum += foo(b * s);
  sum += foo((b + 1) * s);
  sum += foo((b + 2) * s);

However, no optimizations yet reduce the internal redundancy of the three
expressions:

  b * s
  (b + 1) * s
  (b + 2) * s

With SLSR, LLVM can optimize these three expressions into:

  t1 = b * s
  t2 = t1 + s
  t3 = t2 + s

This commit is only an initial step towards implementing a series of such
optimizations. I will implement more (see TODO in the file commentary) in the
near future. This optimization is enabled for the NVPTX backend for now.
However, I am more than happy to push it to the standard optimization pipeline
after more thorough performance tests.

Test Plan: test/StraightLineStrengthReduce/slsr.ll

Reviewers: eliben, HaoLiu, meheff, hfinkel, jholewinski, atrick

Reviewed By: jholewinski, atrick

Subscribers: karthikthecool, jholewinski, llvm-commits

Differential Revision: http://reviews.llvm.org/D7310

llvm-svn: 228016
  • Loading branch information
Jingyue Wu committed Feb 3, 2015
1 parent e5daf3a commit d7966ff
Show file tree
Hide file tree
Showing 8 changed files with 400 additions and 0 deletions.
1 change: 1 addition & 0 deletions llvm/include/llvm/InitializePasses.h
Expand Up @@ -254,6 +254,7 @@ void initializeSpillPlacementPass(PassRegistry&);
void initializeStackProtectorPass(PassRegistry&);
void initializeStackColoringPass(PassRegistry&);
void initializeStackSlotColoringPass(PassRegistry&);
void initializeStraightLineStrengthReducePass(PassRegistry &);
void initializeStripDeadDebugInfoPass(PassRegistry&);
void initializeStripDeadPrototypesPassPass(PassRegistry&);
void initializeStripDebugDeclarePass(PassRegistry&);
Expand Down
1 change: 1 addition & 0 deletions llvm/include/llvm/LinkAllPasses.h
Expand Up @@ -167,6 +167,7 @@ namespace {
(void) llvm::createScalarizerPass();
(void) llvm::createSeparateConstOffsetFromGEPPass();
(void) llvm::createRewriteSymbolsPass();
(void) llvm::createStraightLineStrengthReducePass();

(void)new llvm::IntervalPartition();
(void)new llvm::ScalarEvolution();
Expand Down
2 changes: 2 additions & 0 deletions llvm/include/llvm/Transforms/Scalar.h
Expand Up @@ -412,6 +412,8 @@ createSeparateConstOffsetFromGEPPass(const TargetMachine *TM = nullptr,
//
BasicBlockPass *createLoadCombinePass();

FunctionPass *createStraightLineStrengthReducePass();

} // End llvm namespace

#endif
1 change: 1 addition & 0 deletions llvm/lib/Target/NVPTX/NVPTXTargetMachine.cpp
Expand Up @@ -158,6 +158,7 @@ void NVPTXPassConfig::addIRPasses() {
addPass(createNVPTXAssignValidGlobalNamesPass());
addPass(createGenericToNVVMPass());
addPass(createNVPTXFavorNonGenericAddrSpacesPass());
addPass(createStraightLineStrengthReducePass());
addPass(createSeparateConstOffsetFromGEPPass());
// The SeparateConstOffsetFromGEP pass creates variadic bases that can be used
// by multiple GEPs. Run GVN or EarlyCSE to really reuse them. GVN generates
Expand Down
1 change: 1 addition & 0 deletions llvm/lib/Transforms/Scalar/CMakeLists.txt
Expand Up @@ -38,6 +38,7 @@ add_llvm_library(LLVMScalarOpts
SeparateConstOffsetFromGEP.cpp
SimplifyCFGPass.cpp
Sink.cpp
StraightLineStrengthReduce.cpp
StructurizeCFG.cpp
TailRecursionElimination.cpp
)
Expand Down
1 change: 1 addition & 0 deletions llvm/lib/Transforms/Scalar/Scalar.cpp
Expand Up @@ -69,6 +69,7 @@ void llvm::initializeScalarOpts(PassRegistry &Registry) {
initializeSinkingPass(Registry);
initializeTailCallElimPass(Registry);
initializeSeparateConstOffsetFromGEPPass(Registry);
initializeStraightLineStrengthReducePass(Registry);
initializeLoadCombinePass(Registry);
}

Expand Down
274 changes: 274 additions & 0 deletions llvm/lib/Transforms/Scalar/StraightLineStrengthReduce.cpp
@@ -0,0 +1,274 @@
//===-- StraightLineStrengthReduce.cpp - ------------------------*- C++ -*-===//
//
// The LLVM Compiler Infrastructure
//
// This file is distributed under the University of Illinois Open Source
// License. See LICENSE.TXT for details.
//
//===----------------------------------------------------------------------===//
//
// This file implements straight-line strength reduction (SLSR). Unlike loop
// strength reduction, this algorithm is designed to reduce arithmetic
// redundancy in straight-line code instead of loops. It has proven to be
// effective in simplifying arithmetic statements derived from an unrolled loop.
// It can also simplify the logic of SeparateConstOffsetFromGEP.
//
// There are many optimizations we can perform in the domain of SLSR. This file
// for now contains only an initial step. Specifically, we look for strength
// reduction candidate in the form of
//
// (B + i) * S
//
// where B and S are integer constants or variables, and i is a constant
// integer. If we found two such candidates
//
// S1: X = (B + i) * S S2: Y = (B + i') * S
//
// and S1 dominates S2, we call S1 a basis of S2, and can replace S2 with
//
// Y = X + (i' - i) * S
//
// where (i' - i) * S is folded to the extent possible. When S2 has multiple
// bases, we pick the one that is closest to S2, or S2's "immediate" basis.
//
// TODO:
//
// - Handle candidates in the form of B + i * S
//
// - Handle candidates in the form of pointer arithmetics. e.g., B[i * S]
//
// - Floating point arithmetics when fast math is enabled.
//
// - SLSR may decrease ILP at the architecture level. Targets that are very
// sensitive to ILP may want to disable it. Having SLSR to consider ILP is
// left as future work.
#include <vector>

#include "llvm/ADT/DenseSet.h"
#include "llvm/IR/Dominators.h"
#include "llvm/IR/IRBuilder.h"
#include "llvm/IR/Module.h"
#include "llvm/IR/PatternMatch.h"
#include "llvm/Support/raw_ostream.h"
#include "llvm/Transforms/Scalar.h"

using namespace llvm;
using namespace PatternMatch;

namespace {

class StraightLineStrengthReduce : public FunctionPass {
public:
// SLSR candidate. Such a candidate must be in the form of
// (Base + Index) * Stride
struct Candidate : public ilist_node<Candidate> {
Candidate(Value *B = nullptr, ConstantInt *Idx = nullptr,
Value *S = nullptr, Instruction *I = nullptr)
: Base(B), Index(Idx), Stride(S), Ins(I), Basis(nullptr) {}
Value *Base;
ConstantInt *Index;
Value *Stride;
// The instruction this candidate corresponds to. It helps us to rewrite a
// candidate with respect to its immediate basis. Note that one instruction
// can corresponds to multiple candidates depending on how you associate the
// expression. For instance,
//
// (a + 1) * (b + 2)
//
// can be treated as
//
// <Base: a, Index: 1, Stride: b + 2>
//
// or
//
// <Base: b, Index: 2, Stride: a + 1>
Instruction *Ins;
// Points to the immediate basis of this candidate, or nullptr if we cannot
// find any basis for this candidate.
Candidate *Basis;
};

static char ID;

StraightLineStrengthReduce() : FunctionPass(ID), DT(nullptr) {
initializeStraightLineStrengthReducePass(*PassRegistry::getPassRegistry());
}

void getAnalysisUsage(AnalysisUsage &AU) const override {
AU.addRequired<DominatorTreeWrapperPass>();
// We do not modify the shape of the CFG.
AU.setPreservesCFG();
}

bool runOnFunction(Function &F) override;

private:
// Returns true if Basis is a basis for C, i.e., Basis dominates C and they
// share the same base and stride.
bool isBasisFor(const Candidate &Basis, const Candidate &C);
// Checks whether I is in a candidate form. If so, adds all the matching forms
// to Candidates, and tries to find the immediate basis for each of them.
void allocateCandidateAndFindBasis(Instruction *I);
// Given that I is in the form of "(B + Idx) * S", adds this form to
// Candidates, and finds its immediate basis.
void allocateCandidateAndFindBasis(Value *B, ConstantInt *Idx, Value *S,
Instruction *I);
// Rewrites candidate C with respect to Basis.
void rewriteCandidateWithBasis(const Candidate &C, const Candidate &Basis);

DominatorTree *DT;
ilist<Candidate> Candidates;
// Temporarily holds all instructions that are unlinked (but not deleted) by
// rewriteCandidateWithBasis. These instructions will be actually removed
// after all rewriting finishes.
DenseSet<Instruction *> UnlinkedInstructions;
};
} // anonymous namespace

char StraightLineStrengthReduce::ID = 0;
INITIALIZE_PASS_BEGIN(StraightLineStrengthReduce, "slsr",
"Straight line strength reduction", false, false)
INITIALIZE_PASS_DEPENDENCY(DominatorTreeWrapperPass)
INITIALIZE_PASS_END(StraightLineStrengthReduce, "slsr",
"Straight line strength reduction", false, false)

FunctionPass *llvm::createStraightLineStrengthReducePass() {
return new StraightLineStrengthReduce();
}

bool StraightLineStrengthReduce::isBasisFor(const Candidate &Basis,
const Candidate &C) {
return (Basis.Ins != C.Ins && // skip the same instruction
// Basis must dominate C in order to rewrite C with respect to Basis.
DT->dominates(Basis.Ins->getParent(), C.Ins->getParent()) &&
// They share the same base and stride.
Basis.Base == C.Base &&
Basis.Stride == C.Stride);
}

// TODO: We currently implement an algorithm whose time complexity is linear to
// the number of existing candidates. However, a better algorithm exists. We
// could depth-first search the dominator tree, and maintain a hash table that
// contains all candidates that dominate the node being traversed. This hash
// table is indexed by the base and the stride of a candidate. Therefore,
// finding the immediate basis of a candidate boils down to one hash-table look
// up.
void StraightLineStrengthReduce::allocateCandidateAndFindBasis(Value *B,
ConstantInt *Idx,
Value *S,
Instruction *I) {
Candidate C(B, Idx, S, I);
// Try to compute the immediate basis of C.
unsigned NumIterations = 0;
// Limit the scan radius to avoid running forever.
static const int MaxNumIterations = 50;
for (auto Basis = Candidates.rbegin();
Basis != Candidates.rend() && NumIterations < MaxNumIterations;
++Basis, ++NumIterations) {
if (isBasisFor(*Basis, C)) {
C.Basis = &(*Basis);
break;
}
}
// Regardless of whether we find a basis for C, we need to push C to the
// candidate list.
Candidates.push_back(C);
}

void StraightLineStrengthReduce::allocateCandidateAndFindBasis(Instruction *I) {
Value *B = nullptr;
ConstantInt *Idx = nullptr;
// "(Base + Index) * Stride" must be a Mul instruction at the first hand.
if (I->getOpcode() == Instruction::Mul) {
if (IntegerType *ITy = dyn_cast<IntegerType>(I->getType())) {
Value *LHS = I->getOperand(0), *RHS = I->getOperand(1);
for (unsigned Swapped = 0; Swapped < 2; ++Swapped) {
// Only handle the canonical operand ordering.
if (match(LHS, m_Add(m_Value(B), m_ConstantInt(Idx)))) {
// If LHS is in the form of "Base + Index", then I is in the form of
// "(Base + Index) * RHS".
allocateCandidateAndFindBasis(B, Idx, RHS, I);
} else {
// Otherwise, at least try the form (LHS + 0) * RHS.
allocateCandidateAndFindBasis(LHS, ConstantInt::get(ITy, 0), RHS, I);
}
// Swap LHS and RHS so that we also cover the cases where LHS is the
// stride.
if (LHS == RHS)
break;
std::swap(LHS, RHS);
}
}
}
}

void StraightLineStrengthReduce::rewriteCandidateWithBasis(
const Candidate &C, const Candidate &Basis) {
// An instruction can correspond to multiple candidates. Therefore, instead of
// simply deleting an instruction when we rewrite it, we mark its parent as
// nullptr (i.e. unlink it) so that we can skip the candidates whose
// instruction is already rewritten.
if (!C.Ins->getParent())
return;
assert(C.Base == Basis.Base && C.Stride == Basis.Stride);
// Basis = (B + i) * S
// C = (B + i') * S
// ==>
// C = Basis + (i' - i) * S
IRBuilder<> Builder(C.Ins);
ConstantInt *IndexOffset = ConstantInt::get(
C.Ins->getContext(), C.Index->getValue() - Basis.Index->getValue());
Value *Reduced;
// TODO: preserve nsw/nuw in some cases.
if (IndexOffset->isOne()) {
// If (i' - i) is 1, fold C into Basis + S.
Reduced = Builder.CreateAdd(Basis.Ins, C.Stride);
} else if (IndexOffset->isMinusOne()) {
// If (i' - i) is -1, fold C into Basis - S.
Reduced = Builder.CreateSub(Basis.Ins, C.Stride);
} else {
Value *Bump = Builder.CreateMul(C.Stride, IndexOffset);
Reduced = Builder.CreateAdd(Basis.Ins, Bump);
}
Reduced->takeName(C.Ins);
C.Ins->replaceAllUsesWith(Reduced);
C.Ins->dropAllReferences();
// Unlink C.Ins so that we can skip other candidates also corresponding to
// C.Ins. The actual deletion is postponed to the end of runOnFunction.
C.Ins->removeFromParent();
UnlinkedInstructions.insert(C.Ins);
}

bool StraightLineStrengthReduce::runOnFunction(Function &F) {
if (skipOptnoneFunction(F))
return false;

DT = &getAnalysis<DominatorTreeWrapperPass>().getDomTree();
// Traverse the dominator tree in the depth-first order. This order makes sure
// all bases of a candidate are in Candidates when we process it.
for (auto node = GraphTraits<DominatorTree *>::nodes_begin(DT);
node != GraphTraits<DominatorTree *>::nodes_end(DT); ++node) {
BasicBlock *B = node->getBlock();
for (auto I = B->begin(); I != B->end(); ++I) {
allocateCandidateAndFindBasis(I);
}
}

// Rewrite candidates in the reverse depth-first order. This order makes sure
// a candidate being rewritten is not a basis for any other candidate.
while (!Candidates.empty()) {
const Candidate &C = Candidates.back();
if (C.Basis != nullptr) {
rewriteCandidateWithBasis(C, *C.Basis);
}
Candidates.pop_back();
}

// Delete all unlink instructions.
for (auto I : UnlinkedInstructions) {
delete I;
}
bool Ret = !UnlinkedInstructions.empty();
UnlinkedInstructions.clear();
return Ret;
}

0 comments on commit d7966ff

Please sign in to comment.