Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPIR-V] add convergence region analysis #78456

Merged
merged 4 commits into from
Feb 2, 2024

Conversation

Keenuts
Copy link
Contributor

@Keenuts Keenuts commented Jan 17, 2024

This new analysis returns a hierarchical view of the convergence regions in the given function.
This will allow our passes to query which basic block belongs to which convergence region, and structurize the code in consequence.

Definition

A convergence region is a CFG with:

  • a single entry node.
  • one or multiple exit nodes (different from LLVM's regions).
  • one back-edge
  • zero or more subregions.

Excluding sub-regions nodes, the nodes of a region can only reference a single convergence token. A subregion uses a different convergence token.

Algorithm

This algorithm assumes all loops are in the Simplify form.

Create an initial convergence region for the whole function.

  • the convergence token is the function entry token.
  • the entry is the function entrypoint.
  • Exits are all the basic blocks terminating with a return instruction.

Take the function CFG, and process it in DAG order (ignoring back-edges). If a basic block is a loop header:

  • Create a new region.
    • The parent region is the parent's loop region if any, otherwise, the top level region.
    • The region blocks are all the blocks belonging to this loop.
    • For each loop exit: - visit the rest of the CFG in DAG order (ignore back-edges). - if the region's convergence token is found, add all the blocks dominated by the exit from which the token is reachable to the region.
    • continue the algorithm with the loop headers successors.

This new analysis returns a hierarchical view of the convergence
regions in the given function.
This will allow our passes to query which basic block belongs to
which convergence region, and structurize the code in consequence.

Definition
----------

A convergence region is a CFG with:
 - a single entry node.
 - one or multiple exit nodes (different from LLVM's regions).
 - one back-edge
 - zero or more subregions.

Excluding sub-regions nodes, the nodes of a region can only reference a
single convergence token. A subregion uses a different convergence
token.

Algorithm
---------

This algorithm assumes all loops are in the Simplify form.

Create an initial convergence region for the whole function.
  - the convergence token is the function entry token.
  - the entry is the function entrypoint.
  - Exits are all the basic blocks terminating with a return instruction.

Take the function CFG, and process it in DAG order (ignoring back-edges).
If a basic block is a loop header:
 - Create a new region.
   - The parent region is the parent's loop region if any, otherwise, the
     top level region.
   - The region blocks are all the blocks belonging to this loop.
   - For each loop exit:
        - visit the rest of the CFG in DAG order (ignore back-edges).
        - if the region's convergence token is found, add all the blocks
          dominated by the exit from which the token is reachable to
          the region.
   - continue the algorithm with the loop headers successors.
@llvmbot
Copy link
Collaborator

llvmbot commented Jan 17, 2024

@llvm/pr-subscribers-backend-spir-v

Author: Nathan Gauër (Keenuts)

Changes

This new analysis returns a hierarchical view of the convergence regions in the given function.
This will allow our passes to query which basic block belongs to which convergence region, and structurize the code in consequence.

Definition

A convergence region is a CFG with:

  • a single entry node.
  • one or multiple exit nodes (different from LLVM's regions).
  • one back-edge
  • zero or more subregions.

Excluding sub-regions nodes, the nodes of a region can only reference a single convergence token. A subregion uses a different convergence token.

Algorithm

This algorithm assumes all loops are in the Simplify form.

Create an initial convergence region for the whole function.

  • the convergence token is the function entry token.
  • the entry is the function entrypoint.
  • Exits are all the basic blocks terminating with a return instruction.

Take the function CFG, and process it in DAG order (ignoring back-edges). If a basic block is a loop header:

  • Create a new region.
    • The parent region is the parent's loop region if any, otherwise, the top level region.
    • The region blocks are all the blocks belonging to this loop.
    • For each loop exit: - visit the rest of the CFG in DAG order (ignore back-edges). - if the region's convergence token is found, add all the blocks dominated by the exit from which the token is reachable to the region.
    • continue the algorithm with the loop headers successors.

Patch is 47.82 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/78456.diff

6 Files Affected:

  • (added) llvm/lib/Target/SPIRV/Analysis/CMakeLists.txt (+10)
  • (added) llvm/lib/Target/SPIRV/Analysis/ConvergenceRegionAnalysis.cpp (+310)
  • (added) llvm/lib/Target/SPIRV/Analysis/ConvergenceRegionAnalysis.h (+173)
  • (modified) llvm/lib/Target/SPIRV/CMakeLists.txt (+2)
  • (added) llvm/unittests/Target/SPIRV/CMakeLists.txt (+17)
  • (added) llvm/unittests/Target/SPIRV/ConvergenceRegionAnalysisTests.cpp (+1015)
diff --git a/llvm/lib/Target/SPIRV/Analysis/CMakeLists.txt b/llvm/lib/Target/SPIRV/Analysis/CMakeLists.txt
new file mode 100644
index 00000000000000..374aee3ed1c766
--- /dev/null
+++ b/llvm/lib/Target/SPIRV/Analysis/CMakeLists.txt
@@ -0,0 +1,10 @@
+add_llvm_component_library(LLVMSPIRVAnalysis
+  ConvergenceRegionAnalysis.cpp
+
+  LINK_COMPONENTS
+  Core
+  Support
+
+  ADD_TO_COMPONENT
+  SPIRV
+  )
diff --git a/llvm/lib/Target/SPIRV/Analysis/ConvergenceRegionAnalysis.cpp b/llvm/lib/Target/SPIRV/Analysis/ConvergenceRegionAnalysis.cpp
new file mode 100644
index 00000000000000..5102bc2d4228cc
--- /dev/null
+++ b/llvm/lib/Target/SPIRV/Analysis/ConvergenceRegionAnalysis.cpp
@@ -0,0 +1,310 @@
+//===- ConvergenceRegionAnalysis.h -----------------------------*- C++ -*--===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// The analysis determines the convergence region for each basic block of
+// the module, and provides a tree-like structure describing the region
+// hierarchy.
+//
+//===----------------------------------------------------------------------===//
+
+#include "ConvergenceRegionAnalysis.h"
+#include "llvm/Analysis/LoopInfo.h"
+#include "llvm/IR/Dominators.h"
+#include "llvm/IR/IntrinsicInst.h"
+#include <optional>
+#include <queue>
+
+namespace llvm {
+namespace SPIRV {
+
+namespace {
+
+template <typename BasicBlockType, typename IntrinsicInstType>
+std::optional<IntrinsicInstType *>
+getConvergenceTokenInternal(BasicBlockType *BB) {
+  static_assert(std::is_const_v<IntrinsicInstType> ==
+                    std::is_const_v<BasicBlockType>,
+                "Constness must match between input and output.");
+  static_assert(std::is_same_v<BasicBlock, std::remove_const_t<BasicBlockType>>,
+                "Input must be a basic block.");
+  static_assert(
+      std::is_same_v<IntrinsicInst, std::remove_const_t<IntrinsicInstType>>,
+      "Output type must be an intrinsic instruction.");
+
+  for (auto &I : *BB) {
+    if (auto *II = dyn_cast<IntrinsicInst>(&I)) {
+      if (II->getIntrinsicID() != Intrinsic::experimental_convergence_entry &&
+          II->getIntrinsicID() != Intrinsic::experimental_convergence_loop &&
+          II->getIntrinsicID() != Intrinsic::experimental_convergence_anchor) {
+        continue;
+      }
+
+      if (II->getIntrinsicID() == Intrinsic::experimental_convergence_entry ||
+          II->getIntrinsicID() == Intrinsic::experimental_convergence_loop) {
+        return II;
+      }
+
+      auto Bundle = II->getOperandBundle(LLVMContext::OB_convergencectrl);
+      assert(Bundle->Inputs.size() == 1 &&
+             Bundle->Inputs[0]->getType()->isTokenTy());
+      auto TII = dyn_cast<IntrinsicInst>(Bundle->Inputs[0].get());
+      ;
+      assert(TII != nullptr);
+      return TII;
+    }
+
+    if (auto *CI = dyn_cast<CallInst>(&I)) {
+      auto OB = CI->getOperandBundle(LLVMContext::OB_convergencectrl);
+      if (!OB.has_value())
+        continue;
+      return dyn_cast<IntrinsicInst>(OB.value().Inputs[0]);
+    }
+  }
+
+  return std::nullopt;
+}
+
+} // anonymous namespace
+
+std::optional<IntrinsicInst *> getConvergenceToken(BasicBlock *BB) {
+  return getConvergenceTokenInternal<BasicBlock, IntrinsicInst>(BB);
+}
+
+std::optional<const IntrinsicInst *> getConvergenceToken(const BasicBlock *BB) {
+  return getConvergenceTokenInternal<const BasicBlock, const IntrinsicInst>(BB);
+}
+
+ConvergenceRegion::ConvergenceRegion(DominatorTree &DT, LoopInfo &LI,
+                                     Function &F)
+    : DT(DT), LI(LI), Parent(nullptr) {
+  Entry = &F.getEntryBlock();
+  ConvergenceToken = getConvergenceToken(Entry);
+  for (auto &B : F) {
+    Blocks.insert(&B);
+    if (isa<ReturnInst>(B.getTerminator()))
+      Exits.insert(&B);
+  }
+}
+
+ConvergenceRegion::ConvergenceRegion(
+    DominatorTree &DT, LoopInfo &LI,
+    std::optional<IntrinsicInst *> ConvergenceToken, BasicBlock *Entry,
+    SmallPtrSet<BasicBlock *, 8> &&Blocks, SmallPtrSet<BasicBlock *, 2> &&Exits)
+    : DT(DT), LI(LI), ConvergenceToken(ConvergenceToken), Entry(Entry),
+      Exits(std::move(Exits)), Blocks(std::move(Blocks)) {
+  for (auto *BB : this->Exits)
+    assert(this->Blocks.count(BB) != 0);
+  assert(this->Blocks.count(this->Entry) != 0);
+}
+
+void ConvergenceRegion::releaseMemory() {
+  // Parent memory is owned by the parent.
+  Parent = nullptr;
+  for (auto *Child : Children) {
+    Child->releaseMemory();
+    delete Child;
+  }
+  Children.resize(0);
+}
+
+void ConvergenceRegion::dump(const unsigned IndentSize) const {
+  const std::string Indent(IndentSize, '\t');
+  dbgs() << Indent << this << ": {\n";
+  dbgs() << Indent << "	Parent: " << Parent << "\n";
+
+  if (ConvergenceToken.value_or(nullptr)) {
+    dbgs() << Indent
+           << "	ConvergenceToken: " << ConvergenceToken.value()->getName()
+           << "\n";
+  }
+
+  if (Entry->getName() != "")
+    dbgs() << Indent << "	Entry: " << Entry->getName() << "\n";
+  else
+    dbgs() << Indent << "	Entry: " << Entry << "\n";
+
+  dbgs() << Indent << "	Exits: { ";
+  for (const auto &Exit : Exits) {
+    if (Exit->getName() != "")
+      dbgs() << Exit->getName() << ", ";
+    else
+      dbgs() << Exit << ", ";
+  }
+  dbgs() << "	}\n";
+
+  dbgs() << Indent << "	Blocks: { ";
+  for (const auto &Block : Blocks) {
+    if (Block->getName() != "")
+      dbgs() << Block->getName() << ", ";
+    else
+      dbgs() << Block << ", ";
+  }
+  dbgs() << "	}\n";
+
+  dbgs() << Indent << "	Children: {\n";
+  for (const auto Child : Children)
+    Child->dump(IndentSize + 2);
+  dbgs() << Indent << "	}\n";
+
+  dbgs() << Indent << "}\n";
+}
+
+class ConvergenceRegionAnalyzer {
+
+public:
+  ConvergenceRegionAnalyzer(Function &F, DominatorTree &DT, LoopInfo &LI)
+      : DT(DT), LI(LI), F(F) {}
+
+private:
+  bool isBackEdge(const BasicBlock *From, const BasicBlock *To) const {
+    assert(From != To && "From == To. This is awkward.");
+
+    // We only handle loop in the simplified form. This means:
+    // - a single back-edge, a single latch.
+    // - meaning the back-edge target can only be the loop header.
+    // - meaning the From can only be the loop latch.
+    if (!LI.isLoopHeader(To))
+      return false;
+
+    auto *L = LI.getLoopFor(To);
+    if (L->contains(From) && L->isLoopLatch(From))
+      return true;
+
+    return false;
+  }
+
+  std::unordered_set<BasicBlock *>
+  findPathsToMatch(BasicBlock *From,
+                   std::function<bool(const BasicBlock *)> isMatch) const {
+    std::unordered_set<BasicBlock *> Output;
+
+    if (isMatch(From))
+      Output.insert(From);
+
+    auto *Terminator = From->getTerminator();
+    for (unsigned i = 0; i < Terminator->getNumSuccessors(); ++i) {
+      auto *To = Terminator->getSuccessor(i);
+      if (isBackEdge(From, To))
+        continue;
+
+      auto ChildSet = findPathsToMatch(To, isMatch);
+      if (ChildSet.size() == 0)
+        continue;
+
+      Output.insert(ChildSet.begin(), ChildSet.end());
+      Output.insert(From);
+    }
+
+    return Output;
+  }
+
+  SmallPtrSet<BasicBlock *, 2>
+  findExitNodes(const SmallPtrSetImpl<BasicBlock *> &RegionBlocks) {
+    SmallPtrSet<BasicBlock *, 2> Exits;
+
+    for (auto *B : RegionBlocks) {
+      auto *Terminator = B->getTerminator();
+      for (unsigned i = 0; i < Terminator->getNumSuccessors(); ++i) {
+        auto *Child = Terminator->getSuccessor(i);
+        if (RegionBlocks.count(Child) == 0)
+          Exits.insert(B);
+      }
+    }
+
+    return Exits;
+  }
+
+public:
+  ConvergenceRegionInfo analyze() {
+    ConvergenceRegion *TopLevelRegion = new ConvergenceRegion(DT, LI, F);
+
+    std::unordered_map<Loop *, ConvergenceRegion *> LoopToRegion;
+    std::queue<Loop *> ToProcess;
+    for (auto *L : LI)
+      ToProcess.push(L);
+
+    while (ToProcess.size() != 0) {
+      auto *L = ToProcess.front();
+      ToProcess.pop();
+      for (auto *Child : *L)
+        ToProcess.push(Child);
+
+      assert(L->isLoopSimplifyForm());
+
+      auto CT = getConvergenceToken(L->getHeader());
+      SmallPtrSet<BasicBlock *, 8> RegionBlocks(L->block_begin(),
+                                                L->block_end());
+      SmallVector<BasicBlock *> LoopExits;
+      L->getExitingBlocks(LoopExits);
+      if (CT.has_value()) {
+        for (auto *Exit : LoopExits) {
+          auto N = findPathsToMatch(Exit, [&CT](const BasicBlock *block) {
+            auto Token = getConvergenceToken(block);
+            if (Token == std::nullopt)
+              return false;
+            return Token.value() == CT.value();
+          });
+          RegionBlocks.insert(N.begin(), N.end());
+        }
+      }
+
+      auto RegionExits = findExitNodes(RegionBlocks);
+      ConvergenceRegion *Region = new ConvergenceRegion(
+          DT, LI, CT, L->getHeader(), std::move(RegionBlocks),
+          std::move(RegionExits));
+
+      auto It = LoopToRegion.find(L->getParentLoop());
+      assert(It != LoopToRegion.end() || L->getParentLoop() == nullptr);
+      Region->Parent = It != LoopToRegion.end() ? It->second : TopLevelRegion;
+      Region->Parent->Children.push_back(Region);
+
+      LoopToRegion.emplace(L, Region);
+    }
+
+    return ConvergenceRegionInfo(TopLevelRegion);
+  }
+
+private:
+  DominatorTree &DT;
+  LoopInfo &LI;
+  Function &F;
+};
+
+ConvergenceRegionInfo getConvergenceRegions(Function &F, DominatorTree &DT,
+                                            LoopInfo &LI) {
+  ConvergenceRegionAnalyzer Analyzer(F, DT, LI);
+  return Analyzer.analyze();
+}
+
+char ConvergenceRegionAnalysisWrapperPass::ID = 0;
+
+ConvergenceRegionAnalysisWrapperPass::ConvergenceRegionAnalysisWrapperPass()
+    : FunctionPass(ID) {}
+
+bool ConvergenceRegionAnalysisWrapperPass::runOnFunction(Function &F) {
+  DominatorTree &DT = getAnalysis<DominatorTreeWrapperPass>().getDomTree();
+  LoopInfo &LI = getAnalysis<LoopInfoWrapperPass>().getLoopInfo();
+
+  CRI = getConvergenceRegions(F, DT, LI);
+  // Nothing was modified.
+  return false;
+}
+
+ConvergenceRegionAnalysis::Result
+ConvergenceRegionAnalysis::run(Function &F, FunctionAnalysisManager &AM) {
+  Result CRI;
+  auto &DT = AM.getResult<DominatorTreeAnalysis>(F);
+  auto &LI = AM.getResult<LoopAnalysis>(F);
+  CRI = getConvergenceRegions(F, DT, LI);
+  return CRI;
+}
+
+AnalysisKey ConvergenceRegionAnalysis::Key;
+
+} // namespace SPIRV
+} // namespace llvm
diff --git a/llvm/lib/Target/SPIRV/Analysis/ConvergenceRegionAnalysis.h b/llvm/lib/Target/SPIRV/Analysis/ConvergenceRegionAnalysis.h
new file mode 100644
index 00000000000000..c8cd1c4cd9ddf7
--- /dev/null
+++ b/llvm/lib/Target/SPIRV/Analysis/ConvergenceRegionAnalysis.h
@@ -0,0 +1,173 @@
+//===- ConvergenceRegionAnalysis.h -----------------------------*- C++ -*--===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// The analysis determines the convergence region for each basic block of
+// the module, and provides a tree-like structure describing the region
+// hierarchy.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_LIB_TARGET_SPIRV_SPIRVCONVERGENCEREGIONANALYSIS_H
+#define LLVM_LIB_TARGET_SPIRV_SPIRVCONVERGENCEREGIONANALYSIS_H
+
+#include "llvm/ADT/SmallPtrSet.h"
+#include "llvm/Analysis/CFG.h"
+#include "llvm/Analysis/LoopInfo.h"
+#include "llvm/IR/Dominators.h"
+#include "llvm/IR/IntrinsicInst.h"
+#include <iostream>
+#include <optional>
+#include <unordered_set>
+
+namespace llvm {
+class SPIRVSubtarget;
+class MachineFunction;
+class MachineModuleInfo;
+
+namespace SPIRV {
+
+// Returns the first convergence intrinsic found in |BB|, |nullopt| otherwise.
+std::optional<IntrinsicInst *> getConvergenceToken(BasicBlock *BB);
+std::optional<const IntrinsicInst *> getConvergenceToken(const BasicBlock *BB);
+
+// Describes a hierarchy of convergence regions.
+// A convergence region defines a CFG for which the execution flow can diverge
+// starting from the entry block, but should reconverge back before the end of
+// the exit blocks.
+class ConvergenceRegion {
+  DominatorTree &DT;
+  LoopInfo &LI;
+
+public:
+  // The parent region of this region, if any.
+  ConvergenceRegion *Parent = nullptr;
+  // The sub-regions contained in this region, if any.
+  SmallVector<ConvergenceRegion *> Children = {};
+  // The convergence instruction linked to this region, if any.
+  std::optional<IntrinsicInst *> ConvergenceToken = std::nullopt;
+  // The only block with a predecessor outside of this region.
+  BasicBlock *Entry = nullptr;
+  // All the blocks with an edge leaving this convergence region.
+  SmallPtrSet<BasicBlock *, 2> Exits = {};
+  // All the blocks that belongs to this region, including its subregions'.
+  SmallPtrSet<BasicBlock *, 8> Blocks = {};
+
+  // Creates a single convergence region encapsulating the whole function |F|.
+  ConvergenceRegion(DominatorTree &DT, LoopInfo &LI, Function &F);
+
+  // Creates a single convergence region defined by entry and exits nodes, a
+  // list of blocks, and possibly a convergence token.
+  ConvergenceRegion(DominatorTree &DT, LoopInfo &LI,
+                    std::optional<IntrinsicInst *> ConvergenceToken,
+                    BasicBlock *Entry, SmallPtrSet<BasicBlock *, 8> &&Blocks,
+                    SmallPtrSet<BasicBlock *, 2> &&Exits);
+
+  ConvergenceRegion(ConvergenceRegion &&CR)
+      : DT(CR.DT), LI(CR.LI), Parent(std::move(CR.Parent)),
+        Children(std::move(CR.Children)),
+        ConvergenceToken(std::move(CR.ConvergenceToken)),
+        Entry(std::move(CR.Entry)), Exits(std::move(CR.Exits)),
+        Blocks(std::move(CR.Blocks)) {}
+
+  ConvergenceRegion(const ConvergenceRegion &other) = delete;
+
+  // Returns true if the given basic block belongs to this region, or to one of
+  // its subregion.
+  bool contains(const BasicBlock *BB) const { return Blocks.count(BB) != 0; }
+
+  void releaseMemory();
+
+  // Write to the debug output this region's hierarchy.
+  // |IndentSize| defines the number of tabs to print before any new line.
+  void dump(const unsigned IndentSize = 0) const;
+};
+
+// Holds a ConvergenceRegion hierarchy.
+class ConvergenceRegionInfo {
+  // The convergence region this structure holds.
+  ConvergenceRegion *TopLevelRegion;
+
+public:
+  ConvergenceRegionInfo() : TopLevelRegion(nullptr) {}
+
+  // Creates a new ConvergenceRegionInfo. Ownership of the TopLevelRegion is
+  // passed to this object.
+  ConvergenceRegionInfo(ConvergenceRegion *TopLevelRegion)
+      : TopLevelRegion(TopLevelRegion) {}
+
+  ~ConvergenceRegionInfo() { releaseMemory(); }
+
+  ConvergenceRegionInfo(ConvergenceRegionInfo &&LHS)
+      : TopLevelRegion(LHS.TopLevelRegion) {
+    if (TopLevelRegion != LHS.TopLevelRegion) {
+      releaseMemory();
+      TopLevelRegion = LHS.TopLevelRegion;
+    }
+    LHS.TopLevelRegion = nullptr;
+  }
+
+  ConvergenceRegionInfo &operator=(ConvergenceRegionInfo &&LHS) {
+    if (TopLevelRegion != LHS.TopLevelRegion) {
+      releaseMemory();
+      TopLevelRegion = LHS.TopLevelRegion;
+    }
+    LHS.TopLevelRegion = nullptr;
+    return *this;
+  }
+
+  void releaseMemory() {
+    if (TopLevelRegion == nullptr)
+      return;
+
+    TopLevelRegion->releaseMemory();
+    delete TopLevelRegion;
+    TopLevelRegion = nullptr;
+  }
+
+  const ConvergenceRegion *getTopLevelRegion() const { return TopLevelRegion; }
+};
+
+// Wrapper around the function above to use it with the legacy pass manager.
+class ConvergenceRegionAnalysisWrapperPass : public FunctionPass {
+  ConvergenceRegionInfo CRI;
+
+public:
+  static char ID;
+
+  ConvergenceRegionAnalysisWrapperPass();
+
+  void getAnalysisUsage(AnalysisUsage &AU) const override {
+    AU.setPreservesAll();
+    AU.addRequired<LoopInfoWrapperPass>();
+    AU.addRequired<DominatorTreeWrapperPass>();
+  };
+
+  bool runOnFunction(Function &F) override;
+
+  ConvergenceRegionInfo &getRegionInfo() { return CRI; }
+  const ConvergenceRegionInfo &getRegionInfo() const { return CRI; }
+};
+
+// Wrapper around the function above to use it with the new pass manager.
+class ConvergenceRegionAnalysis
+    : public AnalysisInfoMixin<ConvergenceRegionAnalysis> {
+  friend AnalysisInfoMixin<ConvergenceRegionAnalysis>;
+  static AnalysisKey Key;
+
+public:
+  using Result = ConvergenceRegionInfo;
+
+  Result run(Function &F, FunctionAnalysisManager &AM);
+};
+
+ConvergenceRegionInfo getConvergenceRegions(Function &F, DominatorTree &DT,
+                                            LoopInfo &LI);
+
+} // namespace SPIRV
+} // namespace llvm
+#endif // LLVM_LIB_TARGET_SPIRV_SPIRVCONVERGENCEREGIONANALYSIS_H
diff --git a/llvm/lib/Target/SPIRV/CMakeLists.txt b/llvm/lib/Target/SPIRV/CMakeLists.txt
index 7d17c307db13a0..76710c44767665 100644
--- a/llvm/lib/Target/SPIRV/CMakeLists.txt
+++ b/llvm/lib/Target/SPIRV/CMakeLists.txt
@@ -44,6 +44,7 @@ add_llvm_target(SPIRVCodeGen
   Core
   Demangle
   GlobalISel
+  SPIRVAnalysis
   MC
   SPIRVDesc
   SPIRVInfo
@@ -59,3 +60,4 @@ add_llvm_target(SPIRVCodeGen
 
 add_subdirectory(MCTargetDesc)
 add_subdirectory(TargetInfo)
+add_subdirectory(Analysis)
diff --git a/llvm/unittests/Target/SPIRV/CMakeLists.txt b/llvm/unittests/Target/SPIRV/CMakeLists.txt
new file mode 100644
index 00000000000000..8f9b81c759a00e
--- /dev/null
+++ b/llvm/unittests/Target/SPIRV/CMakeLists.txt
@@ -0,0 +1,17 @@
+include_directories(
+  ${LLVM_MAIN_SRC_DIR}/lib/Target/SPIRV
+  ${LLVM_BINARY_DIR}/lib/Target/SPIRV
+  )
+
+set(LLVM_LINK_COMPONENTS
+  AsmParser
+  Core
+  SPIRVCodeGen
+  SPIRVAnalysis
+  Support
+  )
+
+add_llvm_target_unittest(SPIRVTests
+  ConvergenceRegionAnalysisTests.cpp
+  )
+
diff --git a/llvm/unittests/Target/SPIRV/ConvergenceRegionAnalysisTests.cpp b/llvm/unittests/Target/SPIRV/ConvergenceRegionAnalysisTests.cpp
new file mode 100644
index 00000000000000..2a01b00dad42a8
--- /dev/null
+++ b/llvm/unittests/Target/SPIRV/ConvergenceRegionAnalysisTests.cpp
@@ -0,0 +1,1015 @@
+//===- llvm/unittests/Target/SPIRV/ConvergenceRegionAnalysisTests.cpp -----===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "Analysis/ConvergenceRegionAnalysis.h"
+#include "llvm/Analysis/DominanceFrontier.h"
+#include "llvm/Analysis/PostDominators.h"
+#include "llvm/AsmParser/Parser.h"
+#include "llvm/IR/Instructions.h"
+#include "llvm/IR/LLVMContext.h"
+#include "llvm/IR/LegacyPassManager.h"
+#include "llvm/IR/Type.h"
+#include "llvm/IR/TypedPointerType.h"
+#include "llvm/Support/SourceMgr.h"
+
+#include "gmock/gmock.h"
+#include "gtest/gtest.h"
+#include <queue>
+
+using ::testing::Contains;
+using ::testing::Pair;
+
+using namespace llvm;
+using namespace llvm::SPIRV;
+
+template <typename T> struct IsA {
+  friend bool operator==(const Value *V, const IsA &) { return isa<T>(V); }
+};
+
+class ConvergenceRegionAnalysisTest : public testing::Test {
+protected:
+  void SetUp() override {
+    // Required for tests.
+    FAM.registerPass([&] { return PassInstrumentationAnalysis(); });
+    MAM.registerPass([&] { return PassInstrumentationAnalysis(); });
+
+    // Required for ConvergenceRegionAnalysis.
+    FAM.registerPass([&] { return DominatorTreeAnalysis(); });
+    FAM.registerPass([&] { return LoopAnalysis(); });
+
+    FAM.registerPass([&] { return ConvergenceRegionAnalysis(); });
+  }
+
+  void TearDown() override { M.reset(); }
+
+  ConvergenceRegionAnalysis::Result &runAnalysis(StringRef Assembly) {
+    assert(M == nullptr &&
+           "Calling runAnalysis multiple times is unsafe. See getAnalysis().");
+
+    SMDiagnostic Error;
+    M = parseAssemblyString(Assembly, Error, Context);
+    assert(M && "Bad assembly. Bad test?");
+    auto *F = getFunction();
+
+    ModulePassManager MPM;
+    MPM.run(*M, MAM);
+    return FAM.getResult<Co...
[truncated]

@Keenuts
Copy link
Contributor Author

Keenuts commented Jan 19, 2024

Had to rename the pass and move it outside of the SPIRV namespace.
This was because to use the pass with the old pass manager (hence this backend), I need to use the INITIALIZE_PASS defines.
Those defines require the namespace around the pass to be llvm:: AFAIK.
moved the passes to the global llvm namespace, while keeping 'internal' stuff and structures in the SPIRV namespace.
Because I moved it to the shared namespace, I prefixed the passes with the SPIRV prefix.

Copy link
Contributor

@s-perron s-perron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is only one case I can think of that may not be addressed. Otherwise looks go to me.

@Keenuts
Copy link
Contributor Author

Keenuts commented Jan 29, 2024

@michalpaszkowski I was able to build the next steps of the structurizer. So if you are OK, I'm fine to merge this analysis.

The issue I had was:

  • I needed to generate OpLoopMerge instruction in the backend, hence working on MIR.
  • But convergence intrinsics were only available at the IR level.
  • Convergence intrinsics could not be lowered down to MIR, meaning I needed to strip them.
  • This made this analysis useless.

But I found this PR: #67006
Rebasing my structurizer on it solved all my issues: since I can now keep the intrinsics in the IR, I can do the proper work in the backend, as expected.

@Keenuts Keenuts merged commit 7b08b43 into llvm:main Feb 2, 2024
5 checks passed
@Keenuts Keenuts deleted the identify-convergence-regions branch February 2, 2024 17:22
agozillon pushed a commit to agozillon/llvm-project that referenced this pull request Feb 5, 2024
This new analysis returns a hierarchical view of the convergence regions
in the given function.
This will allow our passes to query which basic block belongs to which
convergence region, and structurize the code in consequence.

Definition
----------

A convergence region is a CFG with:
 - a single entry node.
 - one or multiple exit nodes (different from LLVM's regions).
 - one back-edge
 - zero or more subregions.

Excluding sub-regions nodes, the nodes of a region can only reference a
single convergence token. A subregion uses a different convergence
token.

Algorithm
---------

This algorithm assumes all loops are in the Simplify form.

Create an initial convergence region for the whole function.
  - the convergence token is the function entry token.
  - the entry is the function entrypoint.
- Exits are all the basic blocks terminating with a return instruction.

Take the function CFG, and process it in DAG order (ignoring
back-edges). If a basic block is a loop header:
 - Create a new region.
- The parent region is the parent's loop region if any, otherwise, the
top level region.
   - The region blocks are all the blocks belonging to this loop.
- For each loop exit: - visit the rest of the CFG in DAG order (ignore
back-edges). - if the region's convergence token is found, add all the
blocks dominated by the exit from which the token is reachable to the
region.
   - continue the algorithm with the loop headers successors.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants