-
Notifications
You must be signed in to change notification settings - Fork 15.1k
[mlir][acc] Add ACCImplicitData pass for implicit data attributes #166472
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
This change adds the ACCImplicitData pass which implements the
OpenACC specification for "Variables with Implicitly Determined
Data Attributes" (OpenACC 3.4 spec, section 2.6.2).
The pass automatically generates data clause operations (copyin,
copyout, present, firstprivate, etc.) for variables used within
OpenACC compute constructs (parallel, kernels, serial) that do
not already have explicit data clauses.
Key features:
- Respects default(none) and default(present) clauses
- Handles scalar vs. aggregate variables with different
semantics:
* Aggregates: present clause (if default(present)) or copy
clause
* Scalars: copy clause (kernels) or firstprivate
(parallel/serial)
- Generates privatization recipes when needed for firstprivate
clauses
- Performs alias analysis to avoid redundant data mappings
- Ensures proper data clause ordering for partial entity access
- Generates exit operations (copyout, delete) to match entry
operations
Requirements:
- Types must implement acc::MappableType and/or
acc::PointerLikeType interfaces to be considered candidates.
- Operations accessing partial entities or creating subviews
should implement acc::PartialEntityAccess and/or
mlir::ViewLikeOpInterface for proper clause ordering.
- Optionally pre-register acc::OpenACCSupport and
mlir::AliasAnalysis if custom alias analysis, variable name
determination, or error reporting is needed.
|
@llvm/pr-subscribers-mlir-openacc @llvm/pr-subscribers-openacc Author: Razvan Lupusoru (razvanlupusoru) ChangesThis change adds the ACCImplicitData pass which implements the OpenACC specification for "Variables with Implicitly Determined Data Attributes" (OpenACC 3.4 spec, section 2.6.2). The pass automatically generates data clause operations (copyin, copyout, present, firstprivate, etc.) for variables used within OpenACC compute constructs (parallel, kernels, serial) that do not already have explicit data clauses. Key features:
Requirements:
Patch is 54.24 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/166472.diff 6 Files Affected:
diff --git a/mlir/include/mlir/Dialect/OpenACC/Transforms/Passes.h b/mlir/include/mlir/Dialect/OpenACC/Transforms/Passes.h
index 57d532b078b9e..27f65aa15f040 100644
--- a/mlir/include/mlir/Dialect/OpenACC/Transforms/Passes.h
+++ b/mlir/include/mlir/Dialect/OpenACC/Transforms/Passes.h
@@ -9,6 +9,9 @@
#ifndef MLIR_DIALECT_OPENACC_TRANSFORMS_PASSES_H
#define MLIR_DIALECT_OPENACC_TRANSFORMS_PASSES_H
+#include "mlir/Dialect/Arith/IR/Arith.h"
+#include "mlir/Dialect/MemRef/IR/MemRef.h"
+#include "mlir/Dialect/OpenACC/OpenACC.h"
#include "mlir/Pass/Pass.h"
namespace mlir {
diff --git a/mlir/include/mlir/Dialect/OpenACC/Transforms/Passes.td b/mlir/include/mlir/Dialect/OpenACC/Transforms/Passes.td
index 9ceb91e5679a1..40ccd1fc6c1a0 100644
--- a/mlir/include/mlir/Dialect/OpenACC/Transforms/Passes.td
+++ b/mlir/include/mlir/Dialect/OpenACC/Transforms/Passes.td
@@ -27,4 +27,40 @@ def LegalizeDataValuesInRegion : Pass<"openacc-legalize-data-values", "mlir::fun
];
}
+def ACCImplicitData : Pass<"acc-implicit-data", "mlir::ModuleOp"> {
+ let summary = "Generate implicit data attributes for OpenACC compute constructs";
+ let description = [{
+ This pass implements the OpenACC specification for "Variables with
+ Implicitly Determined Data Attributes" (OpenACC 3.4 spec, section 2.6.2).
+
+ The pass automatically generates data clause operations for variables used
+ within OpenACC compute constructs (parallel, kernels, serial) that do not
+ already have explicit data clauses. The semantics follow these rules:
+
+ 1. If there is a default(none) clause visible, no implicit data actions
+ apply.
+
+ 2. An aggregate variable (arrays, derived types, etc.) will be treated as:
+ - In a present clause when default(present) is visible.
+ - In a copy clause otherwise.
+
+ 3. A scalar variable will be treated as if it appears in:
+ - A copy clause if the compute construct is a kernels construct.
+ - A firstprivate clause otherwise (parallel, serial).
+ }];
+ let dependentDialects = ["mlir::acc::OpenACCDialect",
+ "mlir::memref::MemRefDialect",
+ "mlir::arith::ArithDialect"];
+ let options = [
+ Option<"enableImplicitReductionCopy", "enable-implicit-reduction-copy",
+ "bool", "true",
+ "Enable applying implicit copy in lieu of implicit firstprivate for "
+ "reduction variables. This allows uniform treatment of reduction "
+ "variables between combined constructs (e.g., 'parallel loop') and "
+ "separate constructs (e.g., 'parallel' followed by 'loop'), where "
+ "the OpenACC spec requires copy semantics for the former but "
+ "firstprivate would normally apply for the latter.">
+ ];
+}
+
#endif // MLIR_DIALECT_OPENACC_TRANSFORMS_PASSES
diff --git a/mlir/lib/Dialect/OpenACC/Transforms/ACCImplicitData.cpp b/mlir/lib/Dialect/OpenACC/Transforms/ACCImplicitData.cpp
new file mode 100644
index 0000000000000..88de14950cd70
--- /dev/null
+++ b/mlir/lib/Dialect/OpenACC/Transforms/ACCImplicitData.cpp
@@ -0,0 +1,904 @@
+//===- ACCImplicitData.cpp ------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This pass implements the OpenACC specification for "Variables with
+// Implicitly Determined Data Attributes" (OpenACC 3.4 spec, section 2.6.2).
+//
+// Overview:
+// ---------
+// The pass automatically generates data clause operations for variables used
+// within OpenACC compute constructs (parallel, kernels, serial) that do not
+// already have explicit data clauses. The semantics follow these rules:
+//
+// 1. If there is a default(none) clause visible, no implicit data actions
+// apply.
+//
+// 2. An aggregate variable (arrays, derived types, etc.) will be treated as:
+// - In a present clause when default(present) is visible.
+// - In a copy clause otherwise.
+//
+// 3. A scalar variable will be treated as if it appears in:
+// - A copy clause if the compute construct is a kernels construct.
+// - A firstprivate clause otherwise (parallel, serial).
+//
+// Requirements:
+// -------------
+// To use this pass in a pipeline, the following requirements must be met:
+//
+// 1. Type Interface Implementation: Variables from the dialect being used
+// must implement one or both of the following MLIR interfaces:
+// `acc::MappableType` and/or `acc::PointerLikeType`
+//
+// These interfaces provide the necessary methods for the pass to:
+// - Determine variable type categories (scalar vs. aggregate)
+// - Generate appropriate bounds information
+// - Generate privatization recipes
+//
+// 2. Operation Interface Implementation: Operations that access partial
+// entities or create views should implement the following MLIR
+// interfaces: `acc::PartialEntityAccess` and/or
+// `mlir::ViewLikeOpInterface`
+//
+// These interfaces are used for proper data clause ordering, ensuring
+// that base entities are mapped before derived entities (e.g., a
+// struct is mapped before its fields, an array is mapped before
+// subarray views).
+//
+// 3. Analysis Registration (Optional): If custom behavior is needed for
+// variable name extraction or alias analysis, the dialect should
+// pre-register the `acc::OpenACCSupport` and `mlir::AliasAnalysis` analyses.
+//
+// If not registered, default behavior will be used.
+//
+// Implementation Details:
+// -----------------------
+// The pass performs the following operations:
+//
+// 1. Finds candidate variables which are live-in to the compute region and
+// are not already in a data clause or private clause.
+//
+// 2. Generates both data "entry" and "exit" clause operations that match
+// the intended action depending on variable type:
+// - copy -> acc.copyin (entry) + acc.copyout (exit)
+// - present -> acc.present (entry) + acc.delete (exit)
+// - firstprivate -> acc.firstprivate (entry only, no exit)
+//
+// 3. Ensures that default clause is taken into consideration by looking
+// through current construct and parent constructs to find the "visible
+// default clause".
+//
+// 4. Fixes up SSA value links so that uses in the acc region reference the
+// result of the newly created data clause operations.
+//
+// 5. When generating implicit data clause operations, it also adds variable
+// name information and marks them with the implicit flag.
+//
+// 6. Recipes are generated by calling the appropriate entrypoints in the
+// MappableType and PointerLikeType interfaces.
+//
+// 7. AliasAnalysis is used to determine if a variable is already covered by
+// an existing data clause (e.g., an interior pointer covered by its parent).
+//
+// Examples:
+// ---------
+//
+// Example 1: Scalar in parallel construct (implicit firstprivate)
+//
+// Before:
+// func.func @test() {
+// %scalar = memref.alloca() {acc.var_name = "x"} : memref<f32>
+// acc.parallel {
+// %val = memref.load %scalar[] : memref<f32>
+// acc.yield
+// }
+// }
+//
+// After:
+// func.func @test() {
+// %scalar = memref.alloca() {acc.var_name = "x"} : memref<f32>
+// %firstpriv = acc.firstprivate varPtr(%scalar : memref<f32>)
+// -> memref<f32> {implicit = true, name = "x"}
+// acc.parallel firstprivate(@recipe -> %firstpriv : memref<f32>) {
+// %val = memref.load %firstpriv[] : memref<f32>
+// acc.yield
+// }
+// }
+//
+// Example 2: Scalar in kernels construct (implicit copy)
+//
+// Before:
+// func.func @test() {
+// %scalar = memref.alloca() {acc.var_name = "n"} : memref<i32>
+// acc.kernels {
+// %val = memref.load %scalar[] : memref<i32>
+// acc.terminator
+// }
+// }
+//
+// After:
+// func.func @test() {
+// %scalar = memref.alloca() {acc.var_name = "n"} : memref<i32>
+// %copyin = acc.copyin varPtr(%scalar : memref<i32>) -> memref<i32>
+// {dataClause = #acc<data_clause acc_copy>,
+// implicit = true, name = "n"}
+// acc.kernels dataOperands(%copyin : memref<i32>) {
+// %val = memref.load %copyin[] : memref<i32>
+// acc.terminator
+// }
+// acc.copyout accPtr(%copyin : memref<i32>)
+// to varPtr(%scalar : memref<i32>)
+// {dataClause = #acc<data_clause acc_copy>,
+// implicit = true, name = "n"}
+// }
+//
+// Example 3: Array (aggregate) in parallel (implicit copy)
+//
+// Before:
+// func.func @test() {
+// %array = memref.alloca() {acc.var_name = "arr"} : memref<100xf32>
+// acc.parallel {
+// %c0 = arith.constant 0 : index
+// %val = memref.load %array[%c0] : memref<100xf32>
+// acc.yield
+// }
+// }
+//
+// After:
+// func.func @test() {
+// %array = memref.alloca() {acc.var_name = "arr"} : memref<100xf32>
+// %copyin = acc.copyin varPtr(%array : memref<100xf32>)
+// -> memref<100xf32>
+// {dataClause = #acc<data_clause acc_copy>,
+// implicit = true, name = "arr"}
+// acc.parallel dataOperands(%copyin : memref<100xf32>) {
+// %c0 = arith.constant 0 : index
+// %val = memref.load %copyin[%c0] : memref<100xf32>
+// acc.yield
+// }
+// acc.copyout accPtr(%copyin : memref<100xf32>)
+// to varPtr(%array : memref<100xf32>)
+// {dataClause = #acc<data_clause acc_copy>,
+// implicit = true, name = "arr"}
+// }
+//
+// Example 4: Array with default(present)
+//
+// Before:
+// func.func @test() {
+// %array = memref.alloca() {acc.var_name = "arr"} : memref<100xf32>
+// acc.parallel {
+// %c0 = arith.constant 0 : index
+// %val = memref.load %array[%c0] : memref<100xf32>
+// acc.yield
+// } attributes {defaultAttr = #acc<defaultvalue present>}
+// }
+//
+// After:
+// func.func @test() {
+// %array = memref.alloca() {acc.var_name = "arr"} : memref<100xf32>
+// %present = acc.present varPtr(%array : memref<100xf32>)
+// -> memref<100xf32>
+// {implicit = true, name = "arr"}
+// acc.parallel dataOperands(%present : memref<100xf32>)
+// attributes {defaultAttr = #acc<defaultvalue present>} {
+// %c0 = arith.constant 0 : index
+// %val = memref.load %present[%c0] : memref<100xf32>
+// acc.yield
+// }
+// acc.delete accPtr(%present : memref<100xf32>)
+// {dataClause = #acc<data_clause acc_present>,
+// implicit = true, name = "arr"}
+// }
+//
+//===----------------------------------------------------------------------===//
+
+#include "mlir/Dialect/OpenACC/Transforms/Passes.h"
+
+#include "mlir/Analysis/AliasAnalysis.h"
+#include "mlir/Dialect/Func/IR/FuncOps.h"
+#include "mlir/Dialect/OpenACC/Analysis/OpenACCSupport.h"
+#include "mlir/Dialect/OpenACC/OpenACC.h"
+#include "mlir/Dialect/OpenACC/OpenACCUtils.h"
+#include "mlir/IR/Builders.h"
+#include "mlir/IR/BuiltinOps.h"
+#include "mlir/IR/Dominance.h"
+#include "mlir/IR/Operation.h"
+#include "mlir/IR/Value.h"
+#include "mlir/Interfaces/ViewLikeInterface.h"
+#include "mlir/Transforms/RegionUtils.h"
+#include "llvm/ADT/STLExtras.h"
+#include "llvm/ADT/SmallVector.h"
+#include "llvm/ADT/TypeSwitch.h"
+#include "llvm/Support/ErrorHandling.h"
+#include <type_traits>
+
+namespace mlir {
+namespace acc {
+#define GEN_PASS_DEF_ACCIMPLICITDATA
+#include "mlir/Dialect/OpenACC/Transforms/Passes.h.inc"
+} // namespace acc
+} // namespace mlir
+
+#define DEBUG_TYPE "acc-implicit-data"
+
+using namespace mlir;
+
+namespace {
+
+class ACCImplicitData
+ : public mlir::acc::impl::ACCImplicitDataBase<ACCImplicitData> {
+public:
+ using mlir::acc::impl::ACCImplicitDataBase<
+ ACCImplicitData>::ACCImplicitDataBase;
+
+ void runOnOperation() override;
+
+private:
+ /// Collects all data clauses that dominate the compute construct.
+ /// Needed to determine if a variable is already covered by an existing data
+ /// clause.
+ SmallVector<mlir::Value>
+ getDominatingDataClauses(Operation *computeConstructOp);
+
+ /// Looks through the `dominatingDataClauses` to find the original data clause
+ /// op for an alias. Returns nullptr if no original data clause op is found.
+ template <typename OpT>
+ Operation *getOriginalDataClauseOpForAlias(
+ Value var, OpBuilder &builder, OpT computeConstructOp,
+ const SmallVector<mlir::Value> &dominatingDataClauses);
+
+ /// Generates the appropriate `acc.copyin`, `acc.present`,`acc.firstprivate`,
+ /// etc. data clause op for a candidate variable.
+ template <typename OpT>
+ Operation *generateDataClauseOpForCandidate(
+ Value var, ModuleOp &module, OpBuilder &builder, OpT computeConstructOp,
+ const SmallVector<mlir::Value> &dominatingDataClauses,
+ const std::optional<acc::ClauseDefaultValue> &defaultClause);
+
+ /// Generates the implicit data ops for a compute construct.
+ template <typename OpT>
+ void generateImplicitDataOps(
+ ModuleOp &module, OpT computeConstructOp,
+ std::optional<acc::ClauseDefaultValue> &defaultClause);
+
+ /// Generates a private recipe for a variable.
+ acc::PrivateRecipeOp generatePrivateRecipe(ModuleOp &module, mlir::Value var,
+ mlir::Location loc,
+ OpBuilder &builder,
+ acc::OpenACCSupport &accSupport);
+
+ /// Generates a firstprivate recipe for a variable.
+ acc::FirstprivateRecipeOp
+ generateFirstprivateRecipe(ModuleOp &module, mlir::Value var,
+ mlir::Location loc, OpBuilder &builder,
+ acc::OpenACCSupport &accSupport);
+
+ /// Generates recipes for a list of variables.
+ void generateRecipes(ModuleOp &module, OpBuilder &builder,
+ Operation *computeConstructOp,
+ const SmallVector<Value> &newOperands,
+ SmallVector<Attribute> &newRecipeSyms);
+};
+
+/// Determines if a variable is a candidate for implicit data mapping.
+/// Returns true if the variable is a candidate, false otherwise.
+static bool isCandidateForImplicitData(Value val, Region &accRegion) {
+ // Ensure the variable is an allowed type for data clause.
+ if (!acc::isPointerLikeType(val.getType()) &&
+ !acc::isMappableType(val.getType())) {
+ return false;
+ }
+
+ // If this is already coming from a data clause, we do not need to generate
+ // another.
+ if (isa_and_nonnull<ACC_DATA_ENTRY_OPS>(val.getDefiningOp())) {
+ return false;
+ }
+
+ // If this is only used by private clauses, it is not a real live-in.
+ if (acc::isOnlyUsedByPrivateClauses(val, accRegion)) {
+ return false;
+ }
+
+ return true;
+}
+
+SmallVector<mlir::Value>
+ACCImplicitData::getDominatingDataClauses(Operation *computeConstructOp) {
+ llvm::SmallSetVector<mlir::Value, 8> dominatingDataClauses;
+
+ llvm::TypeSwitch<Operation *>(computeConstructOp)
+ .Case<acc::ParallelOp, acc::KernelsOp, acc::SerialOp>([&](auto op) {
+ for (auto dataClause : op.getDataClauseOperands()) {
+ dominatingDataClauses.insert(dataClause);
+ }
+ })
+ .Default([](Operation *) {});
+
+ // Collect the data clauses from enclosing data constructs.
+ Operation *currParentOp = computeConstructOp->getParentOp();
+ while (currParentOp) {
+ if (isa<acc::DataOp>(currParentOp)) {
+ for (auto dataClause :
+ dyn_cast<acc::DataOp>(currParentOp).getDataClauseOperands()) {
+ dominatingDataClauses.insert(dataClause);
+ }
+ }
+ currParentOp = currParentOp->getParentOp();
+ }
+
+ // Find the enclosing function/subroutine
+ Operation *funcOp = computeConstructOp->getParentOfType<func::FuncOp>();
+
+ if (!funcOp) {
+ return dominatingDataClauses.takeVector();
+ }
+
+ // Walk the function to find `acc.declare_enter`/`acc.declare_exit` pairs that
+ // dominate and post-dominate the compute construct and add their data
+ // clauses to the list.
+ auto &domInfo = this->getAnalysis<DominanceInfo>();
+ auto &postDomInfo = this->getAnalysis<PostDominanceInfo>();
+ funcOp->walk([&](mlir::acc::DeclareEnterOp declareEnterOp) {
+ if (domInfo.dominates(declareEnterOp.getOperation(), computeConstructOp)) {
+ // Collect all `acc.declare_exit` ops for this token.
+ SmallVector<acc::DeclareExitOp> exits;
+ for (auto *user : declareEnterOp.getToken().getUsers()) {
+ if (auto declareExit = dyn_cast<acc::DeclareExitOp>(user)) {
+ exits.push_back(declareExit);
+ }
+ }
+ // Only add clauses if every `acc.declare_exit` op post-dominates the
+ // compute construct.
+ if (!exits.empty() && llvm::all_of(exits, [&](acc::DeclareExitOp exitOp) {
+ return postDomInfo.postDominates(exitOp, computeConstructOp);
+ })) {
+ for (auto dataClause : declareEnterOp.getDataClauseOperands()) {
+ dominatingDataClauses.insert(dataClause);
+ }
+ }
+ }
+ });
+
+ return dominatingDataClauses.takeVector();
+}
+
+template <typename OpT>
+Operation *ACCImplicitData::getOriginalDataClauseOpForAlias(
+ Value var, OpBuilder &builder, OpT computeConstructOp,
+ const SmallVector<mlir::Value> &dominatingDataClauses) {
+ auto &aliasAnalysis = this->getAnalysis<AliasAnalysis>();
+ for (auto dataClause : dominatingDataClauses) {
+ if (auto *dataClauseOp = dataClause.getDefiningOp()) {
+ // Only accept clauses that guarantee that the alias is present.
+ if (isa<acc::CopyinOp, acc::CreateOp, acc::PresentOp, acc::NoCreateOp,
+ acc::DevicePtrOp>(dataClauseOp)) {
+ if (aliasAnalysis.alias(acc::getVar(dataClauseOp), var).isMust()) {
+ return dataClauseOp;
+ }
+ }
+ }
+ }
+ return nullptr;
+}
+
+// Generates bounds for variables that have unknown dimensions
+static void fillInBoundsForUnknownDimensions(mlir::Operation *dataClauseOp,
+ OpBuilder &builder) {
+
+ if (!mlir::acc::getBounds(dataClauseOp).empty())
+ // If bounds are already present, do not overwrite them.
+ return;
+
+ // For types that have unknown dimensions, attempt to generate bounds by
+ // relying on MappableType being able to extract it from the IR.
+ auto var = mlir::acc::getVar(dataClauseOp);
+ auto type = var.getType();
+ if (auto mappableTy = dyn_cast<acc::MappableType>(type)) {
+ if (mappableTy.hasUnknownDimensions()) {
+ TypeSwitch<mlir::Operation *>(dataClauseOp)
+ .Case<ACC_DATA_ENTRY_OPS, ACC_DATA_EXIT_OPS>([&](auto dataClauseOp) {
+ if (std::is_same_v<decltype(dataClauseOp), mlir::acc::DevicePtrOp>)
+ return;
+ OpBuilder::InsertionGuard guard(builder);
+ builder.setInsertionPoint(dataClauseOp);
+ auto bounds = mappableTy.generateAccBounds(var, builder);
+ if (!bounds.empty())
+ dataClauseOp.getBoundsMutable().assign(bounds);
+ });
+ }
+ }
+}
+
+acc::PrivateRecipeOp
+ACCImplicitData::generatePrivateRecipe(ModuleOp &module, mlir::Value var,
+ mlir::Location loc, OpBuilder &builder,
+ acc::OpenACCSupport &accSupport) {
+ auto type = var.getType();
+ std::string recipeName =
+ accSupport.getRecipeName(acc::RecipeKind::private_recipe, type, var);
+
+ // Check if recipe already exists
+ auto existingRecipe = module.lookupSymbol<acc::PrivateRecipeOp>(recipeName);
+ if (existingRecipe)
+ return existingRecipe;
+
+ // Set insertion point to module body in a scoped way
+ OpBuilder::InsertionGuard guard(builder);
+ builder.setInsertionPointToStart(module.getBody());
+
+ auto recipe = mlir::acc::PrivateRecipeOp::createAndPopulate(builder, loc,
+ recipeName, type);
+ if (!recipe.has_value())
+ return accSupport.emitNYI(loc, "implicit private"), nullptr;
+ return recipe.value();
+}
+
+acc::FirstprivateRecipeOp ACCImpli...
[truncated]
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks amazing! Thank you, Razvan, for all the work!
I just have a couple of minor remarks. I am approving this PR, so feel free to ignore them.
This change adds the ACCImplicitData pass which implements the OpenACC specification for "Variables with Implicitly Determined Data Attributes" (OpenACC 3.4 spec, section 2.6.2).
The pass automatically generates data clause operations (copyin, copyout, present, firstprivate, etc.) for variables used within OpenACC compute constructs (parallel, kernels, serial) that do not already have explicit data clauses.
Key features:
Requirements: