Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[mlir] Add global and program memory space handling to the data layout subsystem #77367

Merged

Conversation

agozillon
Copy link
Contributor

This patch is based on a previous PR https://reviews.llvm.org/D144657 that added alloca address space handling to MLIR's DataLayout and DLTI interface. This patch aims to add identical features to import and access the global and program memory space through MLIR's DataLayout/DLTI system.

…t subsystem

This patch is based on a previous PR https://reviews.llvm.org/D144657 that added
alloca address space handling to MLIR's DataLayout and DLTI interface. This patch
aims to add identical features to access the global and program memory space
through MLIR's DataLayout/DLTI system.
@llvmbot
Copy link
Collaborator

llvmbot commented Jan 8, 2024

@llvm/pr-subscribers-mlir-dlti

Author: None (agozillon)

Changes

This patch is based on a previous PR https://reviews.llvm.org/D144657 that added alloca address space handling to MLIR's DataLayout and DLTI interface. This patch aims to add identical features to import and access the global and program memory space through MLIR's DataLayout/DLTI system.


Patch is 24.02 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/77367.diff

12 Files Affected:

  • (modified) mlir/include/mlir/Dialect/DLTI/DLTI.h (+6)
  • (modified) mlir/include/mlir/Dialect/DLTI/DLTIBase.td (+6)
  • (modified) mlir/include/mlir/Interfaces/DataLayoutInterfaces.h (+17-1)
  • (modified) mlir/include/mlir/Interfaces/DataLayoutInterfaces.td (+36)
  • (modified) mlir/lib/Dialect/DLTI/DLTI.cpp (+18)
  • (modified) mlir/lib/Interfaces/DataLayoutInterfaces.cpp (+60-2)
  • (modified) mlir/lib/Target/LLVMIR/DataLayoutImporter.cpp (+19-4)
  • (modified) mlir/lib/Target/LLVMIR/DataLayoutImporter.h (+2-1)
  • (modified) mlir/lib/Target/LLVMIR/ModuleTranslation.cpp (+20)
  • (modified) mlir/test/Dialect/LLVMIR/layout.mlir (+18)
  • (modified) mlir/test/lib/Dialect/DLTI/TestDataLayoutQuery.cpp (+10)
  • (modified) mlir/unittests/Interfaces/DataLayoutInterfacesTest.cpp (+38)
diff --git a/mlir/include/mlir/Dialect/DLTI/DLTI.h b/mlir/include/mlir/Dialect/DLTI/DLTI.h
index b9a8763eb44989..bf23aa2d48a801 100644
--- a/mlir/include/mlir/Dialect/DLTI/DLTI.h
+++ b/mlir/include/mlir/Dialect/DLTI/DLTI.h
@@ -103,6 +103,12 @@ class DataLayoutSpecAttr
   /// Returns the alloca memory space identifier.
   StringAttr getAllocaMemorySpaceIdentifier(MLIRContext *context) const;
 
+  /// Returns the program memory space identifier.
+  StringAttr getProgramMemorySpaceIdentifier(MLIRContext *context) const;
+
+  /// Returns the global memory space identifier.
+  StringAttr getGlobalMemorySpaceIdentifier(MLIRContext *context) const;
+
   /// Returns the stack alignment identifier.
   StringAttr getStackAlignmentIdentifier(MLIRContext *context) const;
 
diff --git a/mlir/include/mlir/Dialect/DLTI/DLTIBase.td b/mlir/include/mlir/Dialect/DLTI/DLTIBase.td
index 7d519f4efcd224..3572a99fad8743 100644
--- a/mlir/include/mlir/Dialect/DLTI/DLTIBase.td
+++ b/mlir/include/mlir/Dialect/DLTI/DLTIBase.td
@@ -39,6 +39,12 @@ def DLTI_Dialect : Dialect {
 
     constexpr const static ::llvm::StringLiteral
     kDataLayoutAllocaMemorySpaceKey = "dlti.alloca_memory_space";
+    
+    constexpr const static ::llvm::StringLiteral
+    kDataLayoutProgramMemorySpaceKey = "dlti.program_memory_space";
+
+    constexpr const static ::llvm::StringLiteral
+    kDataLayoutGlobalMemorySpaceKey = "dlti.global_memory_space";
 
     constexpr const static ::llvm::StringLiteral
     kDataLayoutStackAlignmentKey = "dlti.stack_alignment";
diff --git a/mlir/include/mlir/Interfaces/DataLayoutInterfaces.h b/mlir/include/mlir/Interfaces/DataLayoutInterfaces.h
index f4b9b95fb89f89..4a21f76dfc5d1c 100644
--- a/mlir/include/mlir/Interfaces/DataLayoutInterfaces.h
+++ b/mlir/include/mlir/Interfaces/DataLayoutInterfaces.h
@@ -61,6 +61,14 @@ getDefaultPreferredAlignment(Type type, const DataLayout &dataLayout,
 /// DataLayoutInterface if specified, otherwise returns the default.
 Attribute getDefaultAllocaMemorySpace(DataLayoutEntryInterface entry);
 
+/// Default handler for program memory space request. Dispatches to the
+/// DataLayoutInterface if specified, otherwise returns the default.
+Attribute getDefaultProgramMemorySpace(DataLayoutEntryInterface entry);
+
+/// Default handler for global memory space request. Dispatches to the
+/// DataLayoutInterface if specified, otherwise returns the default.
+Attribute getDefaultGlobalMemorySpace(DataLayoutEntryInterface entry);
+
 /// Default handler for the stack alignment request. Dispatches to the
 /// DataLayoutInterface if specified, otherwise returns the default.
 uint64_t getDefaultStackAlignment(DataLayoutEntryInterface entry);
@@ -175,6 +183,12 @@ class DataLayout {
   /// Returns the memory space used for AllocaOps.
   Attribute getAllocaMemorySpace() const;
 
+  /// Returns the memory space used for program memory operations.
+  Attribute getProgramMemorySpace() const;
+
+  /// Returns the memory space used for global operations.
+  Attribute getGlobalMemorySpace() const;
+
   /// Returns the natural alignment of the stack in bits. Alignment promotion of
   /// stack variables should be limited to the natural stack alignment to
   /// prevent dynamic stack alignment. Returns zero if the stack alignment is
@@ -203,8 +217,10 @@ class DataLayout {
   mutable DenseMap<Type, uint64_t> abiAlignments;
   mutable DenseMap<Type, uint64_t> preferredAlignments;
 
-  /// Cache for alloca memory space.
+  /// Cache for alloca, global, and program memory spaces.
   mutable std::optional<Attribute> allocaMemorySpace;
+  mutable std::optional<Attribute> programMemorySpace;
+  mutable std::optional<Attribute> globalMemorySpace;
 
   /// Cache for stack alignment.
   mutable std::optional<uint64_t> stackAlignment;
diff --git a/mlir/include/mlir/Interfaces/DataLayoutInterfaces.td b/mlir/include/mlir/Interfaces/DataLayoutInterfaces.td
index 2f60a16baf50bc..a8def967fffcfa 100644
--- a/mlir/include/mlir/Interfaces/DataLayoutInterfaces.td
+++ b/mlir/include/mlir/Interfaces/DataLayoutInterfaces.td
@@ -112,6 +112,18 @@ def DataLayoutSpecInterface : AttrInterface<"DataLayoutSpecInterface"> {
       /*methodName=*/"getAllocaMemorySpaceIdentifier",
       /*args=*/(ins "::mlir::MLIRContext *":$context)
     >,
+    InterfaceMethod<
+      /*description=*/"Returns the program memory space identifier.",
+      /*retTy=*/"::mlir::StringAttr",
+      /*methodName=*/"getProgramMemorySpaceIdentifier",
+      /*args=*/(ins "::mlir::MLIRContext *":$context)
+    >,
+    InterfaceMethod<
+      /*description=*/"Returns the global memory space identifier.",
+      /*retTy=*/"::mlir::StringAttr",
+      /*methodName=*/"getGlobalMemorySpaceIdentifier",
+      /*args=*/(ins "::mlir::MLIRContext *":$context)
+    >,
     InterfaceMethod<
       /*description=*/"Returns the stack alignment identifier.",
       /*retTy=*/"::mlir::StringAttr",
@@ -280,6 +292,30 @@ def DataLayoutOpInterface : OpInterface<"DataLayoutOpInterface"> {
         return ::mlir::detail::getDefaultAllocaMemorySpace(entry);
       }]
     >,
+    StaticInterfaceMethod<
+      /*description=*/"Returns the memory space used by the ABI computed "
+                      "using the relevant entries. The data layout object "
+                      "can be used for recursive queries.",
+      /*retTy=*/"::mlir::Attribute",
+      /*methodName=*/"getProgramMemorySpace",
+      /*args=*/(ins "::mlir::DataLayoutEntryInterface":$entry),
+      /*methodBody=*/"",
+      /*defaultImplementation=*/[{
+        return ::mlir::detail::getDefaultProgramMemorySpace(entry);
+      }]
+    >,
+    StaticInterfaceMethod<
+      /*description=*/"Returns the memory space used by the ABI computed "
+                      "using the relevant entries. The data layout object "
+                      "can be used for recursive queries.",
+      /*retTy=*/"::mlir::Attribute",
+      /*methodName=*/"getGlobalMemorySpace",
+      /*args=*/(ins "::mlir::DataLayoutEntryInterface":$entry),
+      /*methodBody=*/"",
+      /*defaultImplementation=*/[{
+        return ::mlir::detail::getDefaultGlobalMemorySpace(entry);
+      }]
+    >,
     StaticInterfaceMethod<
       /*description=*/"Returns the natural stack alignment in bits computed "
                       "using the relevant entries. The data layout object "
diff --git a/mlir/lib/Dialect/DLTI/DLTI.cpp b/mlir/lib/Dialect/DLTI/DLTI.cpp
index aba9e7db0a2fef..daef2349430d0b 100644
--- a/mlir/lib/Dialect/DLTI/DLTI.cpp
+++ b/mlir/lib/Dialect/DLTI/DLTI.cpp
@@ -108,6 +108,11 @@ void DataLayoutEntryAttr::print(AsmPrinter &os) const {
 constexpr const StringLiteral mlir::DataLayoutSpecAttr::kAttrKeyword;
 constexpr const StringLiteral
     mlir::DLTIDialect::kDataLayoutAllocaMemorySpaceKey;
+constexpr const StringLiteral
+    mlir::DLTIDialect::kDataLayoutProgramMemorySpaceKey;
+constexpr const StringLiteral
+    mlir::DLTIDialect::kDataLayoutGlobalMemorySpaceKey;
+
 constexpr const StringLiteral mlir::DLTIDialect::kDataLayoutStackAlignmentKey;
 
 namespace mlir {
@@ -282,6 +287,17 @@ DataLayoutSpecAttr::getAllocaMemorySpaceIdentifier(MLIRContext *context) const {
       DLTIDialect::kDataLayoutAllocaMemorySpaceKey);
 }
 
+StringAttr DataLayoutSpecAttr::getProgramMemorySpaceIdentifier(
+    MLIRContext *context) const {
+  return Builder(context).getStringAttr(
+      DLTIDialect::kDataLayoutProgramMemorySpaceKey);
+}
+
+StringAttr
+DataLayoutSpecAttr::getGlobalMemorySpaceIdentifier(MLIRContext *context) const {
+  return Builder(context).getStringAttr(
+      DLTIDialect::kDataLayoutGlobalMemorySpaceKey);
+}
 StringAttr
 DataLayoutSpecAttr::getStackAlignmentIdentifier(MLIRContext *context) const {
   return Builder(context).getStringAttr(
@@ -345,6 +361,8 @@ class TargetDataLayoutInterface : public DataLayoutDialectInterface {
                             << DLTIDialect::kDataLayoutEndiannessLittle << "'";
     }
     if (entryName == DLTIDialect::kDataLayoutAllocaMemorySpaceKey ||
+        entryName == DLTIDialect::kDataLayoutProgramMemorySpaceKey ||
+        entryName == DLTIDialect::kDataLayoutGlobalMemorySpaceKey ||
         entryName == DLTIDialect::kDataLayoutStackAlignmentKey)
       return success();
     return emitError(loc) << "unknown data layout entry name: " << entryName;
diff --git a/mlir/lib/Interfaces/DataLayoutInterfaces.cpp b/mlir/lib/Interfaces/DataLayoutInterfaces.cpp
index 1178417fd2a6ca..65c41f44192a90 100644
--- a/mlir/lib/Interfaces/DataLayoutInterfaces.cpp
+++ b/mlir/lib/Interfaces/DataLayoutInterfaces.cpp
@@ -230,6 +230,30 @@ mlir::detail::getDefaultAllocaMemorySpace(DataLayoutEntryInterface entry) {
   return entry.getValue();
 }
 
+// Returns the memory space used for the program memory space.  if
+// specified in the given entry. If the entry is empty the default
+// memory space represented by an empty attribute is returned.
+Attribute
+mlir::detail::getDefaultProgramMemorySpace(DataLayoutEntryInterface entry) {
+  if (entry == DataLayoutEntryInterface()) {
+    return Attribute();
+  }
+
+  return entry.getValue();
+}
+
+// Returns the memory space used for global the global memory space. if
+// specified in the given entry. If the entry is empty the default memory
+// space represented by an empty attribute is returned.
+Attribute
+mlir::detail::getDefaultGlobalMemorySpace(DataLayoutEntryInterface entry) {
+  if (entry == DataLayoutEntryInterface()) {
+    return Attribute();
+  }
+
+  return entry.getValue();
+}
+
 // Returns the stack alignment if specified in the given entry. If the entry is
 // empty the default alignment zero is returned.
 uint64_t
@@ -382,7 +406,8 @@ mlir::DataLayout::DataLayout() : DataLayout(ModuleOp()) {}
 
 mlir::DataLayout::DataLayout(DataLayoutOpInterface op)
     : originalLayout(getCombinedDataLayout(op)), scope(op),
-      allocaMemorySpace(std::nullopt), stackAlignment(std::nullopt) {
+      allocaMemorySpace(std::nullopt), programMemorySpace(std::nullopt),
+      globalMemorySpace(std::nullopt), stackAlignment(std::nullopt) {
 #if LLVM_ENABLE_ABI_BREAKING_CHECKS
   checkMissingLayout(originalLayout, op);
   collectParentLayouts(op, layoutStack);
@@ -391,7 +416,8 @@ mlir::DataLayout::DataLayout(DataLayoutOpInterface op)
 
 mlir::DataLayout::DataLayout(ModuleOp op)
     : originalLayout(getCombinedDataLayout(op)), scope(op),
-      allocaMemorySpace(std::nullopt), stackAlignment(std::nullopt) {
+      allocaMemorySpace(std::nullopt), programMemorySpace(std::nullopt),
+      globalMemorySpace(std::nullopt), stackAlignment(std::nullopt) {
 #if LLVM_ENABLE_ABI_BREAKING_CHECKS
   checkMissingLayout(originalLayout, op);
   collectParentLayouts(op, layoutStack);
@@ -510,6 +536,38 @@ mlir::Attribute mlir::DataLayout::getAllocaMemorySpace() const {
   return *allocaMemorySpace;
 }
 
+mlir::Attribute mlir::DataLayout::getProgramMemorySpace() const {
+  checkValid();
+  if (programMemorySpace)
+    return *programMemorySpace;
+  DataLayoutEntryInterface entry;
+  if (originalLayout)
+    entry = originalLayout.getSpecForIdentifier(
+        originalLayout.getProgramMemorySpaceIdentifier(
+            originalLayout.getContext()));
+  if (auto iface = dyn_cast_or_null<DataLayoutOpInterface>(scope))
+    programMemorySpace = iface.getProgramMemorySpace(entry);
+  else
+    programMemorySpace = detail::getDefaultProgramMemorySpace(entry);
+  return *programMemorySpace;
+}
+
+mlir::Attribute mlir::DataLayout::getGlobalMemorySpace() const {
+  checkValid();
+  if (globalMemorySpace)
+    return *globalMemorySpace;
+  DataLayoutEntryInterface entry;
+  if (originalLayout)
+    entry = originalLayout.getSpecForIdentifier(
+        originalLayout.getGlobalMemorySpaceIdentifier(
+            originalLayout.getContext()));
+  if (auto iface = dyn_cast_or_null<DataLayoutOpInterface>(scope))
+    globalMemorySpace = iface.getGlobalMemorySpace(entry);
+  else
+    globalMemorySpace = detail::getDefaultGlobalMemorySpace(entry);
+  return *globalMemorySpace;
+}
+
 uint64_t mlir::DataLayout::getStackAlignment() const {
   checkValid();
   if (stackAlignment)
diff --git a/mlir/lib/Target/LLVMIR/DataLayoutImporter.cpp b/mlir/lib/Target/LLVMIR/DataLayoutImporter.cpp
index 95f3cc074b1ddf..392f552f480cc8 100644
--- a/mlir/lib/Target/LLVMIR/DataLayoutImporter.cpp
+++ b/mlir/lib/Target/LLVMIR/DataLayoutImporter.cpp
@@ -164,9 +164,9 @@ DataLayoutImporter::tryToEmplaceEndiannessEntry(StringRef endianness,
 }
 
 LogicalResult
-DataLayoutImporter::tryToEmplaceAllocaAddrSpaceEntry(StringRef token) {
-  auto key =
-      StringAttr::get(context, DLTIDialect::kDataLayoutAllocaMemorySpaceKey);
+DataLayoutImporter::tryToEmplaceAddrSpaceEntry(StringRef token,
+                                               llvm::StringLiteral spaceKey) {
+  auto key = StringAttr::get(context, spaceKey);
   if (keyEntries.count(key))
     return success();
 
@@ -247,9 +247,24 @@ void DataLayoutImporter::translateDataLayout(
         return;
       continue;
     }
+    // Parse the program address space.
+    if (*prefix == "P") {
+      if (failed(tryToEmplaceAddrSpaceEntry(
+              token, DLTIDialect::kDataLayoutProgramMemorySpaceKey)))
+        return;
+      continue;
+    }
+    // Parse the global address space.
+    if (*prefix == "G") {
+      if (failed(tryToEmplaceAddrSpaceEntry(
+              token, DLTIDialect::kDataLayoutGlobalMemorySpaceKey)))
+        return;
+      continue;
+    }
     // Parse the alloca address space.
     if (*prefix == "A") {
-      if (failed(tryToEmplaceAllocaAddrSpaceEntry(token)))
+      if (failed(tryToEmplaceAddrSpaceEntry(
+              token, DLTIDialect::kDataLayoutAllocaMemorySpaceKey)))
         return;
       continue;
     }
diff --git a/mlir/lib/Target/LLVMIR/DataLayoutImporter.h b/mlir/lib/Target/LLVMIR/DataLayoutImporter.h
index 15f0f7ddf07059..59b60acd24be20 100644
--- a/mlir/lib/Target/LLVMIR/DataLayoutImporter.h
+++ b/mlir/lib/Target/LLVMIR/DataLayoutImporter.h
@@ -97,7 +97,8 @@ class DataLayoutImporter {
                                             StringRef token);
 
   /// Adds an alloca address space entry if there is none yet.
-  LogicalResult tryToEmplaceAllocaAddrSpaceEntry(StringRef token);
+  LogicalResult tryToEmplaceAddrSpaceEntry(StringRef token,
+                                           llvm::StringLiteral spaceKey);
 
   /// Adds a stack alignment entry if there is none yet.
   LogicalResult tryToEmplaceStackAlignmentEntry(StringRef token);
diff --git a/mlir/lib/Target/LLVMIR/ModuleTranslation.cpp b/mlir/lib/Target/LLVMIR/ModuleTranslation.cpp
index 1722d74c08b628..ce46a194ea7d9f 100644
--- a/mlir/lib/Target/LLVMIR/ModuleTranslation.cpp
+++ b/mlir/lib/Target/LLVMIR/ModuleTranslation.cpp
@@ -190,6 +190,26 @@ translateDataLayout(DataLayoutSpecInterface attribute,
       layoutStream.flush();
       continue;
     }
+    if (key.getValue() == DLTIDialect::kDataLayoutProgramMemorySpaceKey) {
+      auto value = cast<IntegerAttr>(entry.getValue());
+      uint64_t space = value.getValue().getZExtValue();
+      // Skip the default address space.
+      if (space == 0)
+        continue;
+      layoutStream << "-P" << space;
+      layoutStream.flush();
+      continue;
+    }
+    if (key.getValue() == DLTIDialect::kDataLayoutGlobalMemorySpaceKey) {
+      auto value = cast<IntegerAttr>(entry.getValue());
+      uint64_t space = value.getValue().getZExtValue();
+      // Skip the default address space.
+      if (space == 0)
+        continue;
+      layoutStream << "-G" << space;
+      layoutStream.flush();
+      continue;
+    }
     if (key.getValue() == DLTIDialect::kDataLayoutAllocaMemorySpaceKey) {
       auto value = cast<IntegerAttr>(entry.getValue());
       uint64_t space = value.getValue().getZExtValue();
diff --git a/mlir/test/Dialect/LLVMIR/layout.mlir b/mlir/test/Dialect/LLVMIR/layout.mlir
index 99d61785384487..2868e1740f861c 100644
--- a/mlir/test/Dialect/LLVMIR/layout.mlir
+++ b/mlir/test/Dialect/LLVMIR/layout.mlir
@@ -6,21 +6,27 @@ module {
     // CHECK: alignment = 8
     // CHECK: alloca_memory_space = 0
     // CHECK: bitsize = 64
+    // CHECK: global_memory_space = 0
     // CHECK: preferred = 8
+    // CHECK: program_memory_space = 0
     // CHECK: size = 8
     // CHECK: stack_alignment = 0
     "test.data_layout_query"() : () -> !llvm.ptr
     // CHECK: alignment = 8
     // CHECK: alloca_memory_space = 0
     // CHECK: bitsize = 64
+    // CHECK: global_memory_space = 0
     // CHECK: preferred = 8
+    // CHECK: program_memory_space = 0
     // CHECK: size = 8
     // CHECK: stack_alignment = 0
     "test.data_layout_query"() : () -> !llvm.ptr<3>
     // CHECK: alignment = 8
     // CHECK: alloca_memory_space = 0
     // CHECK: bitsize = 64
+    // CHECK: global_memory_space = 0
     // CHECK: preferred = 8
+    // CHECK: program_memory_space = 0
     // CHECK: size = 8
     // CHECK: stack_alignment = 0
     "test.data_layout_query"() : () -> !llvm.ptr<5>
@@ -35,6 +41,8 @@ module attributes { dlti.dl_spec = #dlti.dl_spec<
   #dlti.dl_entry<!llvm.ptr<5>, dense<[64, 64, 64]> : vector<3xi64>>,
   #dlti.dl_entry<!llvm.ptr<4>, dense<[32, 64, 64]> : vector<3xi64>>,
   #dlti.dl_entry<"dlti.alloca_memory_space", 5 : ui64>,
+  #dlti.dl_entry<"dlti.global_memory_space", 2 : ui64>,
+  #dlti.dl_entry<"dlti.program_memory_space", 3 : ui64>,
   #dlti.dl_entry<"dlti.stack_alignment", 128 : i64>
 >} {
   // CHECK: @spec
@@ -42,35 +50,45 @@ module attributes { dlti.dl_spec = #dlti.dl_spec<
     // CHECK: alignment = 4
     // CHECK: alloca_memory_space = 5
     // CHECK: bitsize = 32
+    // CHECK: global_memory_space = 2
     // CHECK: preferred = 8
+    // CHECK: program_memory_space = 3
     // CHECK: size = 4
     // CHECK: stack_alignment = 128
     "test.data_layout_query"() : () -> !llvm.ptr
     // CHECK: alignment = 4
     // CHECK: alloca_memory_space = 5
     // CHECK: bitsize = 32
+    // CHECK: global_memory_space = 2
     // CHECK: preferred = 8
+    // CHECK: program_memory_space = 3
     // CHECK: size = 4
     // CHECK: stack_alignment = 128
     "test.data_layout_query"() : () -> !llvm.ptr<3>
     // CHECK: alignment = 8
     // CHECK: alloca_memory_space = 5
     // CHECK: bitsize = 64
+    // CHECK: global_memory_space = 2
     // CHECK: preferred = 8
+    // CHECK: program_memory_space = 3
     // CHECK: size = 8
     // CHECK: stack_alignment = 128
     "test.data_layout_query"() : () -> !llvm.ptr<5>
     // CHECK: alignment = 4
     // CHECK: alloca_memory_space = 5
     // CHECK: bitsize = 32
+    // CHECK: global_memory_space = 2
     // CHECK: preferred = 8
+    // CHECK: program_memory_space = 3
     // CHECK: size = 4
     // CHECK: stack_alignment = 128
     "test.data_layout_query"() : () -> !llvm.ptr<3>
     // CHECK: alignment = 8
     // CHECK: alloca_memory_space = 5
     // CHECK: bitsize = 32
+    // CHECK: global_memory_space = 2
     // CHECK: preferred = 8
+    // CHECK: program_memory_space = 3
     // CHECK: size = 4
     // CHECK: stack_alignment = 128
     "test.data_layout_query"() : () -> !llvm.ptr<4>
diff --git a/mlir/test/lib/Dialect/DLTI/TestDataLayoutQuery.cpp b/mlir/test/lib/Dialect/DLTI/TestDataLayoutQuery.cpp
index 7e3d3f6dc3f0e6..740562e7783024 100644
--- a/mlir/test/lib/Dialect/DLTI/TestDataLayoutQuery.cpp
+++ b/mlir/test/lib/Dialect/DLTI/TestDataLayoutQuery.cpp
@@ -41,6 +41,8 @@ struct TestDataLayoutQuery
       unsigned alignment = layout.getTypeABIAlignment(op.getType());
       unsigned preferred = layout.getTypePreferredAlignment(op.getType());
       Attribute allocaMemorySpace = layout.getAllocaMemorySpace();
+      Attribute programMemorySpace = layout.getProgramMemorySpace();
+      Attribute globalMemorySpace = layout.getGlobalMemorySpace();
       unsigned stackAlignment = layout.getStackAlignment();
       op->setAttrs(
           {builder.getNamedAttr("size", builder.getIndexAttr(size)),
@@ -51,6 +53,14 @@ struct TestDataLayoutQuery
                                 allocaMemorySpace == Attribute()
                                     ? builder.getUI32IntegerAttr(0)
                                     : allocaMemorySpace),
+           builder.getNamedAttr("program_memory_space",
+                                programMemorySpace == Attribute()
+                                    ? builder.getUI32IntegerAttr(0)
+                                    : programMemorySpace),
+           builder.getNamedAttr("global_memory_space",
+                                glo...
[truncated]

@llvmbot
Copy link
Collaborator

llvmbot commented Jan 8, 2024

@llvm/pr-subscribers-mlir

Author: None (agozillon)

Changes

This patch is based on a previous PR https://reviews.llvm.org/D144657 that added alloca address space handling to MLIR's DataLayout and DLTI interface. This patch aims to add identical features to import and access the global and program memory space through MLIR's DataLayout/DLTI system.


Patch is 24.02 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/77367.diff

12 Files Affected:

  • (modified) mlir/include/mlir/Dialect/DLTI/DLTI.h (+6)
  • (modified) mlir/include/mlir/Dialect/DLTI/DLTIBase.td (+6)
  • (modified) mlir/include/mlir/Interfaces/DataLayoutInterfaces.h (+17-1)
  • (modified) mlir/include/mlir/Interfaces/DataLayoutInterfaces.td (+36)
  • (modified) mlir/lib/Dialect/DLTI/DLTI.cpp (+18)
  • (modified) mlir/lib/Interfaces/DataLayoutInterfaces.cpp (+60-2)
  • (modified) mlir/lib/Target/LLVMIR/DataLayoutImporter.cpp (+19-4)
  • (modified) mlir/lib/Target/LLVMIR/DataLayoutImporter.h (+2-1)
  • (modified) mlir/lib/Target/LLVMIR/ModuleTranslation.cpp (+20)
  • (modified) mlir/test/Dialect/LLVMIR/layout.mlir (+18)
  • (modified) mlir/test/lib/Dialect/DLTI/TestDataLayoutQuery.cpp (+10)
  • (modified) mlir/unittests/Interfaces/DataLayoutInterfacesTest.cpp (+38)
diff --git a/mlir/include/mlir/Dialect/DLTI/DLTI.h b/mlir/include/mlir/Dialect/DLTI/DLTI.h
index b9a8763eb44989..bf23aa2d48a801 100644
--- a/mlir/include/mlir/Dialect/DLTI/DLTI.h
+++ b/mlir/include/mlir/Dialect/DLTI/DLTI.h
@@ -103,6 +103,12 @@ class DataLayoutSpecAttr
   /// Returns the alloca memory space identifier.
   StringAttr getAllocaMemorySpaceIdentifier(MLIRContext *context) const;
 
+  /// Returns the program memory space identifier.
+  StringAttr getProgramMemorySpaceIdentifier(MLIRContext *context) const;
+
+  /// Returns the global memory space identifier.
+  StringAttr getGlobalMemorySpaceIdentifier(MLIRContext *context) const;
+
   /// Returns the stack alignment identifier.
   StringAttr getStackAlignmentIdentifier(MLIRContext *context) const;
 
diff --git a/mlir/include/mlir/Dialect/DLTI/DLTIBase.td b/mlir/include/mlir/Dialect/DLTI/DLTIBase.td
index 7d519f4efcd224..3572a99fad8743 100644
--- a/mlir/include/mlir/Dialect/DLTI/DLTIBase.td
+++ b/mlir/include/mlir/Dialect/DLTI/DLTIBase.td
@@ -39,6 +39,12 @@ def DLTI_Dialect : Dialect {
 
     constexpr const static ::llvm::StringLiteral
     kDataLayoutAllocaMemorySpaceKey = "dlti.alloca_memory_space";
+    
+    constexpr const static ::llvm::StringLiteral
+    kDataLayoutProgramMemorySpaceKey = "dlti.program_memory_space";
+
+    constexpr const static ::llvm::StringLiteral
+    kDataLayoutGlobalMemorySpaceKey = "dlti.global_memory_space";
 
     constexpr const static ::llvm::StringLiteral
     kDataLayoutStackAlignmentKey = "dlti.stack_alignment";
diff --git a/mlir/include/mlir/Interfaces/DataLayoutInterfaces.h b/mlir/include/mlir/Interfaces/DataLayoutInterfaces.h
index f4b9b95fb89f89..4a21f76dfc5d1c 100644
--- a/mlir/include/mlir/Interfaces/DataLayoutInterfaces.h
+++ b/mlir/include/mlir/Interfaces/DataLayoutInterfaces.h
@@ -61,6 +61,14 @@ getDefaultPreferredAlignment(Type type, const DataLayout &dataLayout,
 /// DataLayoutInterface if specified, otherwise returns the default.
 Attribute getDefaultAllocaMemorySpace(DataLayoutEntryInterface entry);
 
+/// Default handler for program memory space request. Dispatches to the
+/// DataLayoutInterface if specified, otherwise returns the default.
+Attribute getDefaultProgramMemorySpace(DataLayoutEntryInterface entry);
+
+/// Default handler for global memory space request. Dispatches to the
+/// DataLayoutInterface if specified, otherwise returns the default.
+Attribute getDefaultGlobalMemorySpace(DataLayoutEntryInterface entry);
+
 /// Default handler for the stack alignment request. Dispatches to the
 /// DataLayoutInterface if specified, otherwise returns the default.
 uint64_t getDefaultStackAlignment(DataLayoutEntryInterface entry);
@@ -175,6 +183,12 @@ class DataLayout {
   /// Returns the memory space used for AllocaOps.
   Attribute getAllocaMemorySpace() const;
 
+  /// Returns the memory space used for program memory operations.
+  Attribute getProgramMemorySpace() const;
+
+  /// Returns the memory space used for global operations.
+  Attribute getGlobalMemorySpace() const;
+
   /// Returns the natural alignment of the stack in bits. Alignment promotion of
   /// stack variables should be limited to the natural stack alignment to
   /// prevent dynamic stack alignment. Returns zero if the stack alignment is
@@ -203,8 +217,10 @@ class DataLayout {
   mutable DenseMap<Type, uint64_t> abiAlignments;
   mutable DenseMap<Type, uint64_t> preferredAlignments;
 
-  /// Cache for alloca memory space.
+  /// Cache for alloca, global, and program memory spaces.
   mutable std::optional<Attribute> allocaMemorySpace;
+  mutable std::optional<Attribute> programMemorySpace;
+  mutable std::optional<Attribute> globalMemorySpace;
 
   /// Cache for stack alignment.
   mutable std::optional<uint64_t> stackAlignment;
diff --git a/mlir/include/mlir/Interfaces/DataLayoutInterfaces.td b/mlir/include/mlir/Interfaces/DataLayoutInterfaces.td
index 2f60a16baf50bc..a8def967fffcfa 100644
--- a/mlir/include/mlir/Interfaces/DataLayoutInterfaces.td
+++ b/mlir/include/mlir/Interfaces/DataLayoutInterfaces.td
@@ -112,6 +112,18 @@ def DataLayoutSpecInterface : AttrInterface<"DataLayoutSpecInterface"> {
       /*methodName=*/"getAllocaMemorySpaceIdentifier",
       /*args=*/(ins "::mlir::MLIRContext *":$context)
     >,
+    InterfaceMethod<
+      /*description=*/"Returns the program memory space identifier.",
+      /*retTy=*/"::mlir::StringAttr",
+      /*methodName=*/"getProgramMemorySpaceIdentifier",
+      /*args=*/(ins "::mlir::MLIRContext *":$context)
+    >,
+    InterfaceMethod<
+      /*description=*/"Returns the global memory space identifier.",
+      /*retTy=*/"::mlir::StringAttr",
+      /*methodName=*/"getGlobalMemorySpaceIdentifier",
+      /*args=*/(ins "::mlir::MLIRContext *":$context)
+    >,
     InterfaceMethod<
       /*description=*/"Returns the stack alignment identifier.",
       /*retTy=*/"::mlir::StringAttr",
@@ -280,6 +292,30 @@ def DataLayoutOpInterface : OpInterface<"DataLayoutOpInterface"> {
         return ::mlir::detail::getDefaultAllocaMemorySpace(entry);
       }]
     >,
+    StaticInterfaceMethod<
+      /*description=*/"Returns the memory space used by the ABI computed "
+                      "using the relevant entries. The data layout object "
+                      "can be used for recursive queries.",
+      /*retTy=*/"::mlir::Attribute",
+      /*methodName=*/"getProgramMemorySpace",
+      /*args=*/(ins "::mlir::DataLayoutEntryInterface":$entry),
+      /*methodBody=*/"",
+      /*defaultImplementation=*/[{
+        return ::mlir::detail::getDefaultProgramMemorySpace(entry);
+      }]
+    >,
+    StaticInterfaceMethod<
+      /*description=*/"Returns the memory space used by the ABI computed "
+                      "using the relevant entries. The data layout object "
+                      "can be used for recursive queries.",
+      /*retTy=*/"::mlir::Attribute",
+      /*methodName=*/"getGlobalMemorySpace",
+      /*args=*/(ins "::mlir::DataLayoutEntryInterface":$entry),
+      /*methodBody=*/"",
+      /*defaultImplementation=*/[{
+        return ::mlir::detail::getDefaultGlobalMemorySpace(entry);
+      }]
+    >,
     StaticInterfaceMethod<
       /*description=*/"Returns the natural stack alignment in bits computed "
                       "using the relevant entries. The data layout object "
diff --git a/mlir/lib/Dialect/DLTI/DLTI.cpp b/mlir/lib/Dialect/DLTI/DLTI.cpp
index aba9e7db0a2fef..daef2349430d0b 100644
--- a/mlir/lib/Dialect/DLTI/DLTI.cpp
+++ b/mlir/lib/Dialect/DLTI/DLTI.cpp
@@ -108,6 +108,11 @@ void DataLayoutEntryAttr::print(AsmPrinter &os) const {
 constexpr const StringLiteral mlir::DataLayoutSpecAttr::kAttrKeyword;
 constexpr const StringLiteral
     mlir::DLTIDialect::kDataLayoutAllocaMemorySpaceKey;
+constexpr const StringLiteral
+    mlir::DLTIDialect::kDataLayoutProgramMemorySpaceKey;
+constexpr const StringLiteral
+    mlir::DLTIDialect::kDataLayoutGlobalMemorySpaceKey;
+
 constexpr const StringLiteral mlir::DLTIDialect::kDataLayoutStackAlignmentKey;
 
 namespace mlir {
@@ -282,6 +287,17 @@ DataLayoutSpecAttr::getAllocaMemorySpaceIdentifier(MLIRContext *context) const {
       DLTIDialect::kDataLayoutAllocaMemorySpaceKey);
 }
 
+StringAttr DataLayoutSpecAttr::getProgramMemorySpaceIdentifier(
+    MLIRContext *context) const {
+  return Builder(context).getStringAttr(
+      DLTIDialect::kDataLayoutProgramMemorySpaceKey);
+}
+
+StringAttr
+DataLayoutSpecAttr::getGlobalMemorySpaceIdentifier(MLIRContext *context) const {
+  return Builder(context).getStringAttr(
+      DLTIDialect::kDataLayoutGlobalMemorySpaceKey);
+}
 StringAttr
 DataLayoutSpecAttr::getStackAlignmentIdentifier(MLIRContext *context) const {
   return Builder(context).getStringAttr(
@@ -345,6 +361,8 @@ class TargetDataLayoutInterface : public DataLayoutDialectInterface {
                             << DLTIDialect::kDataLayoutEndiannessLittle << "'";
     }
     if (entryName == DLTIDialect::kDataLayoutAllocaMemorySpaceKey ||
+        entryName == DLTIDialect::kDataLayoutProgramMemorySpaceKey ||
+        entryName == DLTIDialect::kDataLayoutGlobalMemorySpaceKey ||
         entryName == DLTIDialect::kDataLayoutStackAlignmentKey)
       return success();
     return emitError(loc) << "unknown data layout entry name: " << entryName;
diff --git a/mlir/lib/Interfaces/DataLayoutInterfaces.cpp b/mlir/lib/Interfaces/DataLayoutInterfaces.cpp
index 1178417fd2a6ca..65c41f44192a90 100644
--- a/mlir/lib/Interfaces/DataLayoutInterfaces.cpp
+++ b/mlir/lib/Interfaces/DataLayoutInterfaces.cpp
@@ -230,6 +230,30 @@ mlir::detail::getDefaultAllocaMemorySpace(DataLayoutEntryInterface entry) {
   return entry.getValue();
 }
 
+// Returns the memory space used for the program memory space.  if
+// specified in the given entry. If the entry is empty the default
+// memory space represented by an empty attribute is returned.
+Attribute
+mlir::detail::getDefaultProgramMemorySpace(DataLayoutEntryInterface entry) {
+  if (entry == DataLayoutEntryInterface()) {
+    return Attribute();
+  }
+
+  return entry.getValue();
+}
+
+// Returns the memory space used for global the global memory space. if
+// specified in the given entry. If the entry is empty the default memory
+// space represented by an empty attribute is returned.
+Attribute
+mlir::detail::getDefaultGlobalMemorySpace(DataLayoutEntryInterface entry) {
+  if (entry == DataLayoutEntryInterface()) {
+    return Attribute();
+  }
+
+  return entry.getValue();
+}
+
 // Returns the stack alignment if specified in the given entry. If the entry is
 // empty the default alignment zero is returned.
 uint64_t
@@ -382,7 +406,8 @@ mlir::DataLayout::DataLayout() : DataLayout(ModuleOp()) {}
 
 mlir::DataLayout::DataLayout(DataLayoutOpInterface op)
     : originalLayout(getCombinedDataLayout(op)), scope(op),
-      allocaMemorySpace(std::nullopt), stackAlignment(std::nullopt) {
+      allocaMemorySpace(std::nullopt), programMemorySpace(std::nullopt),
+      globalMemorySpace(std::nullopt), stackAlignment(std::nullopt) {
 #if LLVM_ENABLE_ABI_BREAKING_CHECKS
   checkMissingLayout(originalLayout, op);
   collectParentLayouts(op, layoutStack);
@@ -391,7 +416,8 @@ mlir::DataLayout::DataLayout(DataLayoutOpInterface op)
 
 mlir::DataLayout::DataLayout(ModuleOp op)
     : originalLayout(getCombinedDataLayout(op)), scope(op),
-      allocaMemorySpace(std::nullopt), stackAlignment(std::nullopt) {
+      allocaMemorySpace(std::nullopt), programMemorySpace(std::nullopt),
+      globalMemorySpace(std::nullopt), stackAlignment(std::nullopt) {
 #if LLVM_ENABLE_ABI_BREAKING_CHECKS
   checkMissingLayout(originalLayout, op);
   collectParentLayouts(op, layoutStack);
@@ -510,6 +536,38 @@ mlir::Attribute mlir::DataLayout::getAllocaMemorySpace() const {
   return *allocaMemorySpace;
 }
 
+mlir::Attribute mlir::DataLayout::getProgramMemorySpace() const {
+  checkValid();
+  if (programMemorySpace)
+    return *programMemorySpace;
+  DataLayoutEntryInterface entry;
+  if (originalLayout)
+    entry = originalLayout.getSpecForIdentifier(
+        originalLayout.getProgramMemorySpaceIdentifier(
+            originalLayout.getContext()));
+  if (auto iface = dyn_cast_or_null<DataLayoutOpInterface>(scope))
+    programMemorySpace = iface.getProgramMemorySpace(entry);
+  else
+    programMemorySpace = detail::getDefaultProgramMemorySpace(entry);
+  return *programMemorySpace;
+}
+
+mlir::Attribute mlir::DataLayout::getGlobalMemorySpace() const {
+  checkValid();
+  if (globalMemorySpace)
+    return *globalMemorySpace;
+  DataLayoutEntryInterface entry;
+  if (originalLayout)
+    entry = originalLayout.getSpecForIdentifier(
+        originalLayout.getGlobalMemorySpaceIdentifier(
+            originalLayout.getContext()));
+  if (auto iface = dyn_cast_or_null<DataLayoutOpInterface>(scope))
+    globalMemorySpace = iface.getGlobalMemorySpace(entry);
+  else
+    globalMemorySpace = detail::getDefaultGlobalMemorySpace(entry);
+  return *globalMemorySpace;
+}
+
 uint64_t mlir::DataLayout::getStackAlignment() const {
   checkValid();
   if (stackAlignment)
diff --git a/mlir/lib/Target/LLVMIR/DataLayoutImporter.cpp b/mlir/lib/Target/LLVMIR/DataLayoutImporter.cpp
index 95f3cc074b1ddf..392f552f480cc8 100644
--- a/mlir/lib/Target/LLVMIR/DataLayoutImporter.cpp
+++ b/mlir/lib/Target/LLVMIR/DataLayoutImporter.cpp
@@ -164,9 +164,9 @@ DataLayoutImporter::tryToEmplaceEndiannessEntry(StringRef endianness,
 }
 
 LogicalResult
-DataLayoutImporter::tryToEmplaceAllocaAddrSpaceEntry(StringRef token) {
-  auto key =
-      StringAttr::get(context, DLTIDialect::kDataLayoutAllocaMemorySpaceKey);
+DataLayoutImporter::tryToEmplaceAddrSpaceEntry(StringRef token,
+                                               llvm::StringLiteral spaceKey) {
+  auto key = StringAttr::get(context, spaceKey);
   if (keyEntries.count(key))
     return success();
 
@@ -247,9 +247,24 @@ void DataLayoutImporter::translateDataLayout(
         return;
       continue;
     }
+    // Parse the program address space.
+    if (*prefix == "P") {
+      if (failed(tryToEmplaceAddrSpaceEntry(
+              token, DLTIDialect::kDataLayoutProgramMemorySpaceKey)))
+        return;
+      continue;
+    }
+    // Parse the global address space.
+    if (*prefix == "G") {
+      if (failed(tryToEmplaceAddrSpaceEntry(
+              token, DLTIDialect::kDataLayoutGlobalMemorySpaceKey)))
+        return;
+      continue;
+    }
     // Parse the alloca address space.
     if (*prefix == "A") {
-      if (failed(tryToEmplaceAllocaAddrSpaceEntry(token)))
+      if (failed(tryToEmplaceAddrSpaceEntry(
+              token, DLTIDialect::kDataLayoutAllocaMemorySpaceKey)))
         return;
       continue;
     }
diff --git a/mlir/lib/Target/LLVMIR/DataLayoutImporter.h b/mlir/lib/Target/LLVMIR/DataLayoutImporter.h
index 15f0f7ddf07059..59b60acd24be20 100644
--- a/mlir/lib/Target/LLVMIR/DataLayoutImporter.h
+++ b/mlir/lib/Target/LLVMIR/DataLayoutImporter.h
@@ -97,7 +97,8 @@ class DataLayoutImporter {
                                             StringRef token);
 
   /// Adds an alloca address space entry if there is none yet.
-  LogicalResult tryToEmplaceAllocaAddrSpaceEntry(StringRef token);
+  LogicalResult tryToEmplaceAddrSpaceEntry(StringRef token,
+                                           llvm::StringLiteral spaceKey);
 
   /// Adds a stack alignment entry if there is none yet.
   LogicalResult tryToEmplaceStackAlignmentEntry(StringRef token);
diff --git a/mlir/lib/Target/LLVMIR/ModuleTranslation.cpp b/mlir/lib/Target/LLVMIR/ModuleTranslation.cpp
index 1722d74c08b628..ce46a194ea7d9f 100644
--- a/mlir/lib/Target/LLVMIR/ModuleTranslation.cpp
+++ b/mlir/lib/Target/LLVMIR/ModuleTranslation.cpp
@@ -190,6 +190,26 @@ translateDataLayout(DataLayoutSpecInterface attribute,
       layoutStream.flush();
       continue;
     }
+    if (key.getValue() == DLTIDialect::kDataLayoutProgramMemorySpaceKey) {
+      auto value = cast<IntegerAttr>(entry.getValue());
+      uint64_t space = value.getValue().getZExtValue();
+      // Skip the default address space.
+      if (space == 0)
+        continue;
+      layoutStream << "-P" << space;
+      layoutStream.flush();
+      continue;
+    }
+    if (key.getValue() == DLTIDialect::kDataLayoutGlobalMemorySpaceKey) {
+      auto value = cast<IntegerAttr>(entry.getValue());
+      uint64_t space = value.getValue().getZExtValue();
+      // Skip the default address space.
+      if (space == 0)
+        continue;
+      layoutStream << "-G" << space;
+      layoutStream.flush();
+      continue;
+    }
     if (key.getValue() == DLTIDialect::kDataLayoutAllocaMemorySpaceKey) {
       auto value = cast<IntegerAttr>(entry.getValue());
       uint64_t space = value.getValue().getZExtValue();
diff --git a/mlir/test/Dialect/LLVMIR/layout.mlir b/mlir/test/Dialect/LLVMIR/layout.mlir
index 99d61785384487..2868e1740f861c 100644
--- a/mlir/test/Dialect/LLVMIR/layout.mlir
+++ b/mlir/test/Dialect/LLVMIR/layout.mlir
@@ -6,21 +6,27 @@ module {
     // CHECK: alignment = 8
     // CHECK: alloca_memory_space = 0
     // CHECK: bitsize = 64
+    // CHECK: global_memory_space = 0
     // CHECK: preferred = 8
+    // CHECK: program_memory_space = 0
     // CHECK: size = 8
     // CHECK: stack_alignment = 0
     "test.data_layout_query"() : () -> !llvm.ptr
     // CHECK: alignment = 8
     // CHECK: alloca_memory_space = 0
     // CHECK: bitsize = 64
+    // CHECK: global_memory_space = 0
     // CHECK: preferred = 8
+    // CHECK: program_memory_space = 0
     // CHECK: size = 8
     // CHECK: stack_alignment = 0
     "test.data_layout_query"() : () -> !llvm.ptr<3>
     // CHECK: alignment = 8
     // CHECK: alloca_memory_space = 0
     // CHECK: bitsize = 64
+    // CHECK: global_memory_space = 0
     // CHECK: preferred = 8
+    // CHECK: program_memory_space = 0
     // CHECK: size = 8
     // CHECK: stack_alignment = 0
     "test.data_layout_query"() : () -> !llvm.ptr<5>
@@ -35,6 +41,8 @@ module attributes { dlti.dl_spec = #dlti.dl_spec<
   #dlti.dl_entry<!llvm.ptr<5>, dense<[64, 64, 64]> : vector<3xi64>>,
   #dlti.dl_entry<!llvm.ptr<4>, dense<[32, 64, 64]> : vector<3xi64>>,
   #dlti.dl_entry<"dlti.alloca_memory_space", 5 : ui64>,
+  #dlti.dl_entry<"dlti.global_memory_space", 2 : ui64>,
+  #dlti.dl_entry<"dlti.program_memory_space", 3 : ui64>,
   #dlti.dl_entry<"dlti.stack_alignment", 128 : i64>
 >} {
   // CHECK: @spec
@@ -42,35 +50,45 @@ module attributes { dlti.dl_spec = #dlti.dl_spec<
     // CHECK: alignment = 4
     // CHECK: alloca_memory_space = 5
     // CHECK: bitsize = 32
+    // CHECK: global_memory_space = 2
     // CHECK: preferred = 8
+    // CHECK: program_memory_space = 3
     // CHECK: size = 4
     // CHECK: stack_alignment = 128
     "test.data_layout_query"() : () -> !llvm.ptr
     // CHECK: alignment = 4
     // CHECK: alloca_memory_space = 5
     // CHECK: bitsize = 32
+    // CHECK: global_memory_space = 2
     // CHECK: preferred = 8
+    // CHECK: program_memory_space = 3
     // CHECK: size = 4
     // CHECK: stack_alignment = 128
     "test.data_layout_query"() : () -> !llvm.ptr<3>
     // CHECK: alignment = 8
     // CHECK: alloca_memory_space = 5
     // CHECK: bitsize = 64
+    // CHECK: global_memory_space = 2
     // CHECK: preferred = 8
+    // CHECK: program_memory_space = 3
     // CHECK: size = 8
     // CHECK: stack_alignment = 128
     "test.data_layout_query"() : () -> !llvm.ptr<5>
     // CHECK: alignment = 4
     // CHECK: alloca_memory_space = 5
     // CHECK: bitsize = 32
+    // CHECK: global_memory_space = 2
     // CHECK: preferred = 8
+    // CHECK: program_memory_space = 3
     // CHECK: size = 4
     // CHECK: stack_alignment = 128
     "test.data_layout_query"() : () -> !llvm.ptr<3>
     // CHECK: alignment = 8
     // CHECK: alloca_memory_space = 5
     // CHECK: bitsize = 32
+    // CHECK: global_memory_space = 2
     // CHECK: preferred = 8
+    // CHECK: program_memory_space = 3
     // CHECK: size = 4
     // CHECK: stack_alignment = 128
     "test.data_layout_query"() : () -> !llvm.ptr<4>
diff --git a/mlir/test/lib/Dialect/DLTI/TestDataLayoutQuery.cpp b/mlir/test/lib/Dialect/DLTI/TestDataLayoutQuery.cpp
index 7e3d3f6dc3f0e6..740562e7783024 100644
--- a/mlir/test/lib/Dialect/DLTI/TestDataLayoutQuery.cpp
+++ b/mlir/test/lib/Dialect/DLTI/TestDataLayoutQuery.cpp
@@ -41,6 +41,8 @@ struct TestDataLayoutQuery
       unsigned alignment = layout.getTypeABIAlignment(op.getType());
       unsigned preferred = layout.getTypePreferredAlignment(op.getType());
       Attribute allocaMemorySpace = layout.getAllocaMemorySpace();
+      Attribute programMemorySpace = layout.getProgramMemorySpace();
+      Attribute globalMemorySpace = layout.getGlobalMemorySpace();
       unsigned stackAlignment = layout.getStackAlignment();
       op->setAttrs(
           {builder.getNamedAttr("size", builder.getIndexAttr(size)),
@@ -51,6 +53,14 @@ struct TestDataLayoutQuery
                                 allocaMemorySpace == Attribute()
                                     ? builder.getUI32IntegerAttr(0)
                                     : allocaMemorySpace),
+           builder.getNamedAttr("program_memory_space",
+                                programMemorySpace == Attribute()
+                                    ? builder.getUI32IntegerAttr(0)
+                                    : programMemorySpace),
+           builder.getNamedAttr("global_memory_space",
+                                glo...
[truncated]

@llvmbot
Copy link
Collaborator

llvmbot commented Jan 8, 2024

@llvm/pr-subscribers-mlir-llvm

Author: None (agozillon)

Changes

This patch is based on a previous PR https://reviews.llvm.org/D144657 that added alloca address space handling to MLIR's DataLayout and DLTI interface. This patch aims to add identical features to import and access the global and program memory space through MLIR's DataLayout/DLTI system.


Patch is 24.02 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/77367.diff

12 Files Affected:

  • (modified) mlir/include/mlir/Dialect/DLTI/DLTI.h (+6)
  • (modified) mlir/include/mlir/Dialect/DLTI/DLTIBase.td (+6)
  • (modified) mlir/include/mlir/Interfaces/DataLayoutInterfaces.h (+17-1)
  • (modified) mlir/include/mlir/Interfaces/DataLayoutInterfaces.td (+36)
  • (modified) mlir/lib/Dialect/DLTI/DLTI.cpp (+18)
  • (modified) mlir/lib/Interfaces/DataLayoutInterfaces.cpp (+60-2)
  • (modified) mlir/lib/Target/LLVMIR/DataLayoutImporter.cpp (+19-4)
  • (modified) mlir/lib/Target/LLVMIR/DataLayoutImporter.h (+2-1)
  • (modified) mlir/lib/Target/LLVMIR/ModuleTranslation.cpp (+20)
  • (modified) mlir/test/Dialect/LLVMIR/layout.mlir (+18)
  • (modified) mlir/test/lib/Dialect/DLTI/TestDataLayoutQuery.cpp (+10)
  • (modified) mlir/unittests/Interfaces/DataLayoutInterfacesTest.cpp (+38)
diff --git a/mlir/include/mlir/Dialect/DLTI/DLTI.h b/mlir/include/mlir/Dialect/DLTI/DLTI.h
index b9a8763eb44989..bf23aa2d48a801 100644
--- a/mlir/include/mlir/Dialect/DLTI/DLTI.h
+++ b/mlir/include/mlir/Dialect/DLTI/DLTI.h
@@ -103,6 +103,12 @@ class DataLayoutSpecAttr
   /// Returns the alloca memory space identifier.
   StringAttr getAllocaMemorySpaceIdentifier(MLIRContext *context) const;
 
+  /// Returns the program memory space identifier.
+  StringAttr getProgramMemorySpaceIdentifier(MLIRContext *context) const;
+
+  /// Returns the global memory space identifier.
+  StringAttr getGlobalMemorySpaceIdentifier(MLIRContext *context) const;
+
   /// Returns the stack alignment identifier.
   StringAttr getStackAlignmentIdentifier(MLIRContext *context) const;
 
diff --git a/mlir/include/mlir/Dialect/DLTI/DLTIBase.td b/mlir/include/mlir/Dialect/DLTI/DLTIBase.td
index 7d519f4efcd224..3572a99fad8743 100644
--- a/mlir/include/mlir/Dialect/DLTI/DLTIBase.td
+++ b/mlir/include/mlir/Dialect/DLTI/DLTIBase.td
@@ -39,6 +39,12 @@ def DLTI_Dialect : Dialect {
 
     constexpr const static ::llvm::StringLiteral
     kDataLayoutAllocaMemorySpaceKey = "dlti.alloca_memory_space";
+    
+    constexpr const static ::llvm::StringLiteral
+    kDataLayoutProgramMemorySpaceKey = "dlti.program_memory_space";
+
+    constexpr const static ::llvm::StringLiteral
+    kDataLayoutGlobalMemorySpaceKey = "dlti.global_memory_space";
 
     constexpr const static ::llvm::StringLiteral
     kDataLayoutStackAlignmentKey = "dlti.stack_alignment";
diff --git a/mlir/include/mlir/Interfaces/DataLayoutInterfaces.h b/mlir/include/mlir/Interfaces/DataLayoutInterfaces.h
index f4b9b95fb89f89..4a21f76dfc5d1c 100644
--- a/mlir/include/mlir/Interfaces/DataLayoutInterfaces.h
+++ b/mlir/include/mlir/Interfaces/DataLayoutInterfaces.h
@@ -61,6 +61,14 @@ getDefaultPreferredAlignment(Type type, const DataLayout &dataLayout,
 /// DataLayoutInterface if specified, otherwise returns the default.
 Attribute getDefaultAllocaMemorySpace(DataLayoutEntryInterface entry);
 
+/// Default handler for program memory space request. Dispatches to the
+/// DataLayoutInterface if specified, otherwise returns the default.
+Attribute getDefaultProgramMemorySpace(DataLayoutEntryInterface entry);
+
+/// Default handler for global memory space request. Dispatches to the
+/// DataLayoutInterface if specified, otherwise returns the default.
+Attribute getDefaultGlobalMemorySpace(DataLayoutEntryInterface entry);
+
 /// Default handler for the stack alignment request. Dispatches to the
 /// DataLayoutInterface if specified, otherwise returns the default.
 uint64_t getDefaultStackAlignment(DataLayoutEntryInterface entry);
@@ -175,6 +183,12 @@ class DataLayout {
   /// Returns the memory space used for AllocaOps.
   Attribute getAllocaMemorySpace() const;
 
+  /// Returns the memory space used for program memory operations.
+  Attribute getProgramMemorySpace() const;
+
+  /// Returns the memory space used for global operations.
+  Attribute getGlobalMemorySpace() const;
+
   /// Returns the natural alignment of the stack in bits. Alignment promotion of
   /// stack variables should be limited to the natural stack alignment to
   /// prevent dynamic stack alignment. Returns zero if the stack alignment is
@@ -203,8 +217,10 @@ class DataLayout {
   mutable DenseMap<Type, uint64_t> abiAlignments;
   mutable DenseMap<Type, uint64_t> preferredAlignments;
 
-  /// Cache for alloca memory space.
+  /// Cache for alloca, global, and program memory spaces.
   mutable std::optional<Attribute> allocaMemorySpace;
+  mutable std::optional<Attribute> programMemorySpace;
+  mutable std::optional<Attribute> globalMemorySpace;
 
   /// Cache for stack alignment.
   mutable std::optional<uint64_t> stackAlignment;
diff --git a/mlir/include/mlir/Interfaces/DataLayoutInterfaces.td b/mlir/include/mlir/Interfaces/DataLayoutInterfaces.td
index 2f60a16baf50bc..a8def967fffcfa 100644
--- a/mlir/include/mlir/Interfaces/DataLayoutInterfaces.td
+++ b/mlir/include/mlir/Interfaces/DataLayoutInterfaces.td
@@ -112,6 +112,18 @@ def DataLayoutSpecInterface : AttrInterface<"DataLayoutSpecInterface"> {
       /*methodName=*/"getAllocaMemorySpaceIdentifier",
       /*args=*/(ins "::mlir::MLIRContext *":$context)
     >,
+    InterfaceMethod<
+      /*description=*/"Returns the program memory space identifier.",
+      /*retTy=*/"::mlir::StringAttr",
+      /*methodName=*/"getProgramMemorySpaceIdentifier",
+      /*args=*/(ins "::mlir::MLIRContext *":$context)
+    >,
+    InterfaceMethod<
+      /*description=*/"Returns the global memory space identifier.",
+      /*retTy=*/"::mlir::StringAttr",
+      /*methodName=*/"getGlobalMemorySpaceIdentifier",
+      /*args=*/(ins "::mlir::MLIRContext *":$context)
+    >,
     InterfaceMethod<
       /*description=*/"Returns the stack alignment identifier.",
       /*retTy=*/"::mlir::StringAttr",
@@ -280,6 +292,30 @@ def DataLayoutOpInterface : OpInterface<"DataLayoutOpInterface"> {
         return ::mlir::detail::getDefaultAllocaMemorySpace(entry);
       }]
     >,
+    StaticInterfaceMethod<
+      /*description=*/"Returns the memory space used by the ABI computed "
+                      "using the relevant entries. The data layout object "
+                      "can be used for recursive queries.",
+      /*retTy=*/"::mlir::Attribute",
+      /*methodName=*/"getProgramMemorySpace",
+      /*args=*/(ins "::mlir::DataLayoutEntryInterface":$entry),
+      /*methodBody=*/"",
+      /*defaultImplementation=*/[{
+        return ::mlir::detail::getDefaultProgramMemorySpace(entry);
+      }]
+    >,
+    StaticInterfaceMethod<
+      /*description=*/"Returns the memory space used by the ABI computed "
+                      "using the relevant entries. The data layout object "
+                      "can be used for recursive queries.",
+      /*retTy=*/"::mlir::Attribute",
+      /*methodName=*/"getGlobalMemorySpace",
+      /*args=*/(ins "::mlir::DataLayoutEntryInterface":$entry),
+      /*methodBody=*/"",
+      /*defaultImplementation=*/[{
+        return ::mlir::detail::getDefaultGlobalMemorySpace(entry);
+      }]
+    >,
     StaticInterfaceMethod<
       /*description=*/"Returns the natural stack alignment in bits computed "
                       "using the relevant entries. The data layout object "
diff --git a/mlir/lib/Dialect/DLTI/DLTI.cpp b/mlir/lib/Dialect/DLTI/DLTI.cpp
index aba9e7db0a2fef..daef2349430d0b 100644
--- a/mlir/lib/Dialect/DLTI/DLTI.cpp
+++ b/mlir/lib/Dialect/DLTI/DLTI.cpp
@@ -108,6 +108,11 @@ void DataLayoutEntryAttr::print(AsmPrinter &os) const {
 constexpr const StringLiteral mlir::DataLayoutSpecAttr::kAttrKeyword;
 constexpr const StringLiteral
     mlir::DLTIDialect::kDataLayoutAllocaMemorySpaceKey;
+constexpr const StringLiteral
+    mlir::DLTIDialect::kDataLayoutProgramMemorySpaceKey;
+constexpr const StringLiteral
+    mlir::DLTIDialect::kDataLayoutGlobalMemorySpaceKey;
+
 constexpr const StringLiteral mlir::DLTIDialect::kDataLayoutStackAlignmentKey;
 
 namespace mlir {
@@ -282,6 +287,17 @@ DataLayoutSpecAttr::getAllocaMemorySpaceIdentifier(MLIRContext *context) const {
       DLTIDialect::kDataLayoutAllocaMemorySpaceKey);
 }
 
+StringAttr DataLayoutSpecAttr::getProgramMemorySpaceIdentifier(
+    MLIRContext *context) const {
+  return Builder(context).getStringAttr(
+      DLTIDialect::kDataLayoutProgramMemorySpaceKey);
+}
+
+StringAttr
+DataLayoutSpecAttr::getGlobalMemorySpaceIdentifier(MLIRContext *context) const {
+  return Builder(context).getStringAttr(
+      DLTIDialect::kDataLayoutGlobalMemorySpaceKey);
+}
 StringAttr
 DataLayoutSpecAttr::getStackAlignmentIdentifier(MLIRContext *context) const {
   return Builder(context).getStringAttr(
@@ -345,6 +361,8 @@ class TargetDataLayoutInterface : public DataLayoutDialectInterface {
                             << DLTIDialect::kDataLayoutEndiannessLittle << "'";
     }
     if (entryName == DLTIDialect::kDataLayoutAllocaMemorySpaceKey ||
+        entryName == DLTIDialect::kDataLayoutProgramMemorySpaceKey ||
+        entryName == DLTIDialect::kDataLayoutGlobalMemorySpaceKey ||
         entryName == DLTIDialect::kDataLayoutStackAlignmentKey)
       return success();
     return emitError(loc) << "unknown data layout entry name: " << entryName;
diff --git a/mlir/lib/Interfaces/DataLayoutInterfaces.cpp b/mlir/lib/Interfaces/DataLayoutInterfaces.cpp
index 1178417fd2a6ca..65c41f44192a90 100644
--- a/mlir/lib/Interfaces/DataLayoutInterfaces.cpp
+++ b/mlir/lib/Interfaces/DataLayoutInterfaces.cpp
@@ -230,6 +230,30 @@ mlir::detail::getDefaultAllocaMemorySpace(DataLayoutEntryInterface entry) {
   return entry.getValue();
 }
 
+// Returns the memory space used for the program memory space.  if
+// specified in the given entry. If the entry is empty the default
+// memory space represented by an empty attribute is returned.
+Attribute
+mlir::detail::getDefaultProgramMemorySpace(DataLayoutEntryInterface entry) {
+  if (entry == DataLayoutEntryInterface()) {
+    return Attribute();
+  }
+
+  return entry.getValue();
+}
+
+// Returns the memory space used for global the global memory space. if
+// specified in the given entry. If the entry is empty the default memory
+// space represented by an empty attribute is returned.
+Attribute
+mlir::detail::getDefaultGlobalMemorySpace(DataLayoutEntryInterface entry) {
+  if (entry == DataLayoutEntryInterface()) {
+    return Attribute();
+  }
+
+  return entry.getValue();
+}
+
 // Returns the stack alignment if specified in the given entry. If the entry is
 // empty the default alignment zero is returned.
 uint64_t
@@ -382,7 +406,8 @@ mlir::DataLayout::DataLayout() : DataLayout(ModuleOp()) {}
 
 mlir::DataLayout::DataLayout(DataLayoutOpInterface op)
     : originalLayout(getCombinedDataLayout(op)), scope(op),
-      allocaMemorySpace(std::nullopt), stackAlignment(std::nullopt) {
+      allocaMemorySpace(std::nullopt), programMemorySpace(std::nullopt),
+      globalMemorySpace(std::nullopt), stackAlignment(std::nullopt) {
 #if LLVM_ENABLE_ABI_BREAKING_CHECKS
   checkMissingLayout(originalLayout, op);
   collectParentLayouts(op, layoutStack);
@@ -391,7 +416,8 @@ mlir::DataLayout::DataLayout(DataLayoutOpInterface op)
 
 mlir::DataLayout::DataLayout(ModuleOp op)
     : originalLayout(getCombinedDataLayout(op)), scope(op),
-      allocaMemorySpace(std::nullopt), stackAlignment(std::nullopt) {
+      allocaMemorySpace(std::nullopt), programMemorySpace(std::nullopt),
+      globalMemorySpace(std::nullopt), stackAlignment(std::nullopt) {
 #if LLVM_ENABLE_ABI_BREAKING_CHECKS
   checkMissingLayout(originalLayout, op);
   collectParentLayouts(op, layoutStack);
@@ -510,6 +536,38 @@ mlir::Attribute mlir::DataLayout::getAllocaMemorySpace() const {
   return *allocaMemorySpace;
 }
 
+mlir::Attribute mlir::DataLayout::getProgramMemorySpace() const {
+  checkValid();
+  if (programMemorySpace)
+    return *programMemorySpace;
+  DataLayoutEntryInterface entry;
+  if (originalLayout)
+    entry = originalLayout.getSpecForIdentifier(
+        originalLayout.getProgramMemorySpaceIdentifier(
+            originalLayout.getContext()));
+  if (auto iface = dyn_cast_or_null<DataLayoutOpInterface>(scope))
+    programMemorySpace = iface.getProgramMemorySpace(entry);
+  else
+    programMemorySpace = detail::getDefaultProgramMemorySpace(entry);
+  return *programMemorySpace;
+}
+
+mlir::Attribute mlir::DataLayout::getGlobalMemorySpace() const {
+  checkValid();
+  if (globalMemorySpace)
+    return *globalMemorySpace;
+  DataLayoutEntryInterface entry;
+  if (originalLayout)
+    entry = originalLayout.getSpecForIdentifier(
+        originalLayout.getGlobalMemorySpaceIdentifier(
+            originalLayout.getContext()));
+  if (auto iface = dyn_cast_or_null<DataLayoutOpInterface>(scope))
+    globalMemorySpace = iface.getGlobalMemorySpace(entry);
+  else
+    globalMemorySpace = detail::getDefaultGlobalMemorySpace(entry);
+  return *globalMemorySpace;
+}
+
 uint64_t mlir::DataLayout::getStackAlignment() const {
   checkValid();
   if (stackAlignment)
diff --git a/mlir/lib/Target/LLVMIR/DataLayoutImporter.cpp b/mlir/lib/Target/LLVMIR/DataLayoutImporter.cpp
index 95f3cc074b1ddf..392f552f480cc8 100644
--- a/mlir/lib/Target/LLVMIR/DataLayoutImporter.cpp
+++ b/mlir/lib/Target/LLVMIR/DataLayoutImporter.cpp
@@ -164,9 +164,9 @@ DataLayoutImporter::tryToEmplaceEndiannessEntry(StringRef endianness,
 }
 
 LogicalResult
-DataLayoutImporter::tryToEmplaceAllocaAddrSpaceEntry(StringRef token) {
-  auto key =
-      StringAttr::get(context, DLTIDialect::kDataLayoutAllocaMemorySpaceKey);
+DataLayoutImporter::tryToEmplaceAddrSpaceEntry(StringRef token,
+                                               llvm::StringLiteral spaceKey) {
+  auto key = StringAttr::get(context, spaceKey);
   if (keyEntries.count(key))
     return success();
 
@@ -247,9 +247,24 @@ void DataLayoutImporter::translateDataLayout(
         return;
       continue;
     }
+    // Parse the program address space.
+    if (*prefix == "P") {
+      if (failed(tryToEmplaceAddrSpaceEntry(
+              token, DLTIDialect::kDataLayoutProgramMemorySpaceKey)))
+        return;
+      continue;
+    }
+    // Parse the global address space.
+    if (*prefix == "G") {
+      if (failed(tryToEmplaceAddrSpaceEntry(
+              token, DLTIDialect::kDataLayoutGlobalMemorySpaceKey)))
+        return;
+      continue;
+    }
     // Parse the alloca address space.
     if (*prefix == "A") {
-      if (failed(tryToEmplaceAllocaAddrSpaceEntry(token)))
+      if (failed(tryToEmplaceAddrSpaceEntry(
+              token, DLTIDialect::kDataLayoutAllocaMemorySpaceKey)))
         return;
       continue;
     }
diff --git a/mlir/lib/Target/LLVMIR/DataLayoutImporter.h b/mlir/lib/Target/LLVMIR/DataLayoutImporter.h
index 15f0f7ddf07059..59b60acd24be20 100644
--- a/mlir/lib/Target/LLVMIR/DataLayoutImporter.h
+++ b/mlir/lib/Target/LLVMIR/DataLayoutImporter.h
@@ -97,7 +97,8 @@ class DataLayoutImporter {
                                             StringRef token);
 
   /// Adds an alloca address space entry if there is none yet.
-  LogicalResult tryToEmplaceAllocaAddrSpaceEntry(StringRef token);
+  LogicalResult tryToEmplaceAddrSpaceEntry(StringRef token,
+                                           llvm::StringLiteral spaceKey);
 
   /// Adds a stack alignment entry if there is none yet.
   LogicalResult tryToEmplaceStackAlignmentEntry(StringRef token);
diff --git a/mlir/lib/Target/LLVMIR/ModuleTranslation.cpp b/mlir/lib/Target/LLVMIR/ModuleTranslation.cpp
index 1722d74c08b628..ce46a194ea7d9f 100644
--- a/mlir/lib/Target/LLVMIR/ModuleTranslation.cpp
+++ b/mlir/lib/Target/LLVMIR/ModuleTranslation.cpp
@@ -190,6 +190,26 @@ translateDataLayout(DataLayoutSpecInterface attribute,
       layoutStream.flush();
       continue;
     }
+    if (key.getValue() == DLTIDialect::kDataLayoutProgramMemorySpaceKey) {
+      auto value = cast<IntegerAttr>(entry.getValue());
+      uint64_t space = value.getValue().getZExtValue();
+      // Skip the default address space.
+      if (space == 0)
+        continue;
+      layoutStream << "-P" << space;
+      layoutStream.flush();
+      continue;
+    }
+    if (key.getValue() == DLTIDialect::kDataLayoutGlobalMemorySpaceKey) {
+      auto value = cast<IntegerAttr>(entry.getValue());
+      uint64_t space = value.getValue().getZExtValue();
+      // Skip the default address space.
+      if (space == 0)
+        continue;
+      layoutStream << "-G" << space;
+      layoutStream.flush();
+      continue;
+    }
     if (key.getValue() == DLTIDialect::kDataLayoutAllocaMemorySpaceKey) {
       auto value = cast<IntegerAttr>(entry.getValue());
       uint64_t space = value.getValue().getZExtValue();
diff --git a/mlir/test/Dialect/LLVMIR/layout.mlir b/mlir/test/Dialect/LLVMIR/layout.mlir
index 99d61785384487..2868e1740f861c 100644
--- a/mlir/test/Dialect/LLVMIR/layout.mlir
+++ b/mlir/test/Dialect/LLVMIR/layout.mlir
@@ -6,21 +6,27 @@ module {
     // CHECK: alignment = 8
     // CHECK: alloca_memory_space = 0
     // CHECK: bitsize = 64
+    // CHECK: global_memory_space = 0
     // CHECK: preferred = 8
+    // CHECK: program_memory_space = 0
     // CHECK: size = 8
     // CHECK: stack_alignment = 0
     "test.data_layout_query"() : () -> !llvm.ptr
     // CHECK: alignment = 8
     // CHECK: alloca_memory_space = 0
     // CHECK: bitsize = 64
+    // CHECK: global_memory_space = 0
     // CHECK: preferred = 8
+    // CHECK: program_memory_space = 0
     // CHECK: size = 8
     // CHECK: stack_alignment = 0
     "test.data_layout_query"() : () -> !llvm.ptr<3>
     // CHECK: alignment = 8
     // CHECK: alloca_memory_space = 0
     // CHECK: bitsize = 64
+    // CHECK: global_memory_space = 0
     // CHECK: preferred = 8
+    // CHECK: program_memory_space = 0
     // CHECK: size = 8
     // CHECK: stack_alignment = 0
     "test.data_layout_query"() : () -> !llvm.ptr<5>
@@ -35,6 +41,8 @@ module attributes { dlti.dl_spec = #dlti.dl_spec<
   #dlti.dl_entry<!llvm.ptr<5>, dense<[64, 64, 64]> : vector<3xi64>>,
   #dlti.dl_entry<!llvm.ptr<4>, dense<[32, 64, 64]> : vector<3xi64>>,
   #dlti.dl_entry<"dlti.alloca_memory_space", 5 : ui64>,
+  #dlti.dl_entry<"dlti.global_memory_space", 2 : ui64>,
+  #dlti.dl_entry<"dlti.program_memory_space", 3 : ui64>,
   #dlti.dl_entry<"dlti.stack_alignment", 128 : i64>
 >} {
   // CHECK: @spec
@@ -42,35 +50,45 @@ module attributes { dlti.dl_spec = #dlti.dl_spec<
     // CHECK: alignment = 4
     // CHECK: alloca_memory_space = 5
     // CHECK: bitsize = 32
+    // CHECK: global_memory_space = 2
     // CHECK: preferred = 8
+    // CHECK: program_memory_space = 3
     // CHECK: size = 4
     // CHECK: stack_alignment = 128
     "test.data_layout_query"() : () -> !llvm.ptr
     // CHECK: alignment = 4
     // CHECK: alloca_memory_space = 5
     // CHECK: bitsize = 32
+    // CHECK: global_memory_space = 2
     // CHECK: preferred = 8
+    // CHECK: program_memory_space = 3
     // CHECK: size = 4
     // CHECK: stack_alignment = 128
     "test.data_layout_query"() : () -> !llvm.ptr<3>
     // CHECK: alignment = 8
     // CHECK: alloca_memory_space = 5
     // CHECK: bitsize = 64
+    // CHECK: global_memory_space = 2
     // CHECK: preferred = 8
+    // CHECK: program_memory_space = 3
     // CHECK: size = 8
     // CHECK: stack_alignment = 128
     "test.data_layout_query"() : () -> !llvm.ptr<5>
     // CHECK: alignment = 4
     // CHECK: alloca_memory_space = 5
     // CHECK: bitsize = 32
+    // CHECK: global_memory_space = 2
     // CHECK: preferred = 8
+    // CHECK: program_memory_space = 3
     // CHECK: size = 4
     // CHECK: stack_alignment = 128
     "test.data_layout_query"() : () -> !llvm.ptr<3>
     // CHECK: alignment = 8
     // CHECK: alloca_memory_space = 5
     // CHECK: bitsize = 32
+    // CHECK: global_memory_space = 2
     // CHECK: preferred = 8
+    // CHECK: program_memory_space = 3
     // CHECK: size = 4
     // CHECK: stack_alignment = 128
     "test.data_layout_query"() : () -> !llvm.ptr<4>
diff --git a/mlir/test/lib/Dialect/DLTI/TestDataLayoutQuery.cpp b/mlir/test/lib/Dialect/DLTI/TestDataLayoutQuery.cpp
index 7e3d3f6dc3f0e6..740562e7783024 100644
--- a/mlir/test/lib/Dialect/DLTI/TestDataLayoutQuery.cpp
+++ b/mlir/test/lib/Dialect/DLTI/TestDataLayoutQuery.cpp
@@ -41,6 +41,8 @@ struct TestDataLayoutQuery
       unsigned alignment = layout.getTypeABIAlignment(op.getType());
       unsigned preferred = layout.getTypePreferredAlignment(op.getType());
       Attribute allocaMemorySpace = layout.getAllocaMemorySpace();
+      Attribute programMemorySpace = layout.getProgramMemorySpace();
+      Attribute globalMemorySpace = layout.getGlobalMemorySpace();
       unsigned stackAlignment = layout.getStackAlignment();
       op->setAttrs(
           {builder.getNamedAttr("size", builder.getIndexAttr(size)),
@@ -51,6 +53,14 @@ struct TestDataLayoutQuery
                                 allocaMemorySpace == Attribute()
                                     ? builder.getUI32IntegerAttr(0)
                                     : allocaMemorySpace),
+           builder.getNamedAttr("program_memory_space",
+                                programMemorySpace == Attribute()
+                                    ? builder.getUI32IntegerAttr(0)
+                                    : programMemorySpace),
+           builder.getNamedAttr("global_memory_space",
+                                glo...
[truncated]

Copy link
Contributor

@krzysz00 krzysz00 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

I think, long-term, we'll need some way to map GPU memory spaces to their LLVM counterparts in an IR-visible way, but that's not this PR.

@agozillon
Copy link
Contributor Author

Thank you very much for the quick review @krzysz00! I'll land the PR tomorrow afternoon EU time, provided no one else has any issue with the patch in the meantime. The failing tests are windows tests and appear to be unrelated and an unfortunate occurrence of this issue with the windows buildbot: #77086

We're starting to touch a bit on the more intricate need for address spaces in Flang for OpenMP offloading on AMDGPU so perhaps we'll get to that long term plan sooner rather than later!

@agozillon agozillon merged commit c1ed45a into llvm:main Jan 9, 2024
6 of 7 checks passed
StringAttr getProgramMemorySpaceIdentifier(MLIRContext *context) const;

/// Returns the global memory space identifier.
StringAttr getGlobalMemorySpaceIdentifier(MLIRContext *context) const;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a doc that specifies what do these means? (The comments are just paraphrasing the API name right now)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the data layout document in LLVM which describes these segments of the layout in a little more detail: https://llvm.org/docs/LangRef.html#data-layout

I am happy to open a PR to add further comments if you wish however! But yes, I mimicked the comments from the original PR this one is based on which does the same.

Copy link
Collaborator

@joker-eph joker-eph Jan 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this maps to LLVM data layout documentation, that’s fine.
I probably should have checked there first!

thanks

justinfargnoli pushed a commit to justinfargnoli/llvm-project that referenced this pull request Jan 28, 2024
…t subsystem (llvm#77367)

This patch is based on a previous PR https://reviews.llvm.org/D144657
that added alloca address space handling to MLIR's DataLayout and DLTI
interface. This patch aims to add identical features to import and
access the global and program memory space through MLIR's
DataLayout/DLTI system.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants