Introduce MLIR Op Properties

This new features enabled to dedicate custom storage inline within operations.
This storage can be used as an alternative to attributes to store data that is
specific to an operation. Attribute can also be stored inside the properties
storage if desired, but any kind of data can be present as well. This offers
a way to store and mutate data without uniquing in the Context like Attribute.
See the OpPropertiesTest.cpp for an example where a struct with a
std::vector<> is attached to an operation and mutated in-place:

struct TestProperties {
  int a = -1;
  float b = -1.;
  std::vector<int64_t> array = {-33};
};

More complex scheme (including reference-counting) are also possible.

The only constraint to enable storing a C++ object as "properties" on an
operation is to implement three functions:

- convert from the candidate object to an Attribute
- convert from the Attribute to the candidate object
- hash the object

Optional the parsing and printing can also be customized with 2 extra
functions.

A new options is introduced to ODS to allow dialects to specify:

  let usePropertiesForAttributes = 1;

When set to true, the inherent attributes for all the ops in this dialect
will be using properties instead of being stored alongside discardable
attributes.
The TestDialect showcases this feature.

Another change is that we introduce new APIs on the Operation class
to access separately the inherent attributes from the discardable ones.
We envision deprecating and removing the `getAttr()`, `getAttrsDictionary()`,
and other similar method which don't make the distinction explicit, leading
to an entirely separate namespace for discardable attributes.

Differential Revision: https://reviews.llvm.org/D141742

Author: joker-eph

mlir/docs/LangRef.md

@@ -290,12 +290,14 @@ Syntax:
 operation            ::= op-result-list? (generic-operation | custom-operation)
                          trailing-location?
 generic-operation    ::= string-literal `(` value-use-list? `)`  successor-list?
-                         region-list? dictionary-attribute? `:` function-type
+                         dictionary-properties? region-list? dictionary-attribute?
+                         `:` function-type
 custom-operation     ::= bare-id custom-operation-format
 op-result-list       ::= op-result (`,` op-result)* `=`
 op-result            ::= value-id (`:` integer-literal)
 successor-list       ::= `[` successor (`,` successor)* `]`
 successor            ::= caret-id (`:` block-arg-list)?
+dictionary-propertes ::= `<` dictionary-attribute `>`
 region-list          ::= `(` region (`,` region)* `)`
 dictionary-attribute ::= `{` (attribute-entry (`,` attribute-entry)*)? `}`
 trailing-location    ::= (`loc` `(` location `)`)?
@@ -312,9 +314,10 @@ semantics. For example, MLIR supports
 The internal representation of an operation is simple: an operation is
 identified by a unique string (e.g. `dim`, `tf.Conv2d`, `x86.repmovsb`,
 `ppc.eieio`, etc), can return zero or more results, take zero or more operands,
-has a dictionary of [attributes](#attributes), has zero or more successors, and
-zero or more enclosed [regions](#regions). The generic printing form includes
-all these elements literally, with a function type to indicate the types of the
+has storage for [properties](#properties), has a dictionary of
+[attributes](#attributes), has zero or more successors, and zero or more
+enclosed [regions](#regions). The generic printing form includes all these
+elements literally, with a function type to indicate the types of the
 results and operands.
 
 Example:
@@ -328,8 +331,11 @@ Example:
 %foo, %bar = "foo_div"() : () -> (f32, i32)
 
 // Invoke a TensorFlow function called tf.scramble with two inputs
-// and an attribute "fruit".
-%2 = "tf.scramble"(%result#0, %bar) {fruit = "banana"} : (f32, i32) -> f32
+// and an attribute "fruit" stored in properties.
+%2 = "tf.scramble"(%result#0, %bar) <{fruit = "banana"}> : (f32, i32) -> f32
+
+// Invoke an operation with some discardable attributes
+%foo, %bar = "foo_div"() {some_attr = "value", other_attr = 42 : i64} : () -> (f32, i32)
 ```
 
 In addition to the basic syntax above, dialects may register known operations.
@@ -733,6 +739,15 @@ The [builtin dialect](Dialects/Builtin.md) defines a set of types that are
 directly usable by any other dialect in MLIR. These types cover a range from
 primitive integer and floating-point types, function types, and more.
 
+## Properties
+
+Properties are extra data members stored directly on an Operation class. They
+provide a way to store [inherent attributes](#attributes) and other arbitrary
+data. The semantics of the data is specific to a given operation, and may be
+exposed through [Interfaces](Interfaces.md) accessors and other methods.
+Properties can always be serialized to Attribute in order to be printed
+generically.
+
 ## Attributes
 
 Syntax:
@@ -751,9 +766,10 @@ values. MLIR's builtin dialect provides a rich set of
 arrays, dictionaries, strings, etc.). Additionally, dialects can define their
 own [dialect attribute values](#dialect-attribute-values).
 
-The top-level attribute dictionary attached to an operation has special
-semantics. The attribute entries are considered to be of two different kinds
-based on whether their dictionary key has a dialect prefix:
+For dialects which haven't adopted properties yet, the top-level attribute
+dictionary attached to an operation has special semantics. The attribute
+entries are considered to be of two different kinds based on whether their
+dictionary key has a dialect prefix:
 
 -   *inherent attributes* are inherent to the definition of an operation's
     semantics. The operation itself is expected to verify the consistency of
@@ -771,6 +787,10 @@ Note that attribute values are allowed to themselves be dictionary attributes,
 but only the top-level dictionary attribute attached to the operation is subject
 to the classification above.
 
+When properties are adopted, only discardable attributes are stored in the
+top-level dictionary, while inherent attributes are stored in the properties
+storage.
+
 ### Attribute Value Aliases
 
 ```

mlir/include/mlir/Conversion/LLVMCommon/Pattern.h

@@ -145,6 +145,11 @@ class ConvertOpToLLVMPattern : public ConvertToLLVMPattern {
   /// Wrappers around the RewritePattern methods that pass the derived op type.
   void rewrite(Operation *op, ArrayRef<Value> operands,
                ConversionPatternRewriter &rewriter) const final {
+    if constexpr (SourceOp::hasProperties())
+      rewrite(cast<SourceOp>(op),
+              OpAdaptor(operands, op->getAttrDictionary(),
+                        cast<SourceOp>(op).getProperties()),
+              rewriter);
     rewrite(cast<SourceOp>(op), OpAdaptor(operands, op->getAttrDictionary()),
             rewriter);
   }
@@ -154,6 +159,11 @@ class ConvertOpToLLVMPattern : public ConvertToLLVMPattern {
   LogicalResult
   matchAndRewrite(Operation *op, ArrayRef<Value> operands,
                   ConversionPatternRewriter &rewriter) const final {
+    if constexpr (SourceOp::hasProperties())
+      return matchAndRewrite(cast<SourceOp>(op),
+                             OpAdaptor(operands, op->getAttrDictionary(),
+                                       cast<SourceOp>(op).getProperties()),
+                             rewriter);
     return matchAndRewrite(cast<SourceOp>(op),
                            OpAdaptor(operands, op->getAttrDictionary()),
                            rewriter);

mlir/include/mlir/IR/DialectBase.td

@@ -103,6 +103,9 @@ class Dialect {
 
   // If this dialect can be extended at runtime with new operations or types.
   bit isExtensible = 0;
+
+  // Whether inherent Attributes defined in ODS will be stored as Properties.
+  bit usePropertiesForAttributes = 0;
 }
 
 #endif // DIALECTBASE_TD

mlir/include/mlir/IR/ExtensibleDialect.h

@@ -26,6 +26,8 @@
 #include "mlir/IR/OpDefinition.h"
 #include "mlir/Support/TypeID.h"
 #include "llvm/ADT/StringMap.h"
+#include "llvm/Support/ErrorHandling.h"
+#include <optional>
 
 namespace mlir {
 class AsmParser;
@@ -462,6 +464,35 @@ class DynamicOpDefinition : public OperationName::Impl {
     return verifyRegionFn(op);
   }
 
+  /// Implementation for properties (unsupported right now here).
+  std::optional<Attribute> getInherentAttr(Operation *op,
+                                           StringRef name) final {
+    llvm::report_fatal_error("Unsupported getInherentAttr on Dynamic dialects");
+  }
+  void setInherentAttr(Operation *op, StringAttr name, Attribute value) final {
+    llvm::report_fatal_error("Unsupported setInherentAttr on Dynamic dialects");
+  }
+  void populateInherentAttrs(Operation *op, NamedAttrList &attrs) final {}
+  LogicalResult
+  verifyInherentAttrs(OperationName opName, NamedAttrList &attributes,
+                      function_ref<InFlightDiagnostic()> getDiag) final {
+    return success();
+  }
+  int getOpPropertyByteSize() final { return 0; }
+  void initProperties(OperationName opName, OpaqueProperties storage,
+                      OpaqueProperties init) final {}
+  void deleteProperties(OpaqueProperties prop) final {}
+  void populateDefaultProperties(OperationName opName,
+                                 OpaqueProperties properties) final {}
+
+  LogicalResult setPropertiesFromAttr(Operation *op, Attribute attr,
+                                      InFlightDiagnostic *diag) final {
+    return failure();
+  }
+  Attribute getPropertiesAsAttr(Operation *op) final { return {}; }
+  void copyProperties(OpaqueProperties lhs, OpaqueProperties rhs) final {}
+  llvm::hash_code hashProperties(OpaqueProperties prop) final { return {}; }
+
 private:
   DynamicOpDefinition(
       StringRef name, ExtensibleDialect *dialect,

mlir/include/mlir/IR/ODSSupport.h

@@ -0,0 +1,45 @@
+//===- ODSSupport.h ---------------------------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file defines a number of support method for ODS generated code.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef MLIR_IR_ODSSUPPORT_H
+#define MLIR_IR_ODSSUPPORT_H
+
+#include "mlir/IR/Attributes.h"
+
+namespace mlir {
+
+//===----------------------------------------------------------------------===//
+// Support for properties
+//===----------------------------------------------------------------------===//
+
+/// Convert an IntegerAttr attribute to an int64_t, or return an error if the
+/// attribute isn't an IntegerAttr. If the optional diagnostic is provided an
+/// error message is also emitted.
+LogicalResult convertFromAttribute(int64_t &storage, Attribute attr,
+                                   InFlightDiagnostic *diag);
+
+/// Convert the provided int64_t to an IntegerAttr attribute.
+Attribute convertToAttribute(MLIRContext *ctx, int64_t storage);
+
+/// Convert a DenseI64ArrayAttr to the provided storage. It is expected that the
+/// storage has the same size as the array. An error is returned if the
+/// attribute isn't a DenseI64ArrayAttr or it does not have the same size. If
+/// the optional diagnostic is provided an error message is also emitted.
+LogicalResult convertFromAttribute(MutableArrayRef<int64_t> storage,
+                                   Attribute attr, InFlightDiagnostic *diag);
+
+/// Convert the provided ArrayRef<int64_t> to a DenseI64ArrayAttr attribute.
+Attribute convertToAttribute(MLIRContext *ctx, ArrayRef<int64_t> storage);
+
+} // namespace mlir
+
+#endif // MLIR_IR_ODSSUPPORT_H

mlir/include/mlir/IR/OpBase.td

@@ -179,6 +179,69 @@ class TypeConstraint<Pred predicate, string summary = "",
   string cppClassName = cppClassNameParam;
 }
 
+// Base class for defining properties.
+class Property<string storageTypeParam = "", string desc = ""> {
+  // User-readable one line summary used in error reporting messages. If empty,
+  // a generic message will be used.
+  string summary = desc;
+  // The full description of this property.
+  string description = "";
+  code storageType = storageTypeParam;
+  code interfaceType = storageTypeParam;
+
+  // The expression to convert from the storage type to the Interface
+  // type. For example, an enum can be stored as an int but returned as an
+  // enum class.
+  //
+  // Format:
+  // - `$_storage` will contain the property in the storage type.
+  // - `$_ctxt` will contain an `MLIRContext *`.
+  code convertFromStorage = "$_storage";
+
+  // The call expression to build a property storage from the interface type.
+  //
+  // Format:
+  // - `$_storage` will contain the property in the storage type.
+  // - `$_value` will contain the property in the user interface type.
+  code assignToStorage = "$_storage = $_value";
+
+  // The call expression to convert from the storage type to an attribute.
+  //
+  // Format:
+  // - `$_storage` is the storage type value.
+  // - `$_ctxt` is a `MLIRContext *`.
+  //
+  // The expression must result in an Attribute.
+  code convertToAttribute = [{
+    convertToAttribute($_ctxt, $_storage)
+  }];
+
+  // The call expression to convert from an Attribute to the storage type.
+  //
+  // Format:
+  // - `$_storage` is the storage type value.
+  // - `$_attr` is the attribute.
+  // - `$_diag` is an optional Diagnostic pointer to emit error.
+  //
+  // The expression must return a LogicalResult
+  code convertFromAttribute = [{
+    return convertFromAttribute($_storage, $_attr, $_diag);
+  }];
+
+  // The call expression to hash the property.
+  //
+  // Format:
+  // - `$_storage` is the variable to hash.
+  //
+  // The expression should define a llvm::hash_code.
+  code hashProperty = [{
+    llvm::hash_value($_storage);
+  }];
+
+  // Default value for the property.
+  string defaultValue = ?;
+}
+
 // Subclass for constraints on an attribute.
 class AttrConstraint<Pred predicate, string summary = ""> :
     Constraint<predicate, summary>;
@@ -1090,6 +1153,16 @@ class DefaultValuedStrAttr<Attr attr, string val>
 class DefaultValuedOptionalStrAttr<Attr attr, string val>
     : DefaultValuedOptionalAttr<attr, "\"" # val # "\"">;
 
+//===----------------------------------------------------------------------===//
+// Primitive property kinds
+
+class ArrayProperty<string storageTypeParam = "", int n, string desc = ""> :
+  Property<storageTypeParam # "[" # n # "]", desc> {
+  let interfaceType = "::llvm::ArrayRef<" # storageTypeParam # ">";
+  let convertFromStorage = "$_storage";
+  let assignToStorage = "::llvm::copy($_value, $_storage)";
+}
+
 //===----------------------------------------------------------------------===//
 // Primitive attribute kinds