From ea798446f28d6fbff73e2d5b0f408b63c6efd8e5 Mon Sep 17 00:00:00 2001 From: Renato Golin Date: Wed, 3 Sep 2025 11:27:25 +0100 Subject: [PATCH 1/7] [MLIR][Linalg][Docs] Add forms to Linalg rationale docs As discussed in: * https://discourse.llvm.org/t/rfc-linalg-forms/87994/ * https://discourse.llvm.org/t/rfc-extend-linalg-elemwise-named-ops-semantics/83927 * https://discourse.llvm.org/t/rfc-op-explosion-in-linalg/82863/1 * https://discourse.llvm.org/t/rfc-mlir-linalg-operation-tree/83586 --- mlir/docs/Rationale/RationaleLinalgDialect.md | 87 ++++++++++++++++--- 1 file changed, 73 insertions(+), 14 deletions(-) diff --git a/mlir/docs/Rationale/RationaleLinalgDialect.md b/mlir/docs/Rationale/RationaleLinalgDialect.md index 8975b0a7d515e..4426d3fb9b3c2 100644 --- a/mlir/docs/Rationale/RationaleLinalgDialect.md +++ b/mlir/docs/Rationale/RationaleLinalgDialect.md @@ -411,6 +411,20 @@ and is used as a lowering target for Linalg, enabling further transformations and combination of semantically-loaded and lower-level inputs. As such, Linalg is intended to complement Affine rather than replace it. +### Summary of Existing Alternatives a Picture +Lastly, we summarize our observations of lessons from [Prior +Art](#prior-art)---when viewed under the lense of our [Core Guiding +Principles](#guiding_principles)---with the following picture. + +MLIR Codegen Flow + +This figure is not meant to be perfectly accurate but a rough map of how we view +the distribution of structural information in existing systems, from a +codegen-friendly angle. Unsurprisingly, the +[Linalg Dialect](../Dialects/Linalg/_index.md) and its future evolutions aspire +to a position in the top-right of this map. + ## Core Guiding Principles ### Transformations and Simplicity First @@ -506,6 +520,65 @@ potential by introducing lower-level IR ops and *smaller* Linalg ops. This gradually reduces the potential, all the way to Loops + VectorOps and LLVMIR. +### Interchangeability of Forms + +Linalg's various forms (named, generic) also carry information, and that +information should be preserved as much as possible during the progressive +lowering. A `matmul` operation is a special case of a `contract` operation, +which in turn is a special case of `generic` operation. Transformations on +the more special forms should not be converted to the more generic ones +unnecessarily, in the same way that they should not be broken down into +loops + arithmetic if they can still be represented as a Linalg op. + +#### Generic, Category, Named + +The core Linalg operation tree has three forms: +* **Generic:** Represented by `linalg.generic` and can encode all perfectly-nested +loop operations. +* **Category:** Represented by `linalg.contract` and `linalg.elementwise`, +which are special (einsum) forms of the `generic` operation. +* **Named:** All _named_ forms that can lower to either _category_ or +_generic_ forms. For example, `linalg.matmul`, `linalg.add`, etc. + +Unlike lowering to loops, the different Linalg forms that are derived from +`linalg.generic` are *equivalent*. It should always be possible to convert +a named operation into a generic and back to named, if the semantics is +preserved. The various forms in the Linalg dialect are meant to facilitate +pattern matching (single operations or DAGs) and to be able to consider +different forms as *canonical* for different transforms. + +#### Special Operations + +Not all Linalg operations represent perfectly nested loops, and therefore +cannot be represented as a `linalg.generic`. There are two kinds of Linalg +operations that fall into this category: +* **Composite:** Operations that compose multiple Linalg operations, for +example `linalg.softmax`. These can be converted to a DAG of Linalg operations +(in any form). +* **Special Named:** Operations that are usually matched against library calls +or special lowering, but can only be lowered to a combination of Linalg and +non-Linalg operations, for example `linalg.*conv*`, `linalg.winograd*`, +`linalg.pooling*`, etc. + +#### Canonical Forms + +With multiple (often exchangeable) forms, and with transformation simplicity +in mind, compilers should aim for reducing matching and replacing complexity +as much as possible. When matching a single operation with a complex pattern, +having all the information from a `generic` is useful to iteratively match +different patterns in turn. However, when assembling a DAG of operations to +form a pattern, it's much simpler to match against named operations (like +`max` + `div` + `reduce` + `broadcast`) than their generic counterparts. + +This is where the interchangeability of forms comes in handy. Linalg has the +ability to specialize and generalize in order to convert the IR to a form that +is easier for a particular type of transform. With forms being semantically +equivalent, one can convert back-and-forth throughout the various transforms +to match the needs of each transform. For that particular transform, such +form can be considered _canonical_ and therefore "expected" for the pattern +to _match_. This reduces complexity of pattern matchers and simplifies compiler +pipelines. + ### Composable and Declarative Transformations Complex and impactful transformations need not be hard to manipulate, write or maintain. Mixing XLA-style high-level op semantics knowledge with generic @@ -606,17 +679,3 @@ Considering the *potential* described during the discussion on transformation would dictate that the potential remains constant. In contrast, Linalg advocates for ***monotonicity*** under transformations. - -### Summary of Existing Alternatives a Picture -Lastly, we summarize our observations of lessons from [Prior -Art](#prior-art)---when viewed under the lense of our [Core Guiding -Principles](#guiding_principles)---with the following picture. - -MLIR Codegen Flow - -This figure is not meant to be perfectly accurate but a rough map of how we view -the distribution of structural information in existing systems, from a -codegen-friendly angle. Unsurprisingly, the -[Linalg Dialect](../Dialects/Linalg/_index.md) and its future evolutions aspire -to a position in the top-right of this map. From 7ea9ca0617e09090cfc3f07b99cc1ccc1646b3d6 Mon Sep 17 00:00:00 2001 From: Renato Golin Date: Thu, 4 Sep 2025 16:31:29 +0100 Subject: [PATCH 2/7] revert unrelated change --- mlir/docs/Rationale/RationaleLinalgDialect.md | 28 +++++++++---------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/mlir/docs/Rationale/RationaleLinalgDialect.md b/mlir/docs/Rationale/RationaleLinalgDialect.md index 4426d3fb9b3c2..9f85921c43b81 100644 --- a/mlir/docs/Rationale/RationaleLinalgDialect.md +++ b/mlir/docs/Rationale/RationaleLinalgDialect.md @@ -411,20 +411,6 @@ and is used as a lowering target for Linalg, enabling further transformations and combination of semantically-loaded and lower-level inputs. As such, Linalg is intended to complement Affine rather than replace it. -### Summary of Existing Alternatives a Picture -Lastly, we summarize our observations of lessons from [Prior -Art](#prior-art)---when viewed under the lense of our [Core Guiding -Principles](#guiding_principles)---with the following picture. - -MLIR Codegen Flow - -This figure is not meant to be perfectly accurate but a rough map of how we view -the distribution of structural information in existing systems, from a -codegen-friendly angle. Unsurprisingly, the -[Linalg Dialect](../Dialects/Linalg/_index.md) and its future evolutions aspire -to a position in the top-right of this map. - ## Core Guiding Principles ### Transformations and Simplicity First @@ -679,3 +665,17 @@ Considering the *potential* described during the discussion on transformation would dictate that the potential remains constant. In contrast, Linalg advocates for ***monotonicity*** under transformations. + +### Summary of Existing Alternatives a Picture +Lastly, we summarize our observations of lessons from [Prior +Art](#prior-art)---when viewed under the lense of our [Core Guiding +Principles](#guiding_principles)---with the following picture. + +MLIR Codegen Flow + +This figure is not meant to be perfectly accurate but a rough map of how we view +the distribution of structural information in existing systems, from a +codegen-friendly angle. Unsurprisingly, the +[Linalg Dialect](../Dialects/Linalg/_index.md) and its future evolutions aspire +to a position in the top-right of this map. From e57ef2e0fb15d21197e63a1a1220a13cc46aaff0 Mon Sep 17 00:00:00 2001 From: Renato Golin Date: Fri, 5 Sep 2025 14:44:41 +0100 Subject: [PATCH 3/7] in a generic --- mlir/docs/Rationale/RationaleLinalgDialect.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mlir/docs/Rationale/RationaleLinalgDialect.md b/mlir/docs/Rationale/RationaleLinalgDialect.md index 9f85921c43b81..c337bc82193b9 100644 --- a/mlir/docs/Rationale/RationaleLinalgDialect.md +++ b/mlir/docs/Rationale/RationaleLinalgDialect.md @@ -551,7 +551,7 @@ non-Linalg operations, for example `linalg.*conv*`, `linalg.winograd*`, With multiple (often exchangeable) forms, and with transformation simplicity in mind, compilers should aim for reducing matching and replacing complexity as much as possible. When matching a single operation with a complex pattern, -having all the information from a `generic` is useful to iteratively match +having all the information in a `generic` is useful to iteratively match different patterns in turn. However, when assembling a DAG of operations to form a pattern, it's much simpler to match against named operations (like `max` + `div` + `reduce` + `broadcast`) than their generic counterparts. From 6f161cd002135936fc26edb81632ac887a40999d Mon Sep 17 00:00:00 2001 From: Renato Golin Date: Mon, 8 Sep 2025 14:42:44 +0100 Subject: [PATCH 4/7] Apply suggestions from code review MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Andrzej Warzyński --- mlir/docs/Rationale/RationaleLinalgDialect.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mlir/docs/Rationale/RationaleLinalgDialect.md b/mlir/docs/Rationale/RationaleLinalgDialect.md index c337bc82193b9..c573085acc339 100644 --- a/mlir/docs/Rationale/RationaleLinalgDialect.md +++ b/mlir/docs/Rationale/RationaleLinalgDialect.md @@ -516,7 +516,7 @@ the more special forms should not be converted to the more generic ones unnecessarily, in the same way that they should not be broken down into loops + arithmetic if they can still be represented as a Linalg op. -#### Generic, Category, Named +#### The Linalg Forms The core Linalg operation tree has three forms: * **Generic:** Represented by `linalg.generic` and can encode all perfectly-nested @@ -528,7 +528,7 @@ _generic_ forms. For example, `linalg.matmul`, `linalg.add`, etc. Unlike lowering to loops, the different Linalg forms that are derived from `linalg.generic` are *equivalent*. It should always be possible to convert -a named operation into a generic and back to named, if the semantics is +a named operation into a generic and back to named, if the semantics are preserved. The various forms in the Linalg dialect are meant to facilitate pattern matching (single operations or DAGs) and to be able to consider different forms as *canonical* for different transforms. From e746daa3f538ec468f9a2cf5fa526d1b0026ffc9 Mon Sep 17 00:00:00 2001 From: Renato Golin Date: Sun, 14 Sep 2025 21:36:30 +0100 Subject: [PATCH 5/7] Addressed forms discussion, simplified whole section --- mlir/docs/Rationale/RationaleLinalgDialect.md | 35 ++++++++----------- 1 file changed, 14 insertions(+), 21 deletions(-) diff --git a/mlir/docs/Rationale/RationaleLinalgDialect.md b/mlir/docs/Rationale/RationaleLinalgDialect.md index c573085acc339..fe75474e6f5d3 100644 --- a/mlir/docs/Rationale/RationaleLinalgDialect.md +++ b/mlir/docs/Rationale/RationaleLinalgDialect.md @@ -508,21 +508,14 @@ and LLVMIR. ### Interchangeability of Forms -Linalg's various forms (named, generic) also carry information, and that -information should be preserved as much as possible during the progressive -lowering. A `matmul` operation is a special case of a `contract` operation, -which in turn is a special case of `generic` operation. Transformations on -the more special forms should not be converted to the more generic ones -unnecessarily, in the same way that they should not be broken down into -loops + arithmetic if they can still be represented as a Linalg op. - #### The Linalg Forms The core Linalg operation tree has three forms: * **Generic:** Represented by `linalg.generic` and can encode all perfectly-nested loop operations. * **Category:** Represented by `linalg.contract` and `linalg.elementwise`, -which are special (einsum) forms of the `generic` operation. +which are special (einsum) forms of the `generic` operation. In the future, other +category operations are planned (e.g.: `linalg.convolution` and `linalg.pooling`). * **Named:** All _named_ forms that can lower to either _category_ or _generic_ forms. For example, `linalg.matmul`, `linalg.add`, etc. @@ -533,18 +526,18 @@ preserved. The various forms in the Linalg dialect are meant to facilitate pattern matching (single operations or DAGs) and to be able to consider different forms as *canonical* for different transforms. -#### Special Operations - -Not all Linalg operations represent perfectly nested loops, and therefore -cannot be represented as a `linalg.generic`. There are two kinds of Linalg -operations that fall into this category: -* **Composite:** Operations that compose multiple Linalg operations, for -example `linalg.softmax`. These can be converted to a DAG of Linalg operations -(in any form). -* **Special Named:** Operations that are usually matched against library calls -or special lowering, but can only be lowered to a combination of Linalg and -non-Linalg operations, for example `linalg.*conv*`, `linalg.winograd*`, -`linalg.pooling*`, etc. +In addition to the three forms above, there's a separate class that does not +belong to the tree, as it does not generalize. These are **composite:** operations +that compose multiple Linalg operations, for example `linalg.softmax` and +`linalg.winograd*`. These can be converted to a DAG of Linalg operations. + +Linalg's various forms (named, generic) also carry information, and that +information should be preserved as much as possible during the progressive +lowering. A `matmul` operation is a special case of a `contract` operation, +which in turn is a special case of `generic` operation. Transformations on +the more special forms should not be converted to the more generic ones +unnecessarily, in the same way that they should not be broken down into +loops + arithmetic if they can still be represented as a Linalg op. #### Canonical Forms From 31f352fc6ca5db07a882d1471f20d6340021919b Mon Sep 17 00:00:00 2001 From: Renato Golin Date: Wed, 1 Oct 2025 13:44:07 +0100 Subject: [PATCH 6/7] rewriting the forms sub-section --- mlir/docs/Rationale/RationaleLinalgDialect.md | 38 +++++++++++-------- 1 file changed, 22 insertions(+), 16 deletions(-) diff --git a/mlir/docs/Rationale/RationaleLinalgDialect.md b/mlir/docs/Rationale/RationaleLinalgDialect.md index fe75474e6f5d3..1f0fff3a2a01f 100644 --- a/mlir/docs/Rationale/RationaleLinalgDialect.md +++ b/mlir/docs/Rationale/RationaleLinalgDialect.md @@ -513,25 +513,31 @@ and LLVMIR. The core Linalg operation tree has three forms: * **Generic:** Represented by `linalg.generic` and can encode all perfectly-nested loop operations. -* **Category:** Represented by `linalg.contract` and `linalg.elementwise`, -which are special (einsum) forms of the `generic` operation. In the future, other -category operations are planned (e.g.: `linalg.convolution` and `linalg.pooling`). -* **Named:** All _named_ forms that can lower to either _category_ or -_generic_ forms. For example, `linalg.matmul`, `linalg.add`, etc. - -Unlike lowering to loops, the different Linalg forms that are derived from -`linalg.generic` are *equivalent*. It should always be possible to convert -a named operation into a generic and back to named, if the semantics are -preserved. The various forms in the Linalg dialect are meant to facilitate +* **Category:** For example, `linalg.contract` and `linalg.elementwise`, that encode +higher-level semantics of a `linalg.generic` while still representing multiple _named_ +operations via attributes and syntax. In the future, other category operations are +planned (e.g.: `linalg.convolution` and `linalg.pooling`). +* **Named:** For example, `linalg.matmul`, `linalg.add`, etc. All _named_ forms that +can be converted to either a single _category_ or _generic_ forms, ie. are _perfectly nested_. +* **Composite:** For example `linalg.softmax` and the `winograd` variations. These +operations are not perfectly nested, and are converted to a list of other operations +(of various dialects). + +The different Linalg forms that are derived from `linalg.generic` are *equivalent*. +It should always be possible to convert a **named** operation into a **category** and that +into a **generic** and back to **named**. However, it may not be possible to convert a +**generic** into a **named** if there is no such named form. + +**Composite** operations cannot be converted to the other three classes and forms a +sub-set on its own. But they can use other Linalg forms when expanding. There can be +a pattern-matching transform to detect a graph of operations and convert into a +**composite** operation. + +The various forms in the Linalg dialect are meant to facilitate pattern matching (single operations or DAGs) and to be able to consider different forms as *canonical* for different transforms. -In addition to the three forms above, there's a separate class that does not -belong to the tree, as it does not generalize. These are **composite:** operations -that compose multiple Linalg operations, for example `linalg.softmax` and -`linalg.winograd*`. These can be converted to a DAG of Linalg operations. - -Linalg's various forms (named, generic) also carry information, and that +Linalg's various forms also carry information, and that information should be preserved as much as possible during the progressive lowering. A `matmul` operation is a special case of a `contract` operation, which in turn is a special case of `generic` operation. Transformations on From 1d4933dbf79fa767ad6bcf62e59cdda1dd4fb362 Mon Sep 17 00:00:00 2001 From: Renato Golin Date: Wed, 1 Oct 2025 15:34:02 +0100 Subject: [PATCH 7/7] Fixed outdated line, add the op tree for clarification. --- mlir/docs/Rationale/RationaleLinalgDialect.md | 22 +++++++++++++------ 1 file changed, 15 insertions(+), 7 deletions(-) diff --git a/mlir/docs/Rationale/RationaleLinalgDialect.md b/mlir/docs/Rationale/RationaleLinalgDialect.md index 1f0fff3a2a01f..fcef0b4ed788d 100644 --- a/mlir/docs/Rationale/RationaleLinalgDialect.md +++ b/mlir/docs/Rationale/RationaleLinalgDialect.md @@ -510,7 +510,7 @@ and LLVMIR. #### The Linalg Forms -The core Linalg operation tree has three forms: +The core Linalg operation set has four forms: * **Generic:** Represented by `linalg.generic` and can encode all perfectly-nested loop operations. * **Category:** For example, `linalg.contract` and `linalg.elementwise`, that encode @@ -523,15 +523,23 @@ can be converted to either a single _category_ or _generic_ forms, ie. are _perf operations are not perfectly nested, and are converted to a list of other operations (of various dialects). -The different Linalg forms that are derived from `linalg.generic` are *equivalent*. -It should always be possible to convert a **named** operation into a **category** and that -into a **generic** and back to **named**. However, it may not be possible to convert a -**generic** into a **named** if there is no such named form. +The forms correlate in the following manner: +``` ++ generic + \__ + category + \__ + named ++ composite +``` -**Composite** operations cannot be converted to the other three classes and forms a +The `category` and `named` forms are derived from `linalg.generic` and are *equivalent*. +It should always be possible to convert a `named` operation into a `category` and that +into a `generic` and back to `named`. However, it may not be possible to convert a +`generic` into a `named` if there is no such `named` form. + +`Composite` operations cannot be converted to the other three classes and forms a sub-set on its own. But they can use other Linalg forms when expanding. There can be a pattern-matching transform to detect a graph of operations and convert into a -**composite** operation. +`composite` operation. The various forms in the Linalg dialect are meant to facilitate pattern matching (single operations or DAGs) and to be able to consider