[Shape Inference] Add shape inference for MatMulIntegerToFloat op - #31199
Open
roberto-laudani wants to merge 1 commit into
Open
[Shape Inference] Add shape inference for MatMulIntegerToFloat op#31199roberto-laudani wants to merge 1 commit into
roberto-laudani wants to merge 1 commit into
Conversation
### Description Add a `MatMulIntegerToFloat` rule to the `SymbolicShapeInference` dispatcher, mirroring the type and shape inference function registered for the op's schema: the output element type is propagated from `a_scale` and the output shape is the matmul shape of the first two inputs. ### Motivation and Context `MatMulIntegerToFloat` is defined in the `com.microsoft` domain, so `onnx.shape_inference` cannot infer it and `symbolic_shape_infer.py` has to supply the rule. Without a dispatcher entry its output keeps an `UNDEFINED` element type and `infer_shapes()` raises `Incomplete symbolic shape inference` for the whole model at the first such node. Same class of gap as microsoft#21246 (MatMulNBits) and microsoft#24090 (QLinearAdd/QLinearMul).
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
Author
|
@microsoft-github-policy-service agree |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add a shape inference rule for
MatMulIntegerToFloattosymbolic_shape_infer.py. The output type comes froma_scaleand the shape from the first two inputs, same as the op's schema does.Motivation and Context
MatMulIntegerToFloatis acom.microsoftop, soonnx.shape_inferencecannot infer it on its own. Without a rule the output type staysUNDEFINED.infer_shapes()then stops at the firstMatMulIntegerToFloatnode and raisesIncomplete symbolic shape inference.