DML EP add einsum MatMul NHCW ops #13440
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ignol/add-einsum-matmul-nhcw
fdwr
reviewed
Oct 25, 2022
| @@ -176,7 +252,7 @@ void CALLBACK QueryEinSum(IMLOperatorSupportQueryContextPrivate* context, bool* | |||
| EinSumHelper helper(attributes); | |||
| auto recognizedOperatorType = helper.GetRecognizedOperatorType(); | |||
There was a problem hiding this comment.
I forget if we have shape information at this time, but it would be useful to check if available. There's a comment above saying we don't support 4D in the DML EP, which means hypothetically an existing 5D EinSum that used to run (albeit falling back to CPU) will now fail to run. #Resolved
There was a problem hiding this comment.
Actually, IIRC, the shape of the tensor must be compatible in the expression, and so we do already have enough info to proceed safely, and the existing code should work as-is.
fdwr
changed the title
linnealovespie
pushed a commit
that referenced
this pull request
Oct 28, 2022
### Description This adds the "NHCW" format support for einsum MatMul. The logic is basically a merge of the existing Transpose and MatMul Einsum implementations. ### Motivation and Context Some transformer models that I'm tracking use Einsum quite often during a single inference, and about half of those were "NHCW" MatMul Einsums. Supporting them will reduce the number of copies to the CPU.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This adds the "NHCW" format support for einsum MatMul. The logic is basically a merge of the existing Transpose and MatMul Einsum implementations.
Motivation and Context
Some transformer models that I'm tracking use Einsum quite often during a single inference, and about half of those were "NHCW" MatMul Einsums. Supporting them will reduce the number of copies to the CPU.