Create a constant to initializer pass #2156
Assignees
Comments
justinchuby
changed the title
|
Maybe call it constant manipulation or something |
|
If we make this pass, should we have it before model.optimze() or after model.optimize() in the middle of exporting? Where we usually do graph pass is in _core.py (before optimize): |
|
it should be called in optimize not PyTorch I think. It’s a pass to mend the behavior of optimize() |
titaiwangms
added a commit
to microsoft/onnxruntime
that referenced
this issue
Apr 4, 2025
### Description <!-- Describe your changes. --> Essentially, the vision model is traced differently (this time it's without mask.), and the input indices of op.Add and op.MatMul can be different. Also, fp16 and fp32 need different tracing patterns (op.Cast). 1. Add another traced pattern to CLIP attention to cover no attention_mask case 2. Accept different index of input on op.Add and op.MatMul (be more general) 3. fp16 and fp32 shows different pattern (op.Cast after op.Softmax) 4. Refactor test_fastgelu.py to cover torch.onnx.export(..., dynamo=True) 5. Add gemma3 vision attention (SigLip) test to cover both fp16 and fp32 ### Motivation and Context <!-- - Why is this change required? What problem does it solve? - If it fixes an open issue, please link to the issue here. --> To optimize Gemma3 multi-modal model, the changes are needed. https://huggingface.co/google/gemma-3-4b-it NOTE: some related follow-ups (upstream optimizations to onnxscript-optimizer): microsoft/onnxscript#2158 microsoft/onnxscript#2156
justinchuby
added a commit
that referenced
this issue
Apr 10, 2025
Fix #2156 --------- Co-authored-by: Justin Chu <justinchuby@users.noreply.github.com>
quic-zhaoxul
pushed a commit
to CodeLinaro/onnxruntime
that referenced
this issue
Apr 17, 2025
### Description <!-- Describe your changes. --> Essentially, the vision model is traced differently (this time it's without mask.), and the input indices of op.Add and op.MatMul can be different. Also, fp16 and fp32 need different tracing patterns (op.Cast). 1. Add another traced pattern to CLIP attention to cover no attention_mask case 2. Accept different index of input on op.Add and op.MatMul (be more general) 3. fp16 and fp32 shows different pattern (op.Cast after op.Softmax) 4. Refactor test_fastgelu.py to cover torch.onnx.export(..., dynamo=True) 5. Add gemma3 vision attention (SigLip) test to cover both fp16 and fp32 ### Motivation and Context <!-- - Why is this change required? What problem does it solve? - If it fixes an open issue, please link to the issue here. --> To optimize Gemma3 multi-modal model, the changes are needed. https://huggingface.co/google/gemma-3-4b-it NOTE: some related follow-ups (upstream optimizations to onnxscript-optimizer): microsoft/onnxscript#2158 microsoft/onnxscript#2156
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
https://gist.github.com/justinchuby/cf1699d05baeac281fb3e82f9d0fc473
The text was updated successfully, but these errors were encountered: