-
Notifications
You must be signed in to change notification settings - Fork 21.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.
Already on GitHub? Sign in to your account
[ONNX] export() with dynamic shapes fails where dynamo_export(dynamic_shapes=True) succeeds #126607
Comments
This is what I get when run the repro. Specializing some dimensions would result in more efficient code, so it would be nice to either make guards calculation succeed in this case, or be able to ignore guards and treat this error as warning.
|
BTW, my real-world model that I wrote this repro after, does fail even when all dimensions are specified, or when dynamo_export is called directly:
attn_prob is undergoing transformations similar to view() I used in my example. Not sure what I should mark/check as size. |
With the latest PyTorch nightly (05/20), I am also getting more failures on Nemo unit tests when trying to run export() with all dimensions being dynamic with no min/max, compared to running dynamo_export() directly. |
Here, a repro case with fully dynamic Dims that used to work with last week PyTorch nightly (05/15), but fails with 05/20. dynamo_export works but export.export() fails :
|
So torch.export fails and torch.onnx.dynamo_export succeeds? |
Correct: torch.onnx.dynamo_export with dynamic_shapes=True succeeds, but if I have to use export.export.first, even if I specify all the axes as dynamic with no bounds (which should be equivalent to dynamic_shapes=True), it fails in export.export as it can't calculate bounds properly, even with strict=False. |
馃悰 Describe the bug
Here is an example where dynamo_export() succeeds when run directly on a Module with dynamic_shapes=True, but fails if I first call export.export() on it. It happens more with partial dynamic shapes, but also happens when I set all dimensions to be fully dynamic.
Currently I have to call export() first due to another bug (have to run decompositions manually).
So I am trying to pass an equivalent of 'dynamic_shapes=True' to export() and achieve the same result as when I call dynamo_export directly. This example does work if I specify all axes as dynamic, but fails if I only specify one:
Versions
PyTorch nightly 05/15/24
cc @avikchaudhuri @gmagogsfm @zhxchen17 @tugsbayasgalan @angelayi @suo @ydwu4
The text was updated successfully, but these errors were encountered: