Experimental prototype for converting torch.jit.trace modules to export #124449

tugsbayasgalan · 2024-04-19T02:32:49Z

Stack from ghstack (oldest at bottom):

-> Experimental prototype for converting torch.jit.trace modules to export #124449

Differential Revision: D56440613

We want to do this for following reasons:

There is current limitation in export tracing for torch.jit.trace d modules that cannot be easily upstreamed
We need to run internal CI regularly to understand feature gaps and continuously track them
Multiple people will be working on this prototype so it is better to have a checked in version so we don't always run into merge conflicts.

cc @ezyang @msaroufim @bdhirsh @anijain2305 @chauhang @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @kadeng

…gnature [ghstack-poisoned]

pytorch-bot · 2024-04-19T02:32:52Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/124449

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (6 Unrelated Failures)

As of commit 6828070 with merge base 92eb173 ():

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

UNSTABLE - The following jobs failed but were likely due to flakiness present on trunk and has been marked as unstable:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

tugsbayasgalan · 2024-04-22T21:41:50Z

@tugsbayasgalan has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

tugsbayasgalan · 2024-04-22T21:48:09Z

@tugsbayasgalan has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

tugsbayasgalan · 2024-04-22T23:07:52Z

@tugsbayasgalan has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

tugsbayasgalan · 2024-04-23T21:08:18Z

@tugsbayasgalan has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

…gnature ghstack-source-id: 3aaf2a3f3ba4638aef5ce2101e8b22a8a57f0bc8 Pull Request resolved: #124449

…ave code/signature" Differential Revision: [D56440613](https://our.internmc.facebook.com/intern/diff/D56440613) [ghstack-poisoned]

facebook-github-bot · 2024-04-30T21:50:45Z

This pull request was exported from Phabricator. Differential Revision: D56440613

…ave code/signature" Differential Revision: [D56440613](https://our.internmc.facebook.com/intern/diff/D56440613) [ghstack-poisoned]

@voznesenskym

…gnature Pull Request resolved: #124449 @imported-using-ghimport Differential Revision: [D56440613](https://our.internmc.facebook.com/intern/diff/D56440613/) ghstack-source-id: d823a86cff475c1785f140140b49f548a8b921f9 Some update Pull Request resolved: #124450 cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @kadeng @chauhang @imported-using-ghimport Differential Revision: [D56440611](https://our.internmc.facebook.com/intern/diff/D56440611/) ghstack-source-id: d823a86cff475c1785f140140b49f548a8b921f9 Make CI less noisy Pull Request resolved: #124664 cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @kadeng @chauhang @imported-using-ghimport Differential Revision: [D56440612](https://our.internmc.facebook.com/intern/diff/D56440612/) ghstack-source-id: d823a86cff475c1785f140140b49f548a8b921f9

…les to export" Differential Revision: [D56440613](https://our.internmc.facebook.com/intern/diff/D56440613) cc ezyang msaroufim bdhirsh anijain2305 chauhang voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng [ghstack-poisoned]

@voznesenskym

…gnature Pull Request resolved: #124449 @imported-using-ghimport Differential Revision: [D56440613](https://our.internmc.facebook.com/intern/diff/D56440613/) ghstack-source-id: 39182d5d313da5d4d0b6b30fa8e94d974499276b Some update Pull Request resolved: #124450 cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @kadeng @chauhang @imported-using-ghimport Differential Revision: [D56440611](https://our.internmc.facebook.com/intern/diff/D56440611/) ghstack-source-id: 39182d5d313da5d4d0b6b30fa8e94d974499276b Make CI less noisy Pull Request resolved: #124664 cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @kadeng @chauhang @imported-using-ghimport Differential Revision: [D56440612](https://our.internmc.facebook.com/intern/diff/D56440612/) ghstack-source-id: 39182d5d313da5d4d0b6b30fa8e94d974499276b

…les to export" Differential Revision: [D56440613](https://our.internmc.facebook.com/intern/diff/D56440613) cc ezyang msaroufim bdhirsh anijain2305 chauhang voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng [ghstack-poisoned]

@voznesenskym

…gnature Pull Request resolved: #124449 @imported-using-ghimport Differential Revision: [D56440613](https://our.internmc.facebook.com/intern/diff/D56440613/) ghstack-source-id: 44bafb981fef6340a542a5abf8c27fe186a91c8b Some update Pull Request resolved: #124450 cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @kadeng @chauhang @imported-using-ghimport Differential Revision: [D56440611](https://our.internmc.facebook.com/intern/diff/D56440611/) ghstack-source-id: 44bafb981fef6340a542a5abf8c27fe186a91c8b Make CI less noisy Pull Request resolved: #124664 cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @kadeng @chauhang @imported-using-ghimport Differential Revision: [D56440612](https://our.internmc.facebook.com/intern/diff/D56440612/) ghstack-source-id: 44bafb981fef6340a542a5abf8c27fe186a91c8b

…les to export" Differential Revision: [D56440613](https://our.internmc.facebook.com/intern/diff/D56440613) We want to do this for following reasons: 1. There is current limitation in export tracing for torch.jit.trace d modules that cannot be easily upstreamed 2. We need to run internal CI regularly to understand feature gaps and continuously track them 3. Multiple people will be working on this prototype so it is better to have a checked in version so we don't always run into merge conflicts. cc ezyang msaroufim bdhirsh anijain2305 chauhang voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng [ghstack-poisoned]

@voznesenskym

…gnature Pull Request resolved: #124449 @imported-using-ghimport Differential Revision: [D56440613](https://our.internmc.facebook.com/intern/diff/D56440613/) ghstack-source-id: 550e93f774b6f6aa665eee31ff42c15c80907cfb Some update Pull Request resolved: #124450 cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @kadeng @chauhang @imported-using-ghimport Differential Revision: [D56440611](https://our.internmc.facebook.com/intern/diff/D56440611/) ghstack-source-id: 550e93f774b6f6aa665eee31ff42c15c80907cfb Make CI less noisy Pull Request resolved: #124664 cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @kadeng @chauhang @imported-using-ghimport Differential Revision: [D56440612](https://our.internmc.facebook.com/intern/diff/D56440612/) ghstack-source-id: 550e93f774b6f6aa665eee31ff42c15c80907cfb

tugsbayasgalan · 2024-05-15T04:40:14Z

@tugsbayasgalan has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

…les to export" Differential Revision: [D56440613](https://our.internmc.facebook.com/intern/diff/D56440613) We want to do this for following reasons: 1. There is current limitation in export tracing for torch.jit.trace d modules that cannot be easily upstreamed 2. We need to run internal CI regularly to understand feature gaps and continuously track them 3. Multiple people will be working on this prototype so it is better to have a checked in version so we don't always run into merge conflicts. cc ezyang msaroufim bdhirsh anijain2305 chauhang voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng [ghstack-poisoned]

@voznesenskym

…gnature Pull Request resolved: #124449 @imported-using-ghimport Differential Revision: [D56440613](https://our.internmc.facebook.com/intern/diff/D56440613/) ghstack-source-id: c5a0b72a295f371bdb1e0e55601e867c5ec41962 Some update Pull Request resolved: #124450 cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @kadeng @chauhang @imported-using-ghimport Differential Revision: [D56440611](https://our.internmc.facebook.com/intern/diff/D56440611/) ghstack-source-id: c5a0b72a295f371bdb1e0e55601e867c5ec41962 Make CI less noisy Pull Request resolved: #124664 cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @kadeng @chauhang @imported-using-ghimport Differential Revision: [D56440612](https://our.internmc.facebook.com/intern/diff/D56440612/) ghstack-source-id: c5a0b72a295f371bdb1e0e55601e867c5ec41962

tugsbayasgalan · 2024-05-15T17:12:00Z

@tugsbayasgalan has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

angelayi · 2024-05-16T15:41:40Z

torch/_export/non_strict_utils.py

@@ -153,11 +148,22 @@ def make_fake_inputs(nn_module, args, kwargs, dynamic_shapes):
            len(constraints) == 0
        ), "Found constraints when tracing with a toplevel tracing context."
        fake_mode = context.fake_mode
-    else:
+    elif not _is_torch_jit_trace:


can we leave a comment like "HACK" for all the workarounds made in this PR

…les to export" Differential Revision: [D56440613](https://our.internmc.facebook.com/intern/diff/D56440613) We want to do this for following reasons: 1. There is current limitation in export tracing for torch.jit.trace d modules that cannot be easily upstreamed 2. We need to run internal CI regularly to understand feature gaps and continuously track them 3. Multiple people will be working on this prototype so it is better to have a checked in version so we don't always run into merge conflicts. cc ezyang msaroufim bdhirsh anijain2305 chauhang voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng [ghstack-poisoned]

@voznesenskym

…gnature Pull Request resolved: #124449 @imported-using-ghimport Differential Revision: [D56440613](https://our.internmc.facebook.com/intern/diff/D56440613/) ghstack-source-id: c056aa3cce08a23bf3271fd5a27f8c94baf109fa Some update Pull Request resolved: #124450 cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @kadeng @chauhang @imported-using-ghimport Differential Revision: [D56440611](https://our.internmc.facebook.com/intern/diff/D56440611/) ghstack-source-id: c056aa3cce08a23bf3271fd5a27f8c94baf109fa Make CI less noisy Pull Request resolved: #124664 cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @kadeng @chauhang @imported-using-ghimport Differential Revision: [D56440612](https://our.internmc.facebook.com/intern/diff/D56440612/) ghstack-source-id: c056aa3cce08a23bf3271fd5a27f8c94baf109fa

tugsbayasgalan · 2024-05-16T18:29:49Z

@tugsbayasgalan has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

avikchaudhuri · 2024-05-17T16:42:05Z

torch/_export/non_strict_utils.py

+        )
+    else:
+        # FIXME(ycao): This is a hack to get around missing signature from ScriptMethod
+        msg = "dummy constraint violation message"


Wonder if it's better to just have msg = "" here, analogous to other zero-d out things in this mode. It might even be better to have a top-level dict of all the things we're zero-ing out, something like _torch_jit_trace_hacks = {"msg": "", "original_signature": None, ...} etc. so that we have a good account of all the FIXMEs.

facebook-github-bot · 2024-05-17T20:40:36Z

@pytorchbot merge -f 'Landed internally'

(Initiating merge automatically since Phabricator Diff has merged, using force because this PR might not pass merge_rules.json but landed internally)

pytorchmergebot · 2024-05-17T20:42:29Z

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use -f as last resort and instead consider -i/--ignore-current to continue the merge ignoring current failures. This will allow currently pending tests to finish and report signal before the merge.

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

…rt (pytorch#124449) Differential Revision: [D56440613](https://our.internmc.facebook.com/intern/diff/D56440613) We want to do this for following reasons: 1. There is current limitation in export tracing for torch.jit.trace d modules that cannot be easily upstreamed 2. We need to run internal CI regularly to understand feature gaps and continuously track them 3. Multiple people will be working on this prototype so it is better to have a checked in version so we don't always run into merge conflicts. Pull Request resolved: pytorch#124449 Approved by: https://github.com/angelayi, https://github.com/avikchaudhuri

Hacks to work around the fact that ScriptMethod does not have code/si…

9772368

…gnature [ghstack-poisoned]

tugsbayasgalan requested review from avikchaudhuri, gmagogsfm and zhxchen17 as code owners April 19, 2024 02:32

pytorch-bot bot added the ciflow/inductor label Apr 19, 2024

This was referenced Apr 19, 2024

Some update #124450

Open

Make the CI failures less noicy #124558

Open

Make CI less noisy #124664

Open

angelayi pushed a commit that referenced this pull request Apr 25, 2024

Hacks to work around the fact that ScriptMethod does not have code/si…

57d2f6f

…gnature ghstack-source-id: 3aaf2a3f3ba4638aef5ce2101e8b22a8a57f0bc8 Pull Request resolved: #124449

angelayi pushed a commit that referenced this pull request Apr 25, 2024

Hacks to work around the fact that ScriptMethod does not have code/si…

633797c

…gnature ghstack-source-id: 3aaf2a3f3ba4638aef5ce2101e8b22a8a57f0bc8 Pull Request resolved: #124449

Update on "Hacks to work around the fact that ScriptMethod does not h…

835a94d

…ave code/signature" Differential Revision: [D56440613](https://our.internmc.facebook.com/intern/diff/D56440613) [ghstack-poisoned]

This was referenced Apr 29, 2024

temp #125164

Open

ignore verify placeholders #125165

Open

fix more invalid inputs, remove fuse #125166

Open

Disable running before and update error message to be less verbose #125167

Open

Enable running before #125168

Open

Enable fuse #125169

Open

Update on "Hacks to work around the fact that ScriptMethod does not h…

1b7f392

…ave code/signature" Differential Revision: [D56440613](https://our.internmc.facebook.com/intern/diff/D56440613) [ghstack-poisoned]

facebook-github-bot added the fb-exported label Apr 30, 2024

tugsbayasgalan added 2 commits April 30, 2024 15:36

Update on "Hacks to work around the fact that ScriptMethod does not h…

dd400fc

…ave code/signature" Differential Revision: [D56440613](https://our.internmc.facebook.com/intern/diff/D56440613) [ghstack-poisoned]

Update on "Hacks to work around the fact that ScriptMethod does not h…

798579f

…ave code/signature" Differential Revision: [D56440613](https://our.internmc.facebook.com/intern/diff/D56440613) [ghstack-poisoned]

tugsbayasgalan requested review from mikaylagawarecki and jeffdaily as code owners April 30, 2024 22:39

pytorch-bot bot added the module: dynamo label Apr 30, 2024

tugsbayasgalan changed the title ~~Hacks to work around the fact that ScriptMethod does not have code/signature~~ Experimental prototype for converting torch.jit.trace modules to export May 15, 2024

tugsbayasgalan requested review from SherlockNoMad, angelayi and jiashenC May 15, 2024 04:32

angelayi reviewed May 16, 2024

View reviewed changes

angelayi approved these changes May 16, 2024

View reviewed changes

pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label May 16, 2024

avikchaudhuri approved these changes May 17, 2024

View reviewed changes

pytorchmergebot added the merging label May 17, 2024

pytorchmergebot added the Merged label May 17, 2024

pytorchmergebot closed this in bed1c60 May 17, 2024

pytorchmergebot removed the merging label May 17, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Experimental prototype for converting torch.jit.trace modules to export #124449

Experimental prototype for converting torch.jit.trace modules to export #124449

tugsbayasgalan commented Apr 19, 2024 •

edited

pytorch-bot bot commented Apr 19, 2024 •

edited

tugsbayasgalan commented Apr 22, 2024

tugsbayasgalan commented Apr 22, 2024

tugsbayasgalan commented Apr 22, 2024

tugsbayasgalan commented Apr 23, 2024

facebook-github-bot commented Apr 30, 2024

tugsbayasgalan commented May 15, 2024

tugsbayasgalan commented May 15, 2024

angelayi May 16, 2024 •

edited

tugsbayasgalan commented May 16, 2024

avikchaudhuri May 17, 2024 •

edited

facebook-github-bot commented May 17, 2024

pytorchmergebot commented May 17, 2024

Experimental prototype for converting torch.jit.trace modules to export #124449

Experimental prototype for converting torch.jit.trace modules to export #124449

Conversation

tugsbayasgalan commented Apr 19, 2024 • edited

pytorch-bot bot commented Apr 19, 2024 • edited

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/124449

✅ You can merge normally! (6 Unrelated Failures)

tugsbayasgalan commented Apr 22, 2024

tugsbayasgalan commented Apr 22, 2024

tugsbayasgalan commented Apr 22, 2024

tugsbayasgalan commented Apr 23, 2024

facebook-github-bot commented Apr 30, 2024

tugsbayasgalan commented May 15, 2024

tugsbayasgalan commented May 15, 2024

angelayi May 16, 2024 • edited

Choose a reason for hiding this comment

tugsbayasgalan commented May 16, 2024

avikchaudhuri May 17, 2024 • edited

Choose a reason for hiding this comment

facebook-github-bot commented May 17, 2024

pytorchmergebot commented May 17, 2024

Merge started

tugsbayasgalan commented Apr 19, 2024 •

edited

pytorch-bot bot commented Apr 19, 2024 •

edited

angelayi May 16, 2024 •

edited

avikchaudhuri May 17, 2024 •

edited