async fx compile #146135

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Closed

aorenste wants to merge 8 commits into gh/aorenste/214/base from gh/aorenste/214/head

Contributor

aorenste commented Jan 31, 2025 •

edited

Loading

Adds the ability to run the selected out-of-process fx compile scheme in async mode - where we kick off the compile and then run eagerly until the compile is finished.

Added a test which runs a tiny model in a loop making sure that we execute it both eagerly and then compiled.

Stack from ghstack (oldest at bottom):

-> async fx compile #146135

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @amjames @desertfire @chauhang @aakhundov

Differential Revision: D71135546


          WIP: async compile

e879ea1

[ghstack-poisoned]

aorenste mentioned this pull request

Subprocess compile #146134

Closed

pytorch-bot bot commented Jan 31, 2025 •

edited

Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/146135

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (2 Unrelated Failures)

As of commit 205f3e5 with merge base 224cd9f ():

UNSTABLE - The following jobs are marked as unstable, possibly due to flakiness on trunk:

pull / cuda12.4-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu) (gh) (#149370)
REGRESSION: benchmark ('aotdispatcher_training_subclass_cpu', 'compile_time_instruction_count') failed, actual result 9828029001 is 1.72% higher than expected 9662000000 ±+1.50% if this is an expected regression, please update the expected results.
pull / linux-jammy-py3-clang12-executorch / test (executorch, 1, 1, ephemeral.linux.2xlarge) (gh) (#144480)
Process completed with exit code 1.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

pytorch-bot bot added ciflow/inductor module: inductor labels


          Update on "WIP: async compile"

fefc367

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy yf225 chenyang78 kadeng muchulee8 amjames desertfire chauhang aakhundov

[ghstack-poisoned]

aorenste added a commit that referenced this pull request


          WIP: async compile

edd0488

ghstack-source-id: 9e79b08
Pull Request resolved: #146135


          Update on "WIP: async compile"

e1c2f74

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy yf225 chenyang78 kadeng muchulee8 amjames desertfire chauhang aakhundov

[ghstack-poisoned]

aorenste added a commit that referenced this pull request


          WIP: async compile

b2bbda4

ghstack-source-id: 38d2b3d
Pull Request resolved: #146135


          Update on "WIP: async compile"

e12d0f6

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy yf225 chenyang78 kadeng muchulee8 amjames desertfire chauhang aakhundov

[ghstack-poisoned]

aorenste added a commit that referenced this pull request


          WIP: async compile

d039f23

ghstack-source-id: 172e8d0
Pull Request resolved: #146135


          Update on "WIP: async compile"

8fc6c13

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy yf225 chenyang78 kadeng muchulee8 amjames desertfire chauhang aakhundov

[ghstack-poisoned]

aorenste added a commit that referenced this pull request


          WIP: async compile

d2b77c0

ghstack-source-id: 052cb8a
Pull Request resolved: #146135


          Update on "WIP: async compile"

0c1b997

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy yf225 chenyang78 kadeng muchulee8 amjames desertfire chauhang aakhundov

[ghstack-poisoned]

aorenste added a commit that referenced this pull request


          WIP: async compile

dd9dc6b

ghstack-source-id: 275d88c
Pull Request resolved: #146135


          Update on "WIP: async compile"

6c44b66

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy yf225 chenyang78 kadeng muchulee8 amjames desertfire chauhang aakhundov

[ghstack-poisoned]

aorenste added a commit that referenced this pull request


          WIP: async compile

ghstack-source-id: 01e6823
Pull Request resolved: #146135

aorenste added the topic: not user facing label

aorenste changed the title ~~WIP: async compile~~ async fx compile

aorenste marked this pull request as ready for review

March 12, 2025 04:09

aorenste requested review from jamesjwu, jansel and oulgen

March 13, 2025 18:45

Contributor Author

aorenste commented Mar 13, 2025

@aorenste has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

pytorch-bot bot added the ciflow/trunk label

jamesjwu approved these changes

View reviewed changes

Contributor

jamesjwu left a comment

Seems fine to me

test/inductor/test_compile_subprocess.py Show resolved Hide resolved

test/inductor/test_compile_subprocess.py Outdated Show resolved Hide resolved

torch/_inductor/compile_fx_ext.py Show resolved Hide resolved

torch/_inductor/compile_fx_async.py Outdated Show resolved Hide resolved

jansel approved these changes

View reviewed changes

test/inductor/test_compile_subprocess.py Outdated Show resolved Hide resolved

torch/_inductor/compile_fx_async.py Outdated Show resolved Hide resolved

torch/_inductor/compile_fx_async.py Outdated Show resolved Hide resolved


          Update on "async fx compile"

205f3e5

Adds the ability to run the selected out-of-process fx compile scheme in async mode - where we kick off the compile and then run eagerly until the compile is finished.

Added a test which runs a tiny model in a loop making sure that we execute it both eagerly and then compiled.




cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy yf225 chenyang78 kadeng muchulee8 amjames desertfire chauhang aakhundov

Differential Revision: [D71135546](https://our.internmc.facebook.com/intern/diff/D71135546)

[ghstack-poisoned]

aorenste added a commit that referenced this pull request


          WIP: async compile

340344f

ghstack-source-id: 975aea7
Pull Request resolved: #146135

Contributor Author

aorenste commented Mar 18, 2025

@aorenste has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

Contributor Author

aorenste commented Mar 19, 2025

@pytorchbot merge

pytorchmergebot added the merging label

Collaborator

pytorchmergebot commented Mar 19, 2025

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

pytorchmergebot added the Merged label

pytorchmergebot closed this in

2fcfae7

pytorchmergebot removed the merging label

github-actions bot deleted the gh/aorenste/214/head branch

April 22, 2025 02:14

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ciflow/inductor ciflow/trunk Merged module: inductor topic: not user facing