-
Notifications
You must be signed in to change notification settings - Fork 25.6k
[Inductor][fx pass] Add split cat pattern to remove cat nodes #115004
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/115004
Note: Links to docs will display an error until the docs builds have been completed. ✅ You can merge normally! (2 Unrelated Failures)As of commit f8858b8 with merge base 389b56b ( BROKEN TRUNK - The following jobs failed but were present on the merge base:👉 Rebase onto the `viable/strict` branch to avoid these failures
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
This pull request was exported from Phabricator. Differential Revision: D51777617 |
This PR needs a
|
764a8ac
to
f8d21e3
Compare
…h#115004) Summary: Titled Test Plan: # unit test ``` buck2 test 'fbcode//mode/dev-nosan' fbcode//caffe2/test/inductor:split_cat_fx_passes ``` Buck UI: https://www.internalfb.com/buck2/8e4179db-363a-41b5-8bd7-cc445a512f6f Test UI: https://www.internalfb.com/intern/testinfra/testrun/15762598708548039 Network: Up: 91KiB Down: 32KiB (reSessionID-b0985d82-1919-49c5-b307-ee0ab49b4738) Jobs completed: 28. Time elapsed: 1:27.1s. Cache hits: 0%. Commands: 2 (cached: 0, remote: 0, local: 2) Tests finished: Pass 11. Fail 0. Fatal 0. Skip 0. Build failure 0 # local reproduce (IG_CTR) ``` buck2 run mode/opt //scripts/jackiexu0313/pt2:local_model_with_pt2 -- --test_mode split_batch ``` P895047189 Differential Revision: D51777617
This pull request was exported from Phabricator. Differential Revision: D51777617 |
f8d21e3
to
3a991f6
Compare
…h#115004) Summary: Titled Test Plan: # unit test ``` buck2 test 'fbcode//mode/dev-nosan' fbcode//caffe2/test/inductor:split_cat_fx_passes ``` Buck UI: https://www.internalfb.com/buck2/8e4179db-363a-41b5-8bd7-cc445a512f6f Test UI: https://www.internalfb.com/intern/testinfra/testrun/15762598708548039 Network: Up: 91KiB Down: 32KiB (reSessionID-b0985d82-1919-49c5-b307-ee0ab49b4738) Jobs completed: 28. Time elapsed: 1:27.1s. Cache hits: 0%. Commands: 2 (cached: 0, remote: 0, local: 2) Tests finished: Pass 11. Fail 0. Fatal 0. Skip 0. Build failure 0 # local reproduce (IG_CTR) ``` buck2 run mode/opt //scripts/jackiexu0313/pt2:local_model_with_pt2 -- --test_mode split_batch ``` P895047189 Differential Revision: D51777617
This pull request was exported from Phabricator. Differential Revision: D51777617 |
3a991f6
to
5cb342a
Compare
Summary: Titled Test Plan: # unit test ``` buck2 test 'fbcode//mode/dev-nosan' fbcode//caffe2/test/inductor:split_cat_fx_passes ``` Buck UI: https://www.internalfb.com/buck2/8e4179db-363a-41b5-8bd7-cc445a512f6f Test UI: https://www.internalfb.com/intern/testinfra/testrun/15762598708548039 Network: Up: 91KiB Down: 32KiB (reSessionID-b0985d82-1919-49c5-b307-ee0ab49b4738) Jobs completed: 28. Time elapsed: 1:27.1s. Cache hits: 0%. Commands: 2 (cached: 0, remote: 0, local: 2) Tests finished: Pass 11. Fail 0. Fatal 0. Skip 0. Build failure 0 # local reproduce (IG_CTR) ``` buck2 run mode/opt //scripts/jackiexu0313/pt2:local_model_with_pt2 -- --test_mode split_batch ``` P895047189 Differential Revision: D51777617
This pull request was exported from Phabricator. Differential Revision: D51777617 |
5cb342a
to
958e026
Compare
This pull request was exported from Phabricator. Differential Revision: D51777617 |
Summary: Titled Test Plan: # unit test ``` buck2 test 'fbcode//mode/dev-nosan' fbcode//caffe2/test/inductor:split_cat_fx_passes ``` Buck UI: https://www.internalfb.com/buck2/c3a4f237-a159-4802-9467-9372bb5c8cd5 Test UI: https://www.internalfb.com/intern/testinfra/testrun/1688850080636090 Network: Up: 502KiB Down: 4.4MiB (reSessionID-afb98e1b-ea92-42d5-bf1f-c67e5f6ac5a6) Jobs completed: 28. Time elapsed: 2:15.4s. Cache hits: 0%. Commands: 2 (cached: 0, remote: 0, local: 2) Tests finished: Pass 11. Fail 0. Fatal 0. Skip 0. Build failure 0 diffing: https://www.internalfb.com/intern/diffing/?paste_number=1148590250 # local reproduce ``` buck2 run mode/opt //scripts/jackiexu0313/pt2:local_model_with_pt2 -- --test_mode split_batch ``` IG_CTR: P895047189 CMF+FIRST: f525443546 Differential Revision: D51777617
958e026
to
05158a5
Compare
This pull request was exported from Phabricator. Differential Revision: D51777617 |
05158a5
to
e410b17
Compare
…h#115004) Summary: Titled Test Plan: # unit test ``` buck2 test 'fbcode//mode/dev-nosan' fbcode//caffe2/test/inductor:split_cat_fx_passes ``` Buck UI: https://www.internalfb.com/buck2/c3a4f237-a159-4802-9467-9372bb5c8cd5 Test UI: https://www.internalfb.com/intern/testinfra/testrun/1688850080636090 Network: Up: 502KiB Down: 4.4MiB (reSessionID-afb98e1b-ea92-42d5-bf1f-c67e5f6ac5a6) Jobs completed: 28. Time elapsed: 2:15.4s. Cache hits: 0%. Commands: 2 (cached: 0, remote: 0, local: 2) Tests finished: Pass 11. Fail 0. Fatal 0. Skip 0. Build failure 0 diffing: https://www.internalfb.com/intern/diffing/?paste_number=1148590250 # local reproduce ``` buck2 run mode/opt //scripts/jackiexu0313/pt2:local_model_with_pt2 -- --test_mode split_batch ``` IG_CTR: P895047189 CMF+FIRST: baseline: f531170799 proposal: f530913677 # hacky way to test the qps gain We try to test the qps gain by removing big "cat" nodes, here we mutate the cat inputs, NE is not guaranteed, just for test purpose P1182736031 before:https://www.internalfb.com/intern/everpaste/?color=0&handle=GByPmhalaFWCx84AAIKuIt-Xnbcmbr0LAAAz after: https://www.internalfb.com/intern/everpaste/?color=0&handle=GJutQBnLBCghuSgBADU1BjMlvRlPbr0LAAAz ### Results: https://pxl.cl/4m96s trace before transformation: https://pxl.cl/4m96G trace after transformation: https://pxl.cl/4m96R Split cat portion: the green one are totally removed. Differential Revision: D51777617
This pull request was exported from Phabricator. Differential Revision: D51777617 |
e410b17
to
c23f726
Compare
This pull request was exported from Phabricator. Differential Revision: D51777617 |
c23f726
to
65ed89a
Compare
This pull request was exported from Phabricator. Differential Revision: D51777617 |
65ed89a
to
fab3285
Compare
This pull request was exported from Phabricator. Differential Revision: D51777617 |
fab3285
to
a2d0534
Compare
…h#115004) Summary: Titled Test Plan: # unit test ``` buck2 test 'fbcode//mode/dev-nosan' fbcode//caffe2/test/inductor:split_cat_fx_passes ``` Buck UI: https://www.internalfb.com/buck2/1daa9bce-34ba-4abf-b38c-59be356e9b8e Test UI: https://www.internalfb.com/intern/testinfra/testrun/13792273880515191 Network: Up: 126KiB Down: 519KiB (reSessionID-e9a05384-5a97-4ea7-9da5-eb24ea2de23e) Jobs completed: 333475. Time elapsed: 2:34.5s. Cache hits: 88%. Commands: 24 (cached: 21, remote: 1, local: 2) Tests finished: Pass 11. Fail 0. Fatal 0. Skip 0. Build failure 0 ``` buck2 test mode/opt mode/inplace caffe2/test/inductor/fb:test_split_cat_fx_passes_aten_fb ``` Buck UI: https://www.internalfb.com/buck2/6a2a28d9-ead4-4215-afc2-0e08b6bd4740 Test UI: https://www.internalfb.com/intern/testinfra/testrun/17169973601044411 Network: Up: 70KiB Down: 73KiB (reSessionID-df7760d4-392c-4360-8991-bde9e92c7534) Jobs completed: 30. Time elapsed: 2:55.2s. Tests finished: Pass 13. Fail 0. Fatal 0. Skip 0. Build failure 0 diffing: https://www.internalfb.com/intern/diffing/?paste_number=1148590250 # local reproduce ``` buck2 run mode/opt //scripts/jackiexu0313/pt2:local_model_with_pt2 -- --test_mode split_batch ``` IG_CTR: P895047189 CMF+FIRST: baseline: f531170799 proposal: f530913677 # MC local reproduce see D53791839 # hacky way to test the qps gain We try to test the qps gain by removing big "cat" nodes, here we mutate the cat inputs, NE is not guaranteed, just for test purpose P1182736031 before:https://www.internalfb.com/intern/everpaste/?color=0&handle=GByPmhalaFWCx84AAIKuIt-Xnbcmbr0LAAAz after: https://www.internalfb.com/intern/everpaste/?color=0&handle=GJutQBnLBCghuSgBADU1BjMlvRlPbr0LAAAz ### Results: https://pxl.cl/4m96s trace before transformation: https://pxl.cl/4m96G trace after transformation: https://pxl.cl/4m96R Split cat portion: the green one are totally removed. Reviewed By: jackiexu1992 Differential Revision: D51777617
This pull request was exported from Phabricator. Differential Revision: D51777617 |
a2d0534
to
1381eba
Compare
…h#115004) Summary: Pull Request resolved: pytorch#115004 Titled Test Plan: # unit test ``` buck2 test 'fbcode//mode/dev-nosan' fbcode//caffe2/test/inductor:split_cat_fx_passes ``` Buck UI: https://www.internalfb.com/buck2/1daa9bce-34ba-4abf-b38c-59be356e9b8e Test UI: https://www.internalfb.com/intern/testinfra/testrun/13792273880515191 Network: Up: 126KiB Down: 519KiB (reSessionID-e9a05384-5a97-4ea7-9da5-eb24ea2de23e) Jobs completed: 333475. Time elapsed: 2:34.5s. Cache hits: 88%. Commands: 24 (cached: 21, remote: 1, local: 2) Tests finished: Pass 11. Fail 0. Fatal 0. Skip 0. Build failure 0 ``` buck2 test mode/opt mode/inplace caffe2/test/inductor/fb:test_split_cat_fx_passes_aten_fb ``` Buck UI: https://www.internalfb.com/buck2/6a2a28d9-ead4-4215-afc2-0e08b6bd4740 Test UI: https://www.internalfb.com/intern/testinfra/testrun/17169973601044411 Network: Up: 70KiB Down: 73KiB (reSessionID-df7760d4-392c-4360-8991-bde9e92c7534) Jobs completed: 30. Time elapsed: 2:55.2s. Tests finished: Pass 13. Fail 0. Fatal 0. Skip 0. Build failure 0 diffing: https://www.internalfb.com/intern/diffing/?paste_number=1148590250 # local reproduce ``` buck2 run mode/opt //scripts/jackiexu0313/pt2:local_model_with_pt2 -- --test_mode split_batch ``` IG_CTR: P895047189 CMF+FIRST: baseline: f531170799 proposal: f530913677 # MC local reproduce see D53791839 # hacky way to test the qps gain We try to test the qps gain by removing big "cat" nodes, here we mutate the cat inputs, NE is not guaranteed, just for test purpose P1182736031 before:https://www.internalfb.com/intern/everpaste/?color=0&handle=GByPmhalaFWCx84AAIKuIt-Xnbcmbr0LAAAz after: https://www.internalfb.com/intern/everpaste/?color=0&handle=GJutQBnLBCghuSgBADU1BjMlvRlPbr0LAAAz ### Results: https://pxl.cl/4m96s trace before transformation: https://pxl.cl/4m96G trace after transformation: https://pxl.cl/4m96R Split cat portion: the green one are totally removed. Reviewed By: jackiexu1992 Differential Revision: D51777617
This pull request was exported from Phabricator. Differential Revision: D51777617 |
1381eba
to
f8858b8
Compare
@pytorchbot merge -f 'Landed internally' (Initiating merge automatically since Phabricator Diff has merged, using force because this PR might not pass merge_rules.json but landed internally) |
Merge startedYour change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Summary: Titled
Test Plan:
unit test
Buck UI: https://www.internalfb.com/buck2/8e4179db-363a-41b5-8bd7-cc445a512f6f
Test UI: https://www.internalfb.com/intern/testinfra/testrun/15762598708548039
Network: Up: 91KiB Down: 32KiB (reSessionID-b0985d82-1919-49c5-b307-ee0ab49b4738)
Jobs completed: 28. Time elapsed: 1:27.1s.
Cache hits: 0%. Commands: 2 (cached: 0, remote: 0, local: 2)
Tests finished: Pass 11. Fail 0. Fatal 0. Skip 0. Build failure 0
local reproduce (IG_CTR)
P895047189
Differential Revision: D51777617
cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov @ColinPeppler @amjames