[PyTorch] Reporting OOM events to the Pytorch Profiler. #80050

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Closed

c-odrin wants to merge 1 commit into pytorch:master from c-odrin:export-D36268132

Contributor

c-odrin commented Jun 22, 2022

Summary: Similar to reporting alloc and dealloc events in the PyTorch profiler, we are now reporting Out of Memory events as well. This is useful for performance troubleshooting

Test Plan: Added test_oom_tracing to test/test_profiler.py

Differential Revision: D36268132

c-odrin requested review from albanD and soulitzer as code owners

June 22, 2022 17:26

facebook-github-bot added the cla signed label

Contributor

facebook-github-bot commented Jun 22, 2022 •

edited

Loading

🔗 Helpful links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/80050
📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓Need help or want to give feedback on the CI? Visit our office hours

✅ No Failures (0 Pending)

As of commit 5ee5774 (more details on the Dr. CI page):

Expand to see more

💚 💚 Looks good so far! There are no failures yet. 💚 💚

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

facebook-github-bot added the fb-exported label

Contributor

facebook-github-bot commented Jun 22, 2022

This pull request was exported from Phabricator. Differential Revision: D36268132

robieta self-requested a review

June 22, 2022 17:33

robieta approved these changes

View reviewed changes

robieta left a comment

LGTM provided the benchmark shows no regression and OSS tests pass.

Thanks a lot for adding this.

albanD removed their request for review

June 22, 2022 18:38

c-odrin force-pushed the export-D36268132 branch from df3a32c to bfc7ebc Compare

June 22, 2022 19:19

Contributor

facebook-github-bot commented Jun 22, 2022

This pull request was exported from Phabricator. Differential Revision: D36268132

1 similar comment

Contributor

facebook-github-bot commented Jun 22, 2022

This pull request was exported from Phabricator. Differential Revision: D36268132

c-odrin force-pushed the export-D36268132 branch from bfc7ebc to e1e1578 Compare

June 22, 2022 20:00

Contributor

facebook-github-bot commented Jun 22, 2022

This pull request was exported from Phabricator. Differential Revision: D36268132

c-odrin force-pushed the export-D36268132 branch from e1e1578 to 7c9d11e Compare

June 22, 2022 20:04

c-odrin force-pushed the export-D36268132 branch from 7c9d11e to 923a192 Compare

June 22, 2022 21:26

Contributor

facebook-github-bot commented Jun 22, 2022

This pull request was exported from Phabricator. Differential Revision: D36268132

c-odrin force-pushed the export-D36268132 branch from 923a192 to e0905ff Compare

July 8, 2022 19:24

Contributor

facebook-github-bot commented Jul 8, 2022

This pull request was exported from Phabricator. Differential Revision: D36268132

Contributor Author

c-odrin commented Jul 8, 2022

soulitzer can I get a review on this? Many thanks

c-odrin force-pushed the export-D36268132 branch from e0905ff to 3720cd4 Compare

July 8, 2022 19:33

Contributor

facebook-github-bot commented Jul 8, 2022

This pull request was exported from Phabricator. Differential Revision: D36268132

c-odrin force-pushed the export-D36268132 branch from 3720cd4 to 10f02f1 Compare

July 8, 2022 19:40

c-odrin force-pushed the export-D36268132 branch from c014335 to e598896 Compare

July 13, 2022 21:24

Contributor

facebook-github-bot commented Jul 14, 2022

This pull request was exported from Phabricator. Differential Revision: D36268132

c-odrin force-pushed the export-D36268132 branch from e598896 to 09e1371 Compare

July 14, 2022 14:59

Contributor

facebook-github-bot commented Jul 14, 2022

This pull request was exported from Phabricator. Differential Revision: D36268132

c-odrin force-pushed the export-D36268132 branch from 09e1371 to 9aad338 Compare

July 14, 2022 15:07

c-odrin requested a review from ilia-cher

July 15, 2022 01:21

Contributor

facebook-github-bot commented Jul 15, 2022

This pull request was exported from Phabricator. Differential Revision: D36268132

c-odrin force-pushed the export-D36268132 branch from 9aad338 to e27f5b5 Compare

July 15, 2022 01:24

Contributor

facebook-github-bot commented Jul 15, 2022

This pull request was exported from Phabricator. Differential Revision: D36268132

c-odrin force-pushed the export-D36268132 branch from e27f5b5 to 63babc1 Compare

July 15, 2022 10:57

Contributor

facebook-github-bot commented Jul 15, 2022

This pull request was exported from Phabricator. Differential Revision: D36268132

c-odrin force-pushed the export-D36268132 branch from 63babc1 to f225ae0 Compare

July 15, 2022 11:03

Contributor

facebook-github-bot commented Jul 19, 2022

This pull request was exported from Phabricator. Differential Revision: D36268132

c-odrin force-pushed the export-D36268132 branch from f225ae0 to 6a3b906 Compare

July 19, 2022 00:11


          [PyTorch] Reporting OOM events to the Pytorch Profiler. (pytorch#80050)

5ee5774

Summary:
Pull Request resolved: pytorch#80050

Similar to reporting alloc and dealloc events in the PyTorch profiler, we are now reporting Out of Memory events as well. This is useful for performance troubleshooting

Test Plan: Added test_oom_tracing to test/test_profiler.py

Reviewed By: robieta

Differential Revision: D36268132

fbshipit-source-id: 443adac06feaa498796302b2b49073d3e7c8ef91

Contributor

facebook-github-bot commented Jul 19, 2022

This pull request was exported from Phabricator. Differential Revision: D36268132

c-odrin force-pushed the export-D36268132 branch from 6a3b906 to 5ee5774 Compare

July 19, 2022 21:15

Contributor

facebook-github-bot commented Jul 20, 2022

@pytorchbot merge

(Initiating merge automatically since Phabricator Diff has merged)

Collaborator

pytorchmergebot commented Jul 20, 2022

@pytorchbot successfully started a merge job. Check the current status here

pytorchmergebot added the Merged label

pytorchmergebot closed this in

0aedda2

Contributor

github-actions bot commented Jul 20, 2022

Hey @c-odrin.
You've committed this PR, but it does not have both a 'release notes: ...' and 'topics: ...' label. Please add one of each to the PR. The 'release notes: ...' label should represent the part of PyTorch that this PR changes (fx, autograd, distributed, etc) and the 'topics: ...' label should represent the kind of PR it is (not user facing, new feature, bug fix, perf improvement, etc). The list of valid labels can be found here for the 'release notes: ...' and here for the 'topics: ...'.
For changes that are 'topic: not user facing' there is no need for a release notes label.

facebook-github-bot pushed a commit that referenced this pull request


          [PyTorch] Reporting OOM events to the Pytorch Profiler. (#80050)

15c78db

Summary:
Pull Request resolved: #80050

Similar to reporting alloc and dealloc events in the PyTorch profiler, we are now reporting Out of Memory events as well. This is useful for performance troubleshooting

Test Plan: Added test_oom_tracing to test/test_profiler.py

Reviewed By: robieta

Differential Revision: D36268132

fbshipit-source-id: 6f58e168a97a7a7f66c324af1ee573c04a13baf3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla signed fb-exported Merged