Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Include hook's object name when profiling #11026

Merged
merged 15 commits into from Dec 20, 2021
Merged

Include hook's object name when profiling #11026

merged 15 commits into from Dec 20, 2021

Conversation

daniellepintz
Copy link
Contributor

@daniellepintz daniellepintz commented Dec 10, 2021 •
edited by Borda

What does this PR do?

When profiling, we should distinguish between a LM hook vs callback hook vs TTP hook.
Part of #8506

Does your PR introduce any breaking changes? If yes, please list them.

Before submitting

  • Was this discussed/approved via a GitHub issue? (not for typos and docs)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure your PR does only one thing, instead of bundling different changes together?
  • Did you make sure to update the documentation with your changes? (if necessary)
  • Did you write any new necessary tests? (not for typos and docs)
  • Did you verify new and existing tests pass locally with your changes?
  • Did you list all the breaking changes introduced by this pull request?
  • Did you update the CHANGELOG? (not for typos, docs, test updates, or internal minor changes/refactorings)

PR review

Anyone in the community is welcome to review the PR.
Before you start reviewing make sure you have read Review guidelines. In short, see the following bullet-list:

  • Is this pull request ready for review? (if not, please submit in draft mode)
  • Check that all items from Before submitting are resolved
  • Make sure the title is self-explanatory and the description concisely explains the PR
  • Add labels and milestones (and optionally projects) to the PR so it can be classified

Did you have fun?

Make sure you had fun coding 🙃

@mergify mergify bot removed the has conflicts label Dec 10, 2021
ananthsub
ananthsub reviewed Dec 10, 2021
View reviewed changes
Copy link
Contributor

@ananthsub ananthsub left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is really great to have more granular profiling available for debugging!

pytorch_lightning/trainer/trainer.py Outdated Show resolved Hide resolved
pytorch_lightning/trainer/trainer.py Outdated Show resolved Hide resolved
@ananthsub ananthsub added this to the 1.6 milestone Dec 10, 2021
ananthsub
ananthsub approved these changes Dec 10, 2021
View reviewed changes
tests/profiler/test_profiler.py Outdated Show resolved Hide resolved
@mergify mergify bot added the ready PRs ready to be merged label Dec 14, 2021
awaelchli
awaelchli reviewed Dec 14, 2021
View reviewed changes
Copy link
Member

@awaelchli awaelchli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry for nit, isn't it better to mention this in changelog?

tests/profiler/test_profiler.py Outdated Show resolved Hide resolved
carmocca
carmocca reviewed Dec 14, 2021
View reviewed changes
tests/profiler/test_profiler.py Outdated Show resolved Hide resolved
tests/profiler/test_profiler.py Outdated Show resolved Hide resolved
tests/profiler/test_profiler.py Show resolved Hide resolved
@daniellepintz
Copy link
Contributor Author

@awaelchli mention what in changelog? I do have an entry in the changelog

@awaelchli
Copy link
Member

I meant an entry under "changed" that the profiler now indexes and displays the names of the hooks with a new pattern (base class).(class).(hook name)

@daniellepintz
Copy link
Contributor Author

@awaelchli sure, I already have an entry under changed, but I can edit it

carmocca
carmocca previously requested changes Dec 17, 2021
View reviewed changes
Copy link
Member

@carmocca carmocca left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes so I can review before merge :), ping me when you update it!

@mergify mergify bot removed the ready PRs ready to be merged label Dec 17, 2021
@daniellepintz
Copy link
Contributor Author

daniellepintz commented Dec 17, 2021 •
edited

I'm really stumped on this test failure https://dev.azure.com/PytorchLightning/pytorch-lightning/_build/results?buildId=47638&view=logs&j=3afc50db-e620-5b81-6016-870a6976ad29&t=d9f671c5-a304-5675-5394-961fd7f98b9b

Could anyone who has worked with profilers before try taking a look? cc @kaushikb11 @carmocca @rohitgr7

I can't figure out why [TrainingTypePlugin]DDPPlugin.validation_step is not present in pytorch_profiler.function_events

@mergify mergify bot removed the has conflicts label Dec 17, 2021
ananthsub
ananthsub reviewed Dec 18, 2021
View reviewed changes
Copy link
Contributor

@ananthsub ananthsub left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

heads up, there will be conflicts between this and #11142 since the naming is changing
cc @four4fish

@daniellepintz
Copy link
Contributor Author

daniellepintz commented Dec 18, 2021 •
edited

@carmocca ready for your review! All tests passing. Thanks so much @rohitgr7 for the help!

carmocca
carmocca reviewed Dec 19, 2021
View reviewed changes
pytorch_lightning/trainer/trainer.py Outdated Show resolved Hide resolved
pytorch_lightning/profiler/pytorch.py Outdated Show resolved Hide resolved
@carmocca carmocca dismissed their stale review December 19, 2021 17:28

Main review posted

@mergify mergify bot added the ready PRs ready to be merged label Dec 19, 2021
@daniellepintz
Copy link
Contributor Author

Ready to be merged!

@carmocca carmocca merged commit b1baf46 into Lightning-AI:master Dec 20, 2021
@daniellepintz daniellepintz deleted the profile_hooks_separation branch December 20, 2021 17:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ananthsub ananthsub ananthsub approved these changes

@awaelchli awaelchli awaelchli approved these changes

@rohitgr7 rohitgr7 rohitgr7 approved these changes

@Borda Borda Borda approved these changes

@justusschock justusschock Awaiting requested review from justusschock

@kaushikb11 kaushikb11 Awaiting requested review from kaushikb11

@SeanNaren SeanNaren Awaiting requested review from SeanNaren

@williamFalcon williamFalcon Awaiting requested review from williamFalcon

@tchaton tchaton Awaiting requested review from tchaton tchaton is a code owner

@carmocca carmocca Awaiting requested review from carmocca

Assignees
No one assigned
Labels
profiler ready PRs ready to be merged
Projects
None yet
Milestone
pl:1.6
Development

Successfully merging this pull request may close these issues.

None yet
7 participants
@daniellepintz @awaelchli @ananthsub @Borda @carmocca @tchaton @rohitgr7