Skip to content

Nvfuser opt in for decomposition #81134

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 6 commits into from
Closed

Nvfuser opt in for decomposition #81134

wants to merge 6 commits into from

Conversation

jjsjann123
Copy link
Collaborator

@jjsjann123 jjsjann123 commented Jul 8, 2022 •
edited
Loading

Regarding issues reported in #79246, we notice that bias decomposition from conv/linear could actually hurt perf, due to the overhead of compilation. This PR changes it to make decomposition an explicit opt-in from user to avoid these regressions.

@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Jul 8, 2022 •
edited
Loading

🔗 Helpful links

✅ No Failures (0 Pending)

As of commit 1338e4f (more details on the Dr. CI page):

Expand to see more

💚 💚 Looks good so far! There are no failures yet. 💚 💚

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

@facebook-github-bot facebook-github-bot added the oncall: jit Add this issue/PR to JIT oncall triage queue label Jul 8, 2022
@jjsjann123 jjsjann123 requested a review from davidberard98 July 8, 2022 20:05
@jjsjann123
Copy link
Collaborator Author

@pytorchbot rebase

@jjsjann123 jjsjann123 marked this pull request as ready for review July 8, 2022 20:44
@pytorchmergebot
Copy link
Collaborator

@pytorchbot successfully started a rebase job. Check the current status here

@pytorchmergebot
Copy link
Collaborator

Successfully rebased nvfuser_opt_in_for_decomposition onto refs/remotes/origin/master, please pull locally before adding more changes (for example, via git checkout nvfuser_opt_in_for_decomposition && git pull --rebase)

@pytorchmergebot pytorchmergebot force-pushed the nvfuser_opt_in_for_decomposition branch from 0481e83 to 022bfb4 Compare July 8, 2022 20:46
@cpuhrsch cpuhrsch added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Jul 8, 2022
@davidberard98
Copy link
Contributor

@jjsjann123 do you know if you have permissions to push branches to the pytorch/pytorch repo? I believe this is the requirement to be able to run torchbench CI

davidberard98
davidberard98 approved these changes Jul 11, 2022
View reviewed changes
Copy link
Contributor

@davidberard98 davidberard98 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me!

Opened a PR to run torchbench (#81292) just to see if it affects any other models, but I'm guessing that the microbenchmarks won't be affected because they are collected after the decomposition...

Also, it might be worth documenting these in the readme?

@jjsjann123
Copy link
Collaborator Author

looks good to me!

Opened a PR to run torchbench (#81292) just to see if it affects any other models, but I'm guessing that the microbenchmarks won't be affected because they are collected after the decomposition...

Also, it might be worth documenting these in the readme?

I think I have access to push to pytorch repo directly. I'll try that next time~~
Looks like my doc update wasn't checked-in 😢 Will update that~

@jjsjann123
Copy link
Collaborator Author

XLA failure seems unrelated. I'm merging this one.

@jjsjann123
Copy link
Collaborator Author

@pytorchbot merge

@pytorchmergebot
Copy link
Collaborator

@pytorchbot successfully started a merge job. Check the current status here

@pytorchmergebot
Copy link
Collaborator

Merge failed due to Refusing to merge as mandatory check(s) pull failed for rule superuser
Raised by https://github.com/pytorch/pytorch/actions/runs/2658310387

@davidberard98
Copy link
Contributor

@jjsjann123 fyi, the failing xla test is flaky so if it fails again we can just re-run the test and hope it passes.

@jjsjann123
Copy link
Collaborator Author

@pytorchbot merge

@pytorchmergebot
Copy link
Collaborator

@pytorchbot successfully started a merge job. Check the current status here

@github-actions
Copy link
Contributor

Hey @jjsjann123.
You've committed this PR, but it does not have both a 'release notes: ...' and 'topics: ...' label. Please add one of each to the PR. The 'release notes: ...' label should represent the part of PyTorch that this PR changes (fx, autograd, distributed, etc) and the 'topics: ...' label should represent the kind of PR it is (not user facing, new feature, bug fix, perf improvement, etc). The list of valid labels can be found here for the 'release notes: ...' and here for the 'topics: ...'.
For changes that are 'topic: not user facing' there is no need for a release notes label.

facebook-github-bot pushed a commit that referenced this pull request Jul 13, 2022
@jjsjann123 @facebook-github-bot
Nvfuser opt in for decomposition (#81134) (#81134)
acdf7f5 
Summary:
Regarding issues reported in #79246, we notice that bias decomposition from conv/linear could actually hurt perf, due to the overhead of compilation. This PR changes it to make decomposition an explicit opt-in from user to avoid these regressions.

Pull Request resolved: #81134
Approved by: https://github.com/davidberard98

Test Plan: contbuild & OSS CI, see https://hud.pytorch.org/commit/pytorch/pytorch/d3acbc821e4a3a29bf252f990a817b2103658a4c

Reviewed By: DanilBaibak

Differential Revision: D37813470

Pulled By: DanilBaibak

fbshipit-source-id: c6f27699d868e92e1e31232b5d7c94d4762530e6
jjsjann123 added a commit to jjsjann123/nvfuser that referenced this pull request Oct 29, 2022
@jjsjann123 @pytorchmergebot
Nvfuser opt in for decomposition (#81134)
2c4d5f7 
Regarding issues reported in #79246, we notice that bias decomposition from conv/linear could actually hurt perf, due to the overhead of compilation. This PR changes it to make decomposition an explicit opt-in from user to avoid these regressions.
Pull Request resolved: pytorch/pytorch#81134
Approved by: https://github.com/davidberard98
jjsjann123 added a commit to jjsjann123/nvfuser that referenced this pull request Nov 10, 2022
@jjsjann123 @pytorchmergebot
Nvfuser opt in for decomposition (#81134)
bdb896a 
Regarding issues reported in #79246, we notice that bias decomposition from conv/linear could actually hurt perf, due to the overhead of compilation. This PR changes it to make decomposition an explicit opt-in from user to avoid these regressions.
Pull Request resolved: pytorch/pytorch#81134
Approved by: https://github.com/davidberard98
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@davidberard98 davidberard98 davidberard98 approved these changes

Assignees
No one assigned
Labels
cla signed Merged oncall: jit Add this issue/PR to JIT oncall triage queue open source triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@jjsjann123 @facebook-github-bot @pytorchmergebot @davidberard98 @cpuhrsch @pytorchbot