Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

AOTAutograd export path does not support training graphs with parameters that do not receive gradients. #101192

Open
bdhirsh opened this issue May 11, 2023 · 0 comments
Open

AOTAutograd export path does not support training graphs with parameters that do not receive gradients. #101192

bdhirsh opened this issue May 11, 2023 · 0 comments
Labels
module: aotdispatch umbrella label for AOTAutograd issues module: pt2-dispatcher PT2 dispatcher-related issues (e.g., aotdispatch, functionalization, faketensor, custom-op, oncall: pt2 triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Comments

@bdhirsh
Copy link
Contributor

bdhirsh commented May 11, 2023 •
edited by pytorch-bot bot

See the comment here: today, the AOTAutograd export path expects that every parameter in the module you export will receive some sort of gradient. It's possible to create a case where this doesn't happen. We can support it, but I'd like to gauge how much request there is for this first, since it's more of an edge case.

Example:

def f(p1, p2):
    # p1 will never receive any gradients. p2 will though
    return p1.detach() * p2

cc @ezyang @msaroufim @wconstab @anijain2305 @zou3519
@bdhirsh bdhirsh added the module: aotdispatch umbrella label for AOTAutograd issues label May 11, 2023
@bdhirsh bdhirsh mentioned this issue May 11, 2023
Closed
@cpuhrsch cpuhrsch added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label May 12, 2023
@penguinwu penguinwu added module: pt2-dispatcher PT2 dispatcher-related issues (e.g., aotdispatch, functionalization, faketensor, custom-op, oncall: pt2 labels Nov 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
module: aotdispatch umbrella label for AOTAutograd issues module: pt2-dispatcher PT2 dispatcher-related issues (e.g., aotdispatch, functionalization, faketensor, custom-op, oncall: pt2 triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@cpuhrsch @penguinwu @bdhirsh