Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Logging suggestions upon Dynamo error do not log #100109

Closed
awgu opened this issue Apr 26, 2023 · 2 comments
Closed

Logging suggestions upon Dynamo error do not log #100109

awgu opened this issue Apr 26, 2023 · 2 comments
Assignees
Labels
oncall: pt2 triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Comments

@awgu
Copy link
Contributor

awgu commented Apr 26, 2023

When I hit an error during compilation, I see a message like:

torch._dynamo.exc.TorchRuntimeError: 

from user code:
   File "/.../transformers/src/transformers/models/t5/modeling_t5.py", line 559, in <resume in forward>
    scores += position_bias_masked

Set torch._dynamo.config.verbose=True or TORCHDYNAMO_VERBOSE=1 for more information

However, setting torch._dynamo.config.verbose=True and running with TORCHDYNAMO_VERBOSE=1 both do not print any logs (though, running with TORCH_LOGS=dynamo does).

  1. Is the suggestion in the error message out of date?
  2. Where can we find the current selection of logging options and how to enable them?
    Maybe https://pytorch.org/docs/stable/dynamo/troubleshooting.html is not up to date?

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh

@wconstab
Copy link
Contributor

yea, this is out of date info. You want TORCH_LOGS=+dynamo for debug, or drop the + for info. (note: if you type something wrong into TORCH_LOGS itll tell you the valid component options you can choose from, and i am just saying dynamo from memory but could be wrong)

@mlazos do you want to update the docs? or else i can help with that.

@mlazos
Copy link
Contributor

mlazos commented Apr 26, 2023

Yeah I can update the docs, I was planning on doing it for the docathon but I can start now since people are making mistakes with it.

Oh it's just the error message, I can PR that right now.

@mlazos mlazos self-assigned this Apr 26, 2023
@anijain2305 anijain2305 added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Apr 27, 2023
@mlazos mlazos linked a pull request Jun 2, 2023 that will close this issue
@mlazos mlazos closed this as completed Jul 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
oncall: pt2 triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants