Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TypeError with torch.nn.functional.pad #458

Closed
Tracked by #266
athitten opened this issue May 24, 2024 · 1 comment · Fixed by #459
Closed
Tracked by #266

TypeError with torch.nn.functional.pad #458

athitten opened this issue May 24, 2024 · 1 comment · Fixed by #459
Labels
bug Something isn't working nemo Issues needed to support NVIDIA NeMo models.

Comments

@athitten
Copy link

athitten commented May 24, 2024 •
edited by tfogal
Loading

🐛 Bug

NeMo's Stable Diffusion uses torch.nn.functional.pad as below:

x = torch.nn.functional.pad(x, pad, mode="constant", value=0)

When run with thunder.jit, it gives a TypeError:
Expected the numbertype <class 'float'>, corresponding to the dtype float32, but found the numbertype <class 'int'>!
although torch.nn.functional.pad supports value=0 along with float value=0.0

CC: @tfogal
@athitten athitten added the bug Something isn't working label May 24, 2024
tfogal added a commit that referenced this issue May 24, 2024
@tfogal
pad: support alternate value types.
00b49a9 
PyTorch actually allows the `value` argument to have a different
type then the input type, as long as the type of `value` is
convertiable to the type of `input`. This contradicts the
documentation at:

   https://pytorch.org/docs/stable/generated/torch.nn.functional.pad.html

which indicates that the padding value, if specified, must be a
float.

Fixes #458.
tfogal added a commit that referenced this issue May 24, 2024
@tfogal
pad: support alternate value types.
34550eb 
PyTorch actually allows the `value` argument to have a different
type then the input type, as long as the type of `value` is
convertible to the type of `input`. This contradicts the
documentation at:

   https://pytorch.org/docs/stable/generated/torch.nn.functional.pad.html

which indicates that the padding value, if specified, must be a
float.

Fixes #458.
tfogal added a commit that referenced this issue May 24, 2024
@tfogal
pad: support alternate value types.
cb9f786 
PyTorch actually allows the `value` argument to have a different
type than the input type, as long as the type of `value` is
convertible to the type of `input`. This contradicts the
documentation at:

   https://pytorch.org/docs/stable/generated/torch.nn.functional.pad.html

which indicates that the padding value, if specified, must be a
float.

Fixes #458.
@tfogal tfogal mentioned this issue May 24, 2024
Merged
@tfogal
Copy link
Collaborator

tfogal commented May 24, 2024

Thanks, @athitten !
If you have a minute sometime, it would be helpful for you to verify that #459 actually fixes the context that you hit this in NeMo.

@tfogal tfogal added the nemo Issues needed to support NVIDIA NeMo models. label May 24, 2024
@tfogal tfogal mentioned this issue May 24, 2024
Open
10 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working nemo Issues needed to support NVIDIA NeMo models.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

pad: support alternate value types. Lightning-AI/lightning-thunder
2 participants
@tfogal @athitten