[quant][core][gpu][bux fix] Added clone and contiguous() to broadcasted_bias tensor in quantized cudnn linear op #75944
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ed_bias tensor in quantized cudnn linear op Summary: The previous implementation for broadcasted_bias in quantized cudnn linear op has 2 issues. 1) broadcasted_bias is a view of the the input bias tensor. This is not desired as any modifications to broadcasted_bias is also done to the input bias. To remedy this, we clone the input bias tensor. 2) Calling broadcast_to doesn't affect the storage, which is problematic for the cudnn operations. We need to create a fully broadcasted tensor, rather than a view (which is what's returned by broadcast_to). To remedy this, we call contiguous(). Test plan: python test/test_quantization.py -k test_linear_cudnn [ghstack-poisoned]
dzdang
added a commit
that referenced
this pull request
Apr 17, 2022
…ed_bias tensor in quantized cudnn linear op Summary: The previous implementation for broadcasted_bias in quantized cudnn linear op has 2 issues. 1) broadcasted_bias is a view of the the input bias tensor. This is not desired as any modifications to broadcasted_bias is also done to the input bias. To remedy this, we clone the input bias tensor. 2) Calling broadcast_to doesn't affect the storage, which is problematic for the cudnn operations. We need to create a fully broadcasted tensor, rather than a view (which is what's returned by broadcast_to). To remedy this, we call contiguous(). Test plan: python test/test_quantization.py -k test_linear_cudnn ghstack-source-id: 22cabed Pull Request resolved: #75944
🔗 Helpful links
💊 CI failures summary and remediationsAs of commit 18948a3 (more details on the Dr. CI page):
🕵️ 1 new failure recognized by patternsThe following CI failures do not appear to be due to upstream breakages
|
…o broadcasted_bias tensor in quantized cudnn linear op" Summary: The previous implementation for broadcasted_bias in quantized cudnn linear op has 2 issues. 1) broadcasted_bias is a view of the the input bias tensor. This is not desired as any modifications to broadcasted_bias is also done to the input bias. To remedy this, we clone the input bias tensor. 2) Calling broadcast_to doesn't affect the storage, which is problematic for the cudnn operations. We need to create a fully broadcasted tensor, rather than a view (which is what's returned by broadcast_to). To remedy this, we call contiguous(). Test plan: python test/test_quantization.py -k test_linear_cudnn [ghstack-poisoned]
…o broadcasted_bias tensor in quantized cudnn linear op" Summary: The previous implementation for broadcasted_bias in quantized cudnn linear op has 2 issues. 1) broadcasted_bias is a view of the the input bias tensor. This is not desired as any modifications to broadcasted_bias is also done to the input bias. To remedy this, we clone the input bias tensor. 2) Calling broadcast_to doesn't affect the storage, which is problematic for the cudnn operations. We need to create a fully broadcasted tensor, rather than a view (which is what's returned by broadcast_to). To remedy this, we call contiguous(). Test plan: python test/test_quantization.py -k test_linear_cudnn [ghstack-poisoned]
dzdang
added
release notes: quantization
|
@dzdang has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator. |
jerryzh168
approved these changes
Apr 18, 2022
There was a problem hiding this comment.
broadcast_to seems to be the same as expand, https://pytorch.org/docs/stable/generated/torch.broadcast_to.html?highlight=broadcast_to#torch.broadcast_to, I feel expand might be slightly more popular than broadcast_to, maybe we can use that
|
@dzdang has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator. |
|
@pytorchbot merge this (Initiating merge automatically since Phabricator Diff has merged) |
facebook-github-bot
pushed a commit
that referenced
this pull request
Apr 18, 2022
…ed_bias tensor in quantized cudnn linear op (#75944) Summary: Pull Request resolved: #75944 The previous implementation for broadcasted_bias in quantized cudnn linear op has 2 issues. 1) broadcasted_bias is a view of the the input bias tensor. This is not desired as any modifications to broadcasted_bias is also done to the input bias. To remedy this, we clone the input bias tensor. 2) Calling broadcast_to doesn't affect the storage, which is problematic for the cudnn operations. We need to create a fully broadcasted tensor, rather than a view (which is what's returned by broadcast_to). To remedy this, we call contiguous(). (Note: this ignores all push blocking failures!) Test Plan: python test/test_quantization.py -k test_linear_cudnn Reviewed By: jerryzh168 Differential Revision: D35717355 Pulled By: dzdang fbshipit-source-id: bc5e47666e4d0a8e1a544a094008520a290e5d25
malfet
pushed a commit
that referenced
this pull request
Apr 20, 2022
…ed_bias tensor in quantized cudnn linear op Summary: The previous implementation for broadcasted_bias in quantized cudnn linear op has 2 issues. 1) broadcasted_bias is a view of the the input bias tensor. This is not desired as any modifications to broadcasted_bias is also done to the input bias. To remedy this, we clone the input bias tensor. 2) Calling broadcast_to doesn't affect the storage, which is problematic for the cudnn operations. We need to create a fully broadcasted tensor, rather than a view (which is what's returned by broadcast_to). To remedy this, we call contiguous(). Test plan: python test/test_quantization.py -k test_linear_cudnn Pull Request resolved: #75944 Approved by: https://github.com/jerryzh168 (cherry picked from commit 381e725)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Stack from ghstack (oldest at bottom):
Summary:
The previous implementation for broadcasted_bias in quantized cudnn
linear op has 2 issues.
broadcasted_bias is a view of the the input bias tensor. This is not
desired as any modifications to broadcasted_bias is also done to the
input bias. To remedy this, we clone the input bias tensor.
Calling broadcast_to doesn't affect the storage, which is problematic
for the cudnn operations. We need to create a fully broadcasted tensor,
rather than a view (which is what's returned by broadcast_to). To remedy
this, we call contiguous().
Test plan:
python test/test_quantization.py -k test_linear_cudnn
Differential Revision: D35717355