Changes to enable per channel support on dynamic linear. #37623
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Summary: Follows the same strategy as static linear. Same kernel now supports both per-channel and per-tensor linear. Fixed fully connected test. Test Plan: qnnpack tests q8gemm fully-connected-test Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]
This was referenced Apr 30, 2020
💊 CI failures summary and remediationsAs of commit 259e09c (more details on the Dr. CI page):
Extra GitHub checks: 1 failed
ci.pytorch.org: 1 failedThis comment was automatically generated by Dr. CI (expand for details).Follow this link to opt-out of these comments for your Pull Requests.Please report bugs/suggestions on the GitHub issue tracker or post in the (internal) Dr. CI Users group. This comment has been revised 123 times. |
Summary: Follows the same strategy as static linear. Same kernel now supports both per-channel and per-tensor linear. Fixed fully connected test. Test Plan: qnnpack tests q8gemm fully-connected-test Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D21339040](https://our.internmc.facebook.com/intern/diff/D21339040) [ghstack-poisoned]
Summary: Follows the same strategy as static linear. Same kernel now supports both per-channel and per-tensor linear. Fixed fully connected test. Test Plan: qnnpack tests q8gemm fully-connected-test Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D21339040](https://our.internmc.facebook.com/intern/diff/D21339040) [ghstack-poisoned]
Summary: Follows the same strategy as static linear. Same kernel now supports both per-channel and per-tensor linear. Fixed fully connected test. Test Plan: qnnpack tests q8gemm fully-connected-test Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D21339040](https://our.internmc.facebook.com/intern/diff/D21339040) [ghstack-poisoned]
Summary: Follows the same strategy as static linear. Same kernel now supports both per-channel and per-tensor linear. Fixed fully connected test. Test Plan: qnnpack tests q8gemm fully-connected-test Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D21339040](https://our.internmc.facebook.com/intern/diff/D21339040) [ghstack-poisoned]
Summary: Follows the same strategy as static linear. Same kernel now supports both per-channel and per-tensor linear. Fixed fully connected test. Test Plan: qnnpack tests q8gemm fully-connected-test Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D21339040](https://our.internmc.facebook.com/intern/diff/D21339040) [ghstack-poisoned]
Summary: Follows the same strategy as static linear. Same kernel now supports both per-channel and per-tensor linear. Fixed fully connected test. Test Plan: qnnpack tests q8gemm fully-connected-test Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D21339040](https://our.internmc.facebook.com/intern/diff/D21339040) [ghstack-poisoned]
Closed
Summary: Follows the same strategy as static linear. Same kernel now supports both per-channel and per-tensor linear. Fixed fully connected test. Test Plan: qnnpack tests q8gemm fully-connected-test Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D21339040](https://our.internmc.facebook.com/intern/diff/D21339040) [ghstack-poisoned]
Summary: Follows the same strategy as static linear. Same kernel now supports both per-channel and per-tensor linear. Fixed fully connected test. Test Plan: qnnpack tests q8gemm fully-connected-test Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D21339040](https://our.internmc.facebook.com/intern/diff/D21339040) [ghstack-poisoned]
Summary: Follows the same strategy as static linear. Same kernel now supports both per-channel and per-tensor linear. Fixed fully connected test. Test Plan: qnnpack tests q8gemm fully-connected-test Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D21339040](https://our.internmc.facebook.com/intern/diff/D21339040) [ghstack-poisoned]
Summary: Follows the same strategy as static linear. Same kernel now supports both per-channel and per-tensor linear. Fixed fully connected test. Test Plan: qnnpack tests q8gemm fully-connected-test Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D21339040](https://our.internmc.facebook.com/intern/diff/D21339040) [ghstack-poisoned]
Summary: Follows the same strategy as static linear. Same kernel now supports both per-channel and per-tensor linear. Fixed fully connected test. Test Plan: qnnpack tests q8gemm fully-connected-test Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D21339040](https://our.internmc.facebook.com/intern/diff/D21339040) [ghstack-poisoned]
Summary: Follows the same strategy as static linear. Supply pointer to an array of zero points and requantization scales. Thus same kernel now supports both per-channel and per-tensor linear. Test Plan: qnnpack tests q8gemm fully-connected-test Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D21339040](https://our.internmc.facebook.com/intern/diff/D21339040) [ghstack-poisoned]
xuezhou1998
pushed a commit
to xuezhou1998/new_pytorch
that referenced
this pull request
May 9, 2020
Summary: Follows the same strategy as static linear. Same kernel now supports both per-channel and per-tensor linear. Fixed fully connected test. Test Plan: qnnpack tests q8gemm fully-connected-test Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 93fdcb14221310b172caa1d2fa3976b7c088bd4b Pull Request resolved: pytorch/pytorch#37623
Summary: Follows the same strategy as static linear. Supply pointer to an array of zero points and requantization scales. Thus same kernel now supports both per-channel and per-tensor linear. Test Plan: qnnpack tests q8gemm fully-connected-test Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D21339040](https://our.internmc.facebook.com/intern/diff/D21339040) [ghstack-poisoned]
Summary: Follows the same strategy as static linear. Supply pointer to an array of zero points and requantization scales. Thus same kernel now supports both per-channel and per-tensor linear. Test Plan: qnnpack tests q8gemm fully-connected-test Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D21339040](https://our.internmc.facebook.com/intern/diff/D21339040) [ghstack-poisoned]
Summary: Follows the same strategy as static linear. Supply pointer to an array of zero points and requantization scales. Thus same kernel now supports both per-channel and per-tensor linear. Test Plan: qnnpack tests q8gemm fully-connected-test Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D21339040](https://our.internmc.facebook.com/intern/diff/D21339040) [ghstack-poisoned]
Summary: Follows the same strategy as static linear. Supply pointer to an array of zero points and requantization scales. Thus same kernel now supports both per-channel and per-tensor linear. Test Plan: qnnpack tests q8gemm fully-connected-test Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D21339040](https://our.internmc.facebook.com/intern/diff/D21339040) [ghstack-poisoned]
dreiss
reviewed
May 14, 2020
Comment on lines
259
to
261
| // TODO Kimish, we are allocating affine_quantized regardless of per channel or not. | ||
| // This allocation is actually used only for packing weight and thus will be freed. | ||
| // Still we should be consistent. Fix this. |
There was a problem hiding this comment.
Are the scale and offset even used here? Could this just be a normal CPU tensor of dtype uint8?
There was a problem hiding this comment.
Yes it can be. Let me do that after this stack lands.
dreiss
approved these changes
May 14, 2020
Summary: Follows the same strategy as static linear. Supply pointer to an array of zero points and requantization scales. Thus same kernel now supports both per-channel and per-tensor linear. Test Plan: qnnpack tests q8gemm fully-connected-test Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D21339040](https://our.internmc.facebook.com/intern/diff/D21339040) [ghstack-poisoned]
Summary: Follows the same strategy as static linear. Supply pointer to an array of zero points and requantization scales. Thus same kernel now supports both per-channel and per-tensor linear. Test Plan: qnnpack tests q8gemm fully-connected-test Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D21339040](https://our.internmc.facebook.com/intern/diff/D21339040) [ghstack-poisoned]
Summary: Follows the same strategy as static linear. Supply pointer to an array of zero points and requantization scales. Thus same kernel now supports both per-channel and per-tensor linear. Test Plan: qnnpack tests q8gemm fully-connected-test Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D21339040](https://our.internmc.facebook.com/intern/diff/D21339040) [ghstack-poisoned]
Summary: Follows the same strategy as static linear. Supply pointer to an array of zero points and requantization scales. Thus same kernel now supports both per-channel and per-tensor linear. Test Plan: qnnpack tests q8gemm fully-connected-test Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D21339040](https://our.internmc.facebook.com/intern/diff/D21339040) [ghstack-poisoned]
Summary: Follows the same strategy as static linear. Supply pointer to an array of zero points and requantization scales. Thus same kernel now supports both per-channel and per-tensor linear. Test Plan: qnnpack tests q8gemm fully-connected-test Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D21339040](https://our.internmc.facebook.com/intern/diff/D21339040) [ghstack-poisoned]
Summary: Follows the same strategy as static linear. Supply pointer to an array of zero points and requantization scales. Thus same kernel now supports both per-channel and per-tensor linear. Test Plan: qnnpack tests q8gemm fully-connected-test Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D21339040](https://our.internmc.facebook.com/intern/diff/D21339040) [ghstack-poisoned]
Summary: Follows the same strategy as static linear. Supply pointer to an array of zero points and requantization scales. Thus same kernel now supports both per-channel and per-tensor linear. Test Plan: qnnpack tests q8gemm fully-connected-test Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D21339040](https://our.internmc.facebook.com/intern/diff/D21339040) [ghstack-poisoned]
Summary: Follows the same strategy as static linear. Supply pointer to an array of zero points and requantization scales. Thus same kernel now supports both per-channel and per-tensor linear. Test Plan: qnnpack tests q8gemm fully-connected-test Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D21339040](https://our.internmc.facebook.com/intern/diff/D21339040) [ghstack-poisoned]
|
This pull request has been merged in 0a554ae. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Stack from ghstack:
Summary:
Follows the same strategy as static linear. Supply pointer to an array of zero points and requantization scales. Thus same kernel now supports both per-channel and per-tensor linear.
Test Plan:
qnnpack tests
q8gemm
fully-connected-test
Reviewers:
Subscribers:
Tasks:
Tags:
Differential Revision: D21339040