add channels last with mixed data type support for GroupNorm backward #89485

CaoE · 2022-11-22T05:44:08Z

Stack from ghstack (oldest at bottom):

-> add channels last with mixed data type support for GroupNorm backward #89485

Motivation

Add channels last support for GroupNorm backward to make sure GroupNorm fully support channels last.
Same as add mixed data type support for GroupNorm backward on CPU #88663, mixed data type support is also needed for channels last implementation of GroupNorm backward.

Testing

Single socket (28cores):

Contiguous:

shape	forward / s	forward / s	backward / s	backward / s
	fp32	mixed fp32 bf16	fp32	mixed fp32 bf16
[10, 128, 20, 20]	3.20E-05	3.60E-05	8.31E-05	8.13E-05
[10, 128, 50, 50]	0.000126	0.000115	0.000356	0.000257

Channels Last:

shape	forward / s	forward / s	backward / s	backward / s
	fp32	mixed fp32 bf16	fp32	mixed fp32 bf16
[10, 128, 20, 20]	4.11E-05	4.12E-05	9.74E-05	9.66E-05
[10, 128, 50, 50]	0.000179	0.000178	0.000393	0.000317

Single core:

Contiguous:

shape	forward / s	forward / s	backward / s	backward / s
	fp32	mixed fp32 bf16	fp32	mixed fp32 bf16
[10, 128, 20, 20]	2.47E-04	2.53E-04	5.92E-04	4.50E-04
[10, 128, 50, 50]	0.001559	0.001384	0.004343	0.002436

Channels Last:

shape	forward / s	forward / s	backward / s	backward / s
	fp32	mixed fp32 bf16	fp32	mixed fp32 bf16
[10, 128, 20, 20]	2.27E-04	3.24E-04	0.0006224	0.000459
[10, 128, 50, 50]	0.00167	0.001278	0.0041858	0.003027

cc @ezyang @gchanan @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10 @VitalyFedyunin

pytorch-bot · 2022-11-22T05:44:11Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/89485

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit a8eb674:
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

ghstack-source-id: 1776f9ff792812955e00244bc25f03556e11dc85 Pull Request resolved: #89485

ghstack-source-id: dab8486ffbb576871851923259e660390b68c2c4 Pull Request resolved: #89485

ghstack-source-id: 8749cdc844892ba74b744a1d6250f553f934dcd8 Pull Request resolved: #89485

[ghstack-poisoned]

…rm backward" ### Motivation 1. Add channels last support for GroupNorm backward to make sure GroupNorm fully support channels last. 2. Same as #88663, mixed data type support is also needed for channels last implementation of GroupNorm backward. ### Testing Single socket (28cores): * Contiguous: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 3.20E-05 | 3.60E-05 | 8.31E-05 | 8.13E-05 [10, 128, 50, 50] | 0.000126 | 0.000115 | 0.000356 | 0.000257 * Channels Last: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 4.11E-05 | 4.12E-05 | 9.74E-05 | 9.66E-05 [10, 128, 50, 50] | 0.000179 | 0.000178 | 0.000393 | 0.000317 Single core: * Contiguous: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 2.47E-04 | 2.53E-04 | 5.92E-04 | 4.50E-04 [10, 128, 50, 50] | 0.001559 | 0.001384 | 0.004343 | 0.002436 * Channels Last: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 2.27E-04 | 3.24E-04 | 0.0006224 | 0.000459 [10, 128, 50, 50] | 0.00167 | 0.001278 | 0.0041858 | 0.003027 [ghstack-poisoned]

ghstack-source-id: 7fcba9e0c6a1bb4808e20fc802f899857112b080 Pull Request resolved: #89485

ghstack-source-id: 4549af4231508abefd9e443b2d0a13be2f9817df Pull Request resolved: #89485

…rm backward" ### Motivation 1. Add channels last support for GroupNorm backward to make sure GroupNorm fully support channels last. 2. Same as #88663, mixed data type support is also needed for channels last implementation of GroupNorm backward. ### Testing Single socket (28cores): * Contiguous: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 3.20E-05 | 3.60E-05 | 8.31E-05 | 8.13E-05 [10, 128, 50, 50] | 0.000126 | 0.000115 | 0.000356 | 0.000257 * Channels Last: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 4.11E-05 | 4.12E-05 | 9.74E-05 | 9.66E-05 [10, 128, 50, 50] | 0.000179 | 0.000178 | 0.000393 | 0.000317 Single core: * Contiguous: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 2.47E-04 | 2.53E-04 | 5.92E-04 | 4.50E-04 [10, 128, 50, 50] | 0.001559 | 0.001384 | 0.004343 | 0.002436 * Channels Last: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 2.27E-04 | 3.24E-04 | 0.0006224 | 0.000459 [10, 128, 50, 50] | 0.00167 | 0.001278 | 0.0041858 | 0.003027 [ghstack-poisoned]

ghstack-source-id: 065abc108b5be2e623ecb480a668ec27ac0d8d81 Pull Request resolved: #89485

…rm backward" ### Motivation 1. Add channels last support for GroupNorm backward to make sure GroupNorm fully support channels last. 2. Same as #88663, mixed data type support is also needed for channels last implementation of GroupNorm backward. ### Testing Single socket (28cores): * Contiguous: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 3.20E-05 | 3.60E-05 | 8.31E-05 | 8.13E-05 [10, 128, 50, 50] | 0.000126 | 0.000115 | 0.000356 | 0.000257 * Channels Last: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 4.11E-05 | 4.12E-05 | 9.74E-05 | 9.66E-05 [10, 128, 50, 50] | 0.000179 | 0.000178 | 0.000393 | 0.000317 Single core: * Contiguous: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 2.47E-04 | 2.53E-04 | 5.92E-04 | 4.50E-04 [10, 128, 50, 50] | 0.001559 | 0.001384 | 0.004343 | 0.002436 * Channels Last: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 2.27E-04 | 3.24E-04 | 0.0006224 | 0.000459 [10, 128, 50, 50] | 0.00167 | 0.001278 | 0.0041858 | 0.003027 cc VitalyFedyunin jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 [ghstack-poisoned]

ghstack-source-id: bda3595d20f04f7f4905b9d526fcd85c94521f84 Pull Request resolved: #89485

…rm backward" ### Motivation 1. Add channels last support for GroupNorm backward to make sure GroupNorm fully support channels last. 2. Same as #88663, mixed data type support is also needed for channels last implementation of GroupNorm backward. ### Testing Single socket (28cores): * Contiguous: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 3.20E-05 | 3.60E-05 | 8.31E-05 | 8.13E-05 [10, 128, 50, 50] | 0.000126 | 0.000115 | 0.000356 | 0.000257 * Channels Last: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 4.11E-05 | 4.12E-05 | 9.74E-05 | 9.66E-05 [10, 128, 50, 50] | 0.000179 | 0.000178 | 0.000393 | 0.000317 Single core: * Contiguous: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 2.47E-04 | 2.53E-04 | 5.92E-04 | 4.50E-04 [10, 128, 50, 50] | 0.001559 | 0.001384 | 0.004343 | 0.002436 * Channels Last: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 2.27E-04 | 3.24E-04 | 0.0006224 | 0.000459 [10, 128, 50, 50] | 0.00167 | 0.001278 | 0.0041858 | 0.003027 cc VitalyFedyunin jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 [ghstack-poisoned]

ghstack-source-id: 9bb35db126bb6e252de852646bf1b2435f6c80e6 Pull Request resolved: #89485

…rm backward" ### Motivation 1. Add channels last support for GroupNorm backward to make sure GroupNorm fully support channels last. 2. Same as #88663, mixed data type support is also needed for channels last implementation of GroupNorm backward. ### Testing Single socket (28cores): * Contiguous: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 3.20E-05 | 3.60E-05 | 8.31E-05 | 8.13E-05 [10, 128, 50, 50] | 0.000126 | 0.000115 | 0.000356 | 0.000257 * Channels Last: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 4.11E-05 | 4.12E-05 | 9.74E-05 | 9.66E-05 [10, 128, 50, 50] | 0.000179 | 0.000178 | 0.000393 | 0.000317 Single core: * Contiguous: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 2.47E-04 | 2.53E-04 | 5.92E-04 | 4.50E-04 [10, 128, 50, 50] | 0.001559 | 0.001384 | 0.004343 | 0.002436 * Channels Last: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 2.27E-04 | 3.24E-04 | 0.0006224 | 0.000459 [10, 128, 50, 50] | 0.00167 | 0.001278 | 0.0041858 | 0.003027 cc VitalyFedyunin jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 [ghstack-poisoned]

CaoE · 2022-12-23T01:00:59Z

@pytorchbot merge

pytorchmergebot · 2022-12-23T01:03:00Z

Merge failed

Reason: Not merging any PRs at the moment because there is a merge blocking https://github.com/pytorch/pytorch/labels/ci:%20sev issue open at:
#91332

Details for Dev Infra team

Raised by workflow job

ghstack-source-id: e117d1e993249764aafda981bfaa152346ab1398 Pull Request resolved: #89485

…rm backward" ### Motivation 1. Add channels last support for GroupNorm backward to make sure GroupNorm fully support channels last. 2. Same as #88663, mixed data type support is also needed for channels last implementation of GroupNorm backward. ### Testing Single socket (28cores): * Contiguous: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 3.20E-05 | 3.60E-05 | 8.31E-05 | 8.13E-05 [10, 128, 50, 50] | 0.000126 | 0.000115 | 0.000356 | 0.000257 * Channels Last: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 4.11E-05 | 4.12E-05 | 9.74E-05 | 9.66E-05 [10, 128, 50, 50] | 0.000179 | 0.000178 | 0.000393 | 0.000317 Single core: * Contiguous: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 2.47E-04 | 2.53E-04 | 5.92E-04 | 4.50E-04 [10, 128, 50, 50] | 0.001559 | 0.001384 | 0.004343 | 0.002436 * Channels Last: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 2.27E-04 | 3.24E-04 | 0.0006224 | 0.000459 [10, 128, 50, 50] | 0.00167 | 0.001278 | 0.0041858 | 0.003027 cc VitalyFedyunin jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 [ghstack-poisoned]

ghstack-source-id: fee09288d4da1ae17b0b18fc1ce0d4b8efdb6682 Pull Request resolved: #89485

ghstack-source-id: 95514958f41600c08ac65ff65338abf0039f3e42 Pull Request resolved: #89485

…rm backward" ### Motivation 1. Add channels last support for GroupNorm backward to make sure GroupNorm fully support channels last. 2. Same as #88663, mixed data type support is also needed for channels last implementation of GroupNorm backward. ### Testing Single socket (28cores): * Contiguous: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 3.20E-05 | 3.60E-05 | 8.31E-05 | 8.13E-05 [10, 128, 50, 50] | 0.000126 | 0.000115 | 0.000356 | 0.000257 * Channels Last: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 4.11E-05 | 4.12E-05 | 9.74E-05 | 9.66E-05 [10, 128, 50, 50] | 0.000179 | 0.000178 | 0.000393 | 0.000317 Single core: * Contiguous: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 2.47E-04 | 2.53E-04 | 5.92E-04 | 4.50E-04 [10, 128, 50, 50] | 0.001559 | 0.001384 | 0.004343 | 0.002436 * Channels Last: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 2.27E-04 | 3.24E-04 | 0.0006224 | 0.000459 [10, 128, 50, 50] | 0.00167 | 0.001278 | 0.0041858 | 0.003027 cc VitalyFedyunin jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 [ghstack-poisoned]

CaoE · 2022-12-29T05:21:45Z

@pytorchbot merge

pytorchmergebot · 2022-12-29T05:23:32Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

…rm backward" ### Motivation 1. Add channels last support for GroupNorm backward to make sure GroupNorm fully support channels last. 2. Same as #88663, mixed data type support is also needed for channels last implementation of GroupNorm backward. ### Testing Single socket (28cores): * Contiguous: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 3.20E-05 | 3.60E-05 | 8.31E-05 | 8.13E-05 [10, 128, 50, 50] | 0.000126 | 0.000115 | 0.000356 | 0.000257 * Channels Last: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 4.11E-05 | 4.12E-05 | 9.74E-05 | 9.66E-05 [10, 128, 50, 50] | 0.000179 | 0.000178 | 0.000393 | 0.000317 Single core: * Contiguous: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 2.47E-04 | 2.53E-04 | 5.92E-04 | 4.50E-04 [10, 128, 50, 50] | 0.001559 | 0.001384 | 0.004343 | 0.002436 * Channels Last: shape | forward / s | forward / s | backward / s | backward / s -- | -- | -- | -- | -- | fp32 | mixed fp32 bf16 | fp32 | mixed fp32 bf16 [10, 128, 20, 20] | 2.27E-04 | 3.24E-04 | 0.0006224 | 0.000459 [10, 128, 50, 50] | 0.00167 | 0.001278 | 0.0041858 | 0.003027 cc VitalyFedyunin jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 [ghstack-poisoned]

CaoE · 2022-12-29T07:46:13Z

@malfet Is there any adjustments that can be made in later PRs. Thanks for your any suggestions.

jgong5 · 2022-12-29T12:18:37Z

@malfet Is there any adjustments that can be made in later PRs. Thanks for your any suggestions.

@malfet In particular, may I know what code duplicate you were concerned about in this PR. We can address this in the follow-up PR. Thanks.

malfet · 2023-01-19T22:50:07Z

This PR regressed training step for https://github.com/lucidrains/DALLE2-pytorch, see #92166

CaoE · 2023-01-20T03:08:28Z

@malfet Sorry for the incontinence. Can we move the .grads[0].contiguous and input.contiguous from https://github.com/pytorch/pytorch/pull/89485/files#diff-e4c2f99a2404e98c3586e07425da73008f36b1bada790648a7297af141d37f8cL1171 to the native_group_norm_backward function in aten/src/ATen/native/group_norm.cpp and add memory format check for X and dY . In this way, we can neither break the channel last on CPU nor affect CUDA.

malfet · 2023-01-20T03:10:48Z

@malfet Sorry for the incontinence. Can we move the .grads[0].contiguous and input.contiguous from https://github.com/pytorch/pytorch/pull/89485/files#diff-e4c2f99a2404e98c3586e07425da73008f36b1bada790648a7297af141d37f8cL1171 to the native_group_norm_backward function in aten/src/ATen/native/group_norm.cpp and add memory format check for X and dY . In this way, we can neither break the channel last on CPU nor affect CUDA.

That's what I'm doing: forcing the continuous on the CUDA side, but also adding the unit test so it would not regress again

CaoE · 2023-01-20T03:14:49Z

@malfet Thank you very much ! I will submit a PR for the issue.

CaoE · 2023-01-20T04:57:18Z

@malfet I submitted a PR #92668 to fix it but I have no CUDA machine to verify if the issue is fixed. Could you please help to check it ?

Fixes regression introduced by #89485 Adds test to prevent those regressions from happening in the future In process, discovered that GroupNormBackwards on CPU does not produce the same results if input and gradient memory_format is different

malfet · 2023-01-20T06:00:12Z

@CaoE This sound very similar to the approach I've taken in #92671 and it fixes the problem for CUDA, but surprisingly I had to skip part of the test on CPU, as I think current implementation silently produces wrong results, but probably requires some further refactoring...

Fixes regression introduced by #89485 Adds test to prevent those regressions from happening in the future In process, discovered that GroupNormBackwards on CPU does not produce the same results if input and gradient memory_format is different Fixes #92166 Pull Request resolved: #92671 Approved by: https://github.com/ngimel, https://github.com/xuzhao9

CaoE requested review from albanD and soulitzer as code owners November 22, 2022 05:44

CaoE mentioned this pull request Nov 22, 2022

add mixed data type support for LayerNorm and GroupNorm #88662

Closed

pytorch-bot bot added the release notes: nn release notes category label Nov 22, 2022

CaoE mentioned this pull request Nov 22, 2022

add mixed data type support for GroupNorm backward on CPU #88663

Closed

github-actions bot added the module: cpu CPU specific problem (e.g., perf, algorithm) label Nov 22, 2022

CaoE added a commit that referenced this pull request Nov 22, 2022

add channels last with mixed data type support for GroupNorm backward

6c748db

ghstack-source-id: 1776f9ff792812955e00244bc25f03556e11dc85 Pull Request resolved: #89485

CaoE marked this pull request as draft November 22, 2022 05:45

pytorchbot added the open source label Nov 22, 2022

CaoE requested review from jgong5 and mingfeima November 22, 2022 05:55

CaoE added a commit that referenced this pull request Nov 22, 2022

add channels last with mixed data type support for GroupNorm backward

c0ba08f

ghstack-source-id: dab8486ffbb576871851923259e660390b68c2c4 Pull Request resolved: #89485

CaoE added a commit that referenced this pull request Nov 22, 2022

add channels last with mixed data type support for GroupNorm backward

8e46193

ghstack-source-id: 8749cdc844892ba74b744a1d6250f553f934dcd8 Pull Request resolved: #89485

CaoE added 3 commits November 22, 2022 12:42

add channels last with mixed data type support for GroupNorm backward

7d92503

[ghstack-poisoned]

CaoE added a commit that referenced this pull request Nov 23, 2022

add channels last with mixed data type support for GroupNorm backward

3632957

ghstack-source-id: 7fcba9e0c6a1bb4808e20fc802f899857112b080 Pull Request resolved: #89485

CaoE added a commit that referenced this pull request Nov 23, 2022

add channels last with mixed data type support for GroupNorm backward

7580cf1

ghstack-source-id: 4549af4231508abefd9e443b2d0a13be2f9817df Pull Request resolved: #89485

jgong5 approved these changes Nov 23, 2022

View reviewed changes

CaoE added 2 commits November 23, 2022 08:45

CaoE added the intel This tag is for PR from Intel label Nov 30, 2022

CaoE added a commit that referenced this pull request Dec 5, 2022

add channels last with mixed data type support for GroupNorm backward

5e6347d

ghstack-source-id: 065abc108b5be2e623ecb480a668ec27ac0d8d81 Pull Request resolved: #89485

CaoE added a commit that referenced this pull request Dec 9, 2022

add channels last with mixed data type support for GroupNorm backward

3fd6cad

ghstack-source-id: bda3595d20f04f7f4905b9d526fcd85c94521f84 Pull Request resolved: #89485

CaoE added the ciflow/trunk Trigger trunk jobs on your pull request label Dec 13, 2022

CaoE added a commit that referenced this pull request Dec 13, 2022

add channels last with mixed data type support for GroupNorm backward

2fe54ee

ghstack-source-id: 9bb35db126bb6e252de852646bf1b2435f6c80e6 Pull Request resolved: #89485

CaoE added the intel priority matters to intel architecture from performance wise label Dec 13, 2022

CaoE added 2 commits December 21, 2022 09:35

CaoE added a commit that referenced this pull request Dec 26, 2022

add channels last with mixed data type support for GroupNorm backward

0439d51

ghstack-source-id: e117d1e993249764aafda981bfaa152346ab1398 Pull Request resolved: #89485

CaoE added a commit that referenced this pull request Dec 28, 2022

add channels last with mixed data type support for GroupNorm backward

3916f5e

ghstack-source-id: fee09288d4da1ae17b0b18fc1ce0d4b8efdb6682 Pull Request resolved: #89485

CaoE added a commit that referenced this pull request Dec 28, 2022

add channels last with mixed data type support for GroupNorm backward

60d74c5

ghstack-source-id: 95514958f41600c08ac65ff65338abf0039f3e42 Pull Request resolved: #89485

pytorchmergebot added the Merged label Dec 29, 2022

pytorchmergebot closed this in 5030929 Dec 29, 2022

malfet added the module: bc-breaking Related to a BC-breaking change label Jan 19, 2023

pytorch-bot bot added the topic: bc breaking topic category label Jan 19, 2023

malfet mentioned this pull request Jan 20, 2023

Fix GroupNorm backward prop on CUDA #92671

Closed

CaoE mentioned this pull request Jan 28, 2023

Align input memory format and grad memory format for GroupNorm backward #92668

Closed

facebook-github-bot deleted the gh/CaoE/7/head branch June 8, 2023 14:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add channels last with mixed data type support for GroupNorm backward #89485

add channels last with mixed data type support for GroupNorm backward #89485

CaoE commented Nov 22, 2022 •

edited by pytorch-bot bot

pytorch-bot bot commented Nov 22, 2022 •

edited

CaoE commented Dec 23, 2022

pytorchmergebot commented Dec 23, 2022

CaoE commented Dec 29, 2022

pytorchmergebot commented Dec 29, 2022

CaoE commented Dec 29, 2022

jgong5 commented Dec 29, 2022

malfet commented Jan 19, 2023

CaoE commented Jan 20, 2023

malfet commented Jan 20, 2023

CaoE commented Jan 20, 2023 •

edited

CaoE commented Jan 20, 2023

malfet commented Jan 20, 2023

add channels last with mixed data type support for GroupNorm backward #89485

add channels last with mixed data type support for GroupNorm backward #89485

Conversation

CaoE commented Nov 22, 2022 • edited by pytorch-bot bot

Motivation

Testing

pytorch-bot bot commented Nov 22, 2022 • edited

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/89485

✅ No Failures

CaoE commented Dec 23, 2022

pytorchmergebot commented Dec 23, 2022

Merge failed

CaoE commented Dec 29, 2022

pytorchmergebot commented Dec 29, 2022

Merge started

CaoE commented Dec 29, 2022

jgong5 commented Dec 29, 2022

malfet commented Jan 19, 2023

CaoE commented Jan 20, 2023

malfet commented Jan 20, 2023

CaoE commented Jan 20, 2023 • edited

CaoE commented Jan 20, 2023

malfet commented Jan 20, 2023

CaoE commented Nov 22, 2022 •

edited by pytorch-bot bot

pytorch-bot bot commented Nov 22, 2022 •

edited

CaoE commented Jan 20, 2023 •

edited