SumKernel (BFloat16): use float as accumulation type #55217

mingfeima · 2021-04-02T06:53:35Z

Stack from ghstack:

add BFloat16 support for BatchNorm2d on CPU #61015 add BFloat16 support for BatchNorm2d on CPU
add BFloat16 support for UpSample on CPU #58297 add BFloat16 support for UpSample on CPU
optimize transpose copy for float32 and bfloat16 on CPU #56904 optimize transpose copy for float32 and bfloat16 on CPU
add BFloat16 support for MaxPool2d on CPU #56903 add BFloat16 support for MaxPool2d on CPU
add BFloat16 support for AdaptiveAvgPool2d on CPU #56902 add BFloat16 support for AdaptiveAvgPool2d on CPU
add BFloat16 support for bernoulli and Dropout on CPU #56372 add BFloat16 support for bernoulli and Dropout on CPU
add bf16 support for bucketize #55588 add bf16 support for bucketize
optimize BFloat16 elemwise operators CPU: sigmoid, sigmoid_backward, tanh_backward, addcmul, addcdiv #55221 optimize BFloat16 elemwise operators CPU: sigmoid, sigmoid_backward, tanh_backward, addcmul, addcdiv
SumKernel (BFloat16): use float as accumulation type #55217 SumKernel (BFloat16): use float as accumulation type

Differential Revision: D28836794

[ghstack-poisoned]

facebook-github-bot · 2021-04-02T06:53:42Z

💊 CI failures summary and remediations

As of commit 5b0a4c6 (more details on the Dr. CI page and at hud.pytorch.org/pr/55217):

💚 💚 Looks good so far! There are no failures yet. 💚 💚

Preview docs built from this PR

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

ghstack-source-id: a3e967e Pull Request resolved: #55217

mingfeima · 2021-04-02T06:57:08Z

use float as accumulation type when input is of BFloat16.

Since this PR is not related to parallelization feature, only single core perf is tested:

performance update on avx512 machine: Xeon(R) Gold 6248 CPU @ 2.50GHz

### before
sum size: 100000, fp32: 0.005 ms; bf16: 0.023 ms
sum size: 1000000, fp32: 0.153 ms; bf16: 0.217 ms
sum size: 10000000, fp32: 2.272 ms; bf16: 2.130 ms

### after
sum size: 100000, fp32: 0.006 ms; bf16: 0.007 ms
sum size: 1000000, fp32: 0.153 ms; bf16: 0.084 ms
sum size: 10000000, fp32: 2.365 ms; bf16: 0.840 ms

performance update on avx2 machine: Xeon(R) CPU E5-2680 v3 @ 2.50GHz

### before
sum size: 100000, fp32: 0.025 ms; bf16: 0.042 ms
sum size: 1000000, fp32: 0.149 ms; bf16: 0.343 ms
sum size: 10000000, fp32: 3.494 ms; bf16: 3.293 ms

### after
sum size: 100000, fp32: 0.024 ms; bf16: 0.016 ms
sum size: 1000000, fp32: 0.135 ms; bf16: 0.093 ms
sum size: 10000000, fp32: 3.229 ms; bf16: 1.078 ms

[ghstack-poisoned]

ghstack-source-id: 4f2c31e Pull Request resolved: pytorch#55217

[ghstack-poisoned]

ghstack-source-id: 69bace8 Pull Request resolved: pytorch#55217

VitalyFedyunin · 2021-06-02T16:12:44Z

@VitalyFedyunin has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Differential Revision: [D28836794](https://our.internmc.facebook.com/intern/diff/D28836794) [ghstack-poisoned]

VitalyFedyunin · 2021-06-23T16:41:24Z

@VitalyFedyunin has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Differential Revision: [D28836794](https://our.internmc.facebook.com/intern/diff/D28836794) [ghstack-poisoned]

VitalyFedyunin · 2021-06-29T18:12:15Z

@VitalyFedyunin has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Differential Revision: [D28836794](https://our.internmc.facebook.com/intern/diff/D28836794) [ghstack-poisoned]

peterbell10

Just noticed this PR, but as of #60387 BFloat16 is accumulated in floats.

facebook-github-bot · 2021-06-30T19:28:59Z

@VitalyFedyunin merged this pull request in 4f5c688.

VitalyFedyunin · 2021-06-30T20:42:03Z

@mingfeima can you please take a look if we should revert as per @peterbell10 comment

peterbell10 · 2021-06-30T22:18:27Z

Oh well, too late I guess. It's probably easier for me to revert it myself as I'm working on other sum PRs that might conflict.

A similar concept was implemented in gh-60387 which made this dead code (scalar_t in multi_row_sum will never be BFloat16 or Half). [ghstack-poisoned]

mingfeima · 2021-07-01T01:03:12Z

Oh well, too late I guess. It's probably easier for me to revert it myself as I'm working on other sum PRs that might conflict.

@peterbell10 @VitalyFedyunin feel free to revert this one. #60387 is enough to do the job :)

VitalyFedyunin · 2021-07-01T15:12:16Z

Reverted. Please rebase (and update) rest of the stack.

facebook-github-bot · 2021-07-01T15:14:02Z

This pull request has been reverted by cb7d813.

SumKernel (BFloat16): use float as accumulation type

38007d0

[ghstack-poisoned]

mingfeima mentioned this pull request Apr 2, 2021

Optimize some reduction operators on CPU BFloat16 #55202

Closed

facebook-github-bot added the cla signed label Apr 2, 2021

mingfeima mentioned this pull request Apr 2, 2021

add BFloat16 support for LayerNorm CPU #55210

Closed

mingfeima added a commit that referenced this pull request Apr 2, 2021

SumKernel (BFloat16): use float as accumulation type

c8c87da

ghstack-source-id: a3e967e Pull Request resolved: #55217

pytorchbot added the open source label Apr 2, 2021

mingfeima mentioned this pull request Apr 2, 2021

optimize BFloat16 elemwise operators CPU: sigmoid, sigmoid_backward, tanh_backward, addcmul, addcdiv #55221

Closed

Update on "SumKernel (BFloat16): use float as accumulation type"

76f1a28

[ghstack-poisoned]

mingfeima mentioned this pull request Apr 8, 2021

add bf16 support for bucketize #55588

Closed

mingfeima added 2 commits April 12, 2021 19:27

Update on "SumKernel (BFloat16): use float as accumulation type"

e4ed824

[ghstack-poisoned]

Update on "SumKernel (BFloat16): use float as accumulation type"

9fd708f

[ghstack-poisoned]

mingfeima mentioned this pull request Apr 19, 2021

add BFloat16 support for bernoulli and Dropout on CPU #56372

Closed

Update on "SumKernel (BFloat16): use float as accumulation type"

b03f7cb

[ghstack-poisoned]

This was referenced Apr 26, 2021

add BFloat16 support for AdaptiveAvgPool2d on CPU #56902

Closed

add BFloat16 support for MaxPool2d on CPU #56903

Closed

optimize transpose copy for float32 and bfloat16 on CPU #56904

Closed

mingfeima added 2 commits April 27, 2021 09:55

Update on "SumKernel (BFloat16): use float as accumulation type"

b758234

[ghstack-poisoned]

Update on "SumKernel (BFloat16): use float as accumulation type"

8fa1c25

[ghstack-poisoned]

mingfeima added a commit to mingfeima/pytorch that referenced this pull request Apr 28, 2021

SumKernel (BFloat16): use float as accumulation type

9aef413

ghstack-source-id: 4f2c31e Pull Request resolved: pytorch#55217

Update on "SumKernel (BFloat16): use float as accumulation type"

7e877a0

[ghstack-poisoned]

mingfeima mentioned this pull request May 13, 2021

use mkldnn for Linear on CPU BFloat16 dtype #58210

Closed

mingfeima added 2 commits May 13, 2021 10:52

Update on "SumKernel (BFloat16): use float as accumulation type"

a5a6feb

[ghstack-poisoned]

Update on "SumKernel (BFloat16): use float as accumulation type"

f2aa23f

[ghstack-poisoned]

mingfeima mentioned this pull request May 14, 2021

add BFloat16 support for UpSample on CPU #58297

Closed

dgl-intel pushed a commit to dgl-intel/pytorch that referenced this pull request May 14, 2021

SumKernel (BFloat16): use float as accumulation type

f5ef0c8

ghstack-source-id: 69bace8 Pull Request resolved: pytorch#55217

mingfeima added 2 commits June 4, 2021 14:25

Update on "SumKernel (BFloat16): use float as accumulation type"

15c15a0

Differential Revision: [D28836794](https://our.internmc.facebook.com/intern/diff/D28836794) [ghstack-poisoned]

Update on "SumKernel (BFloat16): use float as accumulation type"

5982ea0

Differential Revision: [D28836794](https://our.internmc.facebook.com/intern/diff/D28836794) [ghstack-poisoned]

mingfeima added 2 commits June 18, 2021 10:16

Update on "SumKernel (BFloat16): use float as accumulation type"

1fbae61

Differential Revision: [D28836794](https://our.internmc.facebook.com/intern/diff/D28836794) [ghstack-poisoned]

Update on "SumKernel (BFloat16): use float as accumulation type"

ed400c4

Differential Revision: [D28836794](https://our.internmc.facebook.com/intern/diff/D28836794) [ghstack-poisoned]

VitalyFedyunin approved these changes Jun 25, 2021

View reviewed changes

Update on "SumKernel (BFloat16): use float as accumulation type"

f0d6b01

Differential Revision: [D28836794](https://our.internmc.facebook.com/intern/diff/D28836794) [ghstack-poisoned]

mingfeima mentioned this pull request Jun 30, 2021

add BFloat16 support for BatchNorm2d on CPU #61015

Closed

Update on "SumKernel (BFloat16): use float as accumulation type"

5b0a4c6

Differential Revision: [D28836794](https://our.internmc.facebook.com/intern/diff/D28836794) [ghstack-poisoned]

peterbell10 requested changes Jun 30, 2021

View reviewed changes

facebook-github-bot closed this in 4f5c688 Jun 30, 2021

facebook-github-bot added the Merged label Jun 30, 2021

This was referenced Jun 30, 2021

Revert gh-55217: SumKernel (BFloat16): use float as accumulation type #61081

Closed

Use cascade-summation to improve nansum accuracy #61082

Closed

Add BFloat16 support to CPU nansum #61083

Closed

facebook-github-bot added the Reverted label Jul 1, 2021

facebook-github-bot deleted the gh/mingfeima/18/head branch July 4, 2021 14:17

SumKernel (BFloat16): use float as accumulation type #55217

SumKernel (BFloat16): use float as accumulation type #55217

Uh oh!

Conversation

mingfeima commented Apr 2, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

facebook-github-bot commented Apr 2, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💊 CI failures summary and remediations

Uh oh!

mingfeima commented Apr 2, 2021

Uh oh!

VitalyFedyunin commented Jun 2, 2021

Uh oh!

VitalyFedyunin commented Jun 23, 2021

Uh oh!

VitalyFedyunin commented Jun 29, 2021

Uh oh!

peterbell10 left a comment

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented Jun 30, 2021

Uh oh!

VitalyFedyunin commented Jun 30, 2021

Uh oh!

peterbell10 commented Jun 30, 2021

Uh oh!

mingfeima commented Jul 1, 2021

Uh oh!

VitalyFedyunin commented Jul 1, 2021

Uh oh!

facebook-github-bot commented Jul 1, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

mingfeima commented Apr 2, 2021 •

edited

Loading

facebook-github-bot commented Apr 2, 2021 •

edited

Loading