add BFloat16 support for BatchNorm2d on CPU #61015

mingfeima · 2021-06-30T04:58:46Z

Stack from ghstack:

Differential Revision: D30476323

[ghstack-poisoned]

facebook-github-bot · 2021-06-30T04:58:52Z

🔗 Helpful links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/61015
📄 Preview docs built from this PR
📄 Preview C++ docs built from this PR
🔧 Opt-in to CIFlow to control what jobs run on your PRs

💊 CI failures summary and remediations

As of commit d596dde (more details on the Dr. CI page):

8/8 failures introduced in this PR

🕵️ 8 new failures recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

linux-xenial-py3.7-gcc7 / test (default, 1, 2, linux.2xlarge) (1/8)

Step: "Test" (full log | diagnosis details | 🔁 rerun)

2022-03-14T06:23:31.6803300Z FAIL [0.080s]: tes...ctional_instance_norm_cpu (__main__.TestCommonCPU)

2022-03-14T06:23:31.6800379Z     self.assertEqual(supported_dtypes, claimed_supported, msg=msg)
2022-03-14T06:23:31.6800781Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_internal/common_utils.py", line 2154, in assertEqual
2022-03-14T06:23:31.6801037Z     msg=msg,
2022-03-14T06:23:31.6801359Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_comparison.py", line 1074, in assert_equal
2022-03-14T06:23:31.6801628Z     raise error_metas[0].to_error()
2022-03-14T06:23:31.6802075Z AssertionError: The supported dtypes for nn.functional.batch_norm on cpu according to its OpInfo are
2022-03-14T06:23:31.6802463Z         {torch.float64, torch.float32}, but the detected supported dtypes are {torch.float64, torch.float32, torch.bfloat16}.
2022-03-14T06:23:31.6802802Z         The following dtypes should be added to the OpInfo: {torch.bfloat16}. 
2022-03-14T06:23:31.6802968Z 
2022-03-14T06:23:31.6803044Z ======================================================================
2022-03-14T06:23:31.6803300Z FAIL [0.080s]: test_dtypes_nn_functional_instance_norm_cpu (__main__.TestCommonCPU)
2022-03-14T06:23:31.6803663Z ----------------------------------------------------------------------
2022-03-14T06:23:31.6803900Z Traceback (most recent call last):
2022-03-14T06:23:31.6804298Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_internal/common_device_type.py", line 376, in instantiated_test
2022-03-14T06:23:31.6804582Z     result = test(self, **param_kwargs)
2022-03-14T06:23:31.6804966Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_internal/common_device_type.py", line 761, in test_wrapper
2022-03-14T06:23:31.6805332Z     return test(*args, **kwargs)
2022-03-14T06:23:31.6805688Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_internal/common_device_type.py", line 796, in dep_fn
2022-03-14T06:23:31.6805963Z     return fn(slf, *args, **kwargs)
2022-03-14T06:23:31.6806321Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_internal/common_device_type.py", line 954, in only_fn
2022-03-14T06:23:31.6806598Z     return fn(self, *args, **kwargs)

linux-xenial-py3.7-clang7-asan / test (default, 1, 3, linux.2xlarge) (2/8)

Step: "Test" (full log | diagnosis details | 🔁 rerun)

2022-03-14T08:37:14.4748680Z FAIL [0.240s]: tes...ctional_instance_norm_cpu (__main__.TestCommonCPU)

2022-03-14T08:37:14.4745657Z     self.assertEqual(supported_dtypes, claimed_supported, msg=msg)
2022-03-14T08:37:14.4746073Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_internal/common_utils.py", line 2154, in assertEqual
2022-03-14T08:37:14.4746327Z     msg=msg,
2022-03-14T08:37:14.4746853Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_comparison.py", line 1074, in assert_equal
2022-03-14T08:37:14.4747126Z     raise error_metas[0].to_error()
2022-03-14T08:37:14.4747402Z AssertionError: The supported dtypes for nn.functional.batch_norm on cpu according to its OpInfo are
2022-03-14T08:37:14.4747843Z         {torch.float32, torch.float64}, but the detected supported dtypes are {torch.float32, torch.bfloat16, torch.float64}.
2022-03-14T08:37:14.4748184Z         The following dtypes should be added to the OpInfo: {torch.bfloat16}. 
2022-03-14T08:37:14.4748338Z 
2022-03-14T08:37:14.4748426Z ======================================================================
2022-03-14T08:37:14.4748680Z FAIL [0.240s]: test_dtypes_nn_functional_instance_norm_cpu (__main__.TestCommonCPU)
2022-03-14T08:37:14.4749029Z ----------------------------------------------------------------------
2022-03-14T08:37:14.4749280Z Traceback (most recent call last):
2022-03-14T08:37:14.4749678Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_internal/common_device_type.py", line 376, in instantiated_test
2022-03-14T08:37:14.4749960Z     result = test(self, **param_kwargs)
2022-03-14T08:37:14.4750347Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_internal/common_device_type.py", line 761, in test_wrapper
2022-03-14T08:37:14.4750617Z     return test(*args, **kwargs)
2022-03-14T08:37:14.4750986Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_internal/common_device_type.py", line 796, in dep_fn
2022-03-14T08:37:14.4751264Z     return fn(slf, *args, **kwargs)
2022-03-14T08:37:14.4751622Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_internal/common_device_type.py", line 954, in only_fn
2022-03-14T08:37:14.4751896Z     return fn(self, *args, **kwargs)

linux-bionic-py3.7-clang9 / test (noarch, 1, 1, linux.2xlarge) (3/8)

Step: "Test" (full log | diagnosis details | 🔁 rerun)

2022-03-14T06:43:46.1020826Z FAIL [0.083s]: tes...ctional_instance_norm_cpu (__main__.TestCommonCPU)

2022-03-14T06:43:46.1017301Z     self.assertEqual(supported_dtypes, claimed_supported, msg=msg)
2022-03-14T06:43:46.1018019Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_internal/common_utils.py", line 2154, in assertEqual
2022-03-14T06:43:46.1018501Z     msg=msg,
2022-03-14T06:43:46.1018864Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_comparison.py", line 1074, in assert_equal
2022-03-14T06:43:46.1019133Z     raise error_metas[0].to_error()
2022-03-14T06:43:46.1019422Z AssertionError: The supported dtypes for nn.functional.batch_norm on cpu according to its OpInfo are
2022-03-14T06:43:46.1019954Z         {torch.float64, torch.float32}, but the detected supported dtypes are {torch.float64, torch.float32, torch.bfloat16}.
2022-03-14T06:43:46.1020324Z         The following dtypes should be added to the OpInfo: {torch.bfloat16}. 
2022-03-14T06:43:46.1020492Z 
2022-03-14T06:43:46.1020582Z ======================================================================
2022-03-14T06:43:46.1020826Z FAIL [0.083s]: test_dtypes_nn_functional_instance_norm_cpu (__main__.TestCommonCPU)
2022-03-14T06:43:46.1021189Z ----------------------------------------------------------------------
2022-03-14T06:43:46.1021425Z Traceback (most recent call last):
2022-03-14T06:43:46.1021824Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_internal/common_device_type.py", line 376, in instantiated_test
2022-03-14T06:43:46.1022125Z     result = test(self, **param_kwargs)
2022-03-14T06:43:46.1022497Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_internal/common_device_type.py", line 761, in test_wrapper
2022-03-14T06:43:46.1022783Z     return test(*args, **kwargs)
2022-03-14T06:43:46.1023247Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_internal/common_device_type.py", line 796, in dep_fn
2022-03-14T06:43:46.1023746Z     return fn(slf, *args, **kwargs)
2022-03-14T06:43:46.1024132Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_internal/common_device_type.py", line 954, in only_fn
2022-03-14T06:43:46.1024396Z     return fn(self, *args, **kwargs)

win-vs2019-cuda11.3-py3 / test (default, 1, 2, windows.8xlarge.nvidia.gpu) (4/8)

Step: "Test" (full log | diagnosis details | 🔁 rerun)

2022-03-14T08:10:45.9978219Z FAIL [0.063s]: tes...ctional_instance_norm_cpu (__main__.TestCommonCPU)

2022-03-14T08:10:45.9969508Z     self.assertEqual(supported_dtypes, claimed_supported, msg=msg)
2022-03-14T08:10:45.9970621Z   File "C:\actions-runner\_work\pytorch\pytorch\build\win_tmp\build\torch\testing\_internal\common_utils.py", line 2121, in assertEqual
2022-03-14T08:10:45.9971445Z     assert_equal(
2022-03-14T08:10:45.9972318Z   File "C:\actions-runner\_work\pytorch\pytorch\build\win_tmp\build\torch\testing\_comparison.py", line 1074, in assert_equal
2022-03-14T08:10:45.9973185Z     raise error_metas[0].to_error()
2022-03-14T08:10:45.9974027Z AssertionError: The supported dtypes for nn.functional.batch_norm on cpu according to its OpInfo are
2022-03-14T08:10:45.9975290Z         {torch.float64, torch.float32}, but the detected supported dtypes are {torch.float64, torch.float32, torch.bfloat16}.
2022-03-14T08:10:45.9976582Z         The following dtypes should be added to the OpInfo: {torch.bfloat16}. 
2022-03-14T08:10:45.9977213Z 
2022-03-14T08:10:45.9977579Z ======================================================================
2022-03-14T08:10:45.9978219Z FAIL [0.063s]: test_dtypes_nn_functional_instance_norm_cpu (__main__.TestCommonCPU)
2022-03-14T08:10:45.9978981Z ----------------------------------------------------------------------
2022-03-14T08:10:45.9979560Z Traceback (most recent call last):
2022-03-14T08:10:45.9980604Z   File "C:\actions-runner\_work\pytorch\pytorch\build\win_tmp\build\torch\testing\_internal\common_device_type.py", line 376, in instantiated_test
2022-03-14T08:10:45.9981566Z     result = test(self, **param_kwargs)
2022-03-14T08:10:45.9982537Z   File "C:\actions-runner\_work\pytorch\pytorch\build\win_tmp\build\torch\testing\_internal\common_device_type.py", line 761, in test_wrapper
2022-03-14T08:10:45.9983430Z     return test(*args, **kwargs)
2022-03-14T08:10:45.9984359Z   File "C:\actions-runner\_work\pytorch\pytorch\build\win_tmp\build\torch\testing\_internal\common_device_type.py", line 796, in dep_fn
2022-03-14T08:10:45.9985259Z     return fn(slf, *args, **kwargs)
2022-03-14T08:10:45.9986197Z   File "C:\actions-runner\_work\pytorch\pytorch\build\win_tmp\build\torch\testing\_internal\common_device_type.py", line 954, in only_fn
2022-03-14T08:10:45.9987204Z     return fn(self, *args, **kwargs)

win-vs2019-cuda11.3-py3 / test (force_on_cpu, 1, 1, windows.4xlarge) (5/8)

Step: "Test" (full log | diagnosis details | 🔁 rerun)

2022-03-14T08:03:40.9459126Z FAIL [0.046s]: tes...ctional_instance_norm_cpu (__main__.TestCommonCPU)

2022-03-14T08:03:40.9452784Z     self.assertEqual(supported_dtypes, claimed_supported, msg=msg)
2022-03-14T08:03:40.9453686Z   File "C:\actions-runner\_work\pytorch\pytorch\build\win_tmp\build\torch\testing\_internal\common_utils.py", line 2121, in assertEqual
2022-03-14T08:03:40.9454301Z     assert_equal(
2022-03-14T08:03:40.9455064Z   File "C:\actions-runner\_work\pytorch\pytorch\build\win_tmp\build\torch\testing\_comparison.py", line 1074, in assert_equal
2022-03-14T08:03:40.9455772Z     raise error_metas[0].to_error()
2022-03-14T08:03:40.9456380Z AssertionError: The supported dtypes for nn.functional.batch_norm on cpu according to its OpInfo are
2022-03-14T08:03:40.9457270Z         {torch.float32, torch.float64}, but the detected supported dtypes are {torch.float32, torch.bfloat16, torch.float64}.
2022-03-14T08:03:40.9458052Z         The following dtypes should be added to the OpInfo: {torch.bfloat16}. 
2022-03-14T08:03:40.9458401Z 
2022-03-14T08:03:40.9458652Z ======================================================================
2022-03-14T08:03:40.9459126Z FAIL [0.046s]: test_dtypes_nn_functional_instance_norm_cpu (__main__.TestCommonCPU)
2022-03-14T08:03:40.9459818Z ----------------------------------------------------------------------
2022-03-14T08:03:40.9460263Z Traceback (most recent call last):
2022-03-14T08:03:40.9461115Z   File "C:\actions-runner\_work\pytorch\pytorch\build\win_tmp\build\torch\testing\_internal\common_device_type.py", line 376, in instantiated_test
2022-03-14T08:03:40.9461800Z     result = test(self, **param_kwargs)
2022-03-14T08:03:40.9462628Z   File "C:\actions-runner\_work\pytorch\pytorch\build\win_tmp\build\torch\testing\_internal\common_device_type.py", line 761, in test_wrapper
2022-03-14T08:03:40.9463285Z     return test(*args, **kwargs)
2022-03-14T08:03:40.9463969Z   File "C:\actions-runner\_work\pytorch\pytorch\build\win_tmp\build\torch\testing\_internal\common_device_type.py", line 796, in dep_fn
2022-03-14T08:03:40.9464661Z     return fn(slf, *args, **kwargs)
2022-03-14T08:03:40.9465369Z   File "C:\actions-runner\_work\pytorch\pytorch\build\win_tmp\build\torch\testing\_internal\common_device_type.py", line 954, in only_fn
2022-03-14T08:03:40.9466025Z     return fn(self, *args, **kwargs)

linux-bionic-py3.7-clang9 / test (default, 1, 2, linux.2xlarge) (6/8)

Step: "Test" (full log | diagnosis details | 🔁 rerun)

2022-03-14T06:25:24.0707062Z FAIL [0.084s]: tes...ctional_instance_norm_cpu (__main__.TestCommonCPU)

2022-03-14T06:25:24.0683410Z     self.assertEqual(supported_dtypes, claimed_supported, msg=msg)
2022-03-14T06:25:24.0684086Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_internal/common_utils.py", line 2154, in assertEqual
2022-03-14T06:25:24.0684514Z     msg=msg,
2022-03-14T06:25:24.0684993Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_comparison.py", line 1074, in assert_equal
2022-03-14T06:25:24.0685763Z     raise error_metas[0].to_error()
2022-03-14T06:25:24.0686064Z AssertionError: The supported dtypes for nn.functional.batch_norm on cpu according to its OpInfo are
2022-03-14T06:25:24.0686500Z         {torch.float64, torch.float32}, but the detected supported dtypes are {torch.float64, torch.float32, torch.bfloat16}.
2022-03-14T06:25:24.0686846Z         The following dtypes should be added to the OpInfo: {torch.bfloat16}. 
2022-03-14T06:25:24.0687010Z 
2022-03-14T06:25:24.0687100Z ======================================================================
2022-03-14T06:25:24.0707062Z FAIL [0.084s]: test_dtypes_nn_functional_instance_norm_cpu (__main__.TestCommonCPU)
2022-03-14T06:25:24.0707947Z ----------------------------------------------------------------------
2022-03-14T06:25:24.0708224Z Traceback (most recent call last):
2022-03-14T06:25:24.0708630Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_internal/common_device_type.py", line 376, in instantiated_test
2022-03-14T06:25:24.0708939Z     result = test(self, **param_kwargs)
2022-03-14T06:25:24.0709330Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_internal/common_device_type.py", line 761, in test_wrapper
2022-03-14T06:25:24.0709612Z     return test(*args, **kwargs)
2022-03-14T06:25:24.0709990Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_internal/common_device_type.py", line 796, in dep_fn
2022-03-14T06:25:24.0710259Z     return fn(slf, *args, **kwargs)
2022-03-14T06:25:24.0710642Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_internal/common_device_type.py", line 954, in only_fn
2022-03-14T06:25:24.0710920Z     return fn(self, *args, **kwargs)

win-vs2019-cpu-py3 / test (default, 1, 2, windows.4xlarge) (7/8)

Step: "Test" (full log | diagnosis details | 🔁 rerun)

2022-03-14T06:55:40.2904892Z FAIL [0.031s]: tes...ctional_instance_norm_cpu (__main__.TestCommonCPU)

2022-03-14T06:55:40.2892487Z     self.assertEqual(supported_dtypes, claimed_supported, msg=msg)
2022-03-14T06:55:40.2893333Z   File "C:\actions-runner\_work\pytorch\pytorch\build\win_tmp\build\torch\testing\_internal\common_utils.py", line 2121, in assertEqual
2022-03-14T06:55:40.2893965Z     assert_equal(
2022-03-14T06:55:40.2894600Z   File "C:\actions-runner\_work\pytorch\pytorch\build\win_tmp\build\torch\testing\_comparison.py", line 1074, in assert_equal
2022-03-14T06:55:40.2901143Z     raise error_metas[0].to_error()
2022-03-14T06:55:40.2901853Z AssertionError: The supported dtypes for nn.functional.batch_norm on cpu according to its OpInfo are
2022-03-14T06:55:40.2902805Z         {torch.float64, torch.float32}, but the detected supported dtypes are {torch.bfloat16, torch.float64, torch.float32}.
2022-03-14T06:55:40.2903757Z         The following dtypes should be added to the OpInfo: {torch.bfloat16}. 
2022-03-14T06:55:40.2904129Z 
2022-03-14T06:55:40.2904408Z ======================================================================
2022-03-14T06:55:40.2904892Z FAIL [0.031s]: test_dtypes_nn_functional_instance_norm_cpu (__main__.TestCommonCPU)
2022-03-14T06:55:40.2905485Z ----------------------------------------------------------------------
2022-03-14T06:55:40.2905945Z Traceback (most recent call last):
2022-03-14T06:55:40.2906810Z   File "C:\actions-runner\_work\pytorch\pytorch\build\win_tmp\build\torch\testing\_internal\common_device_type.py", line 376, in instantiated_test
2022-03-14T06:55:40.2907547Z     result = test(self, **param_kwargs)
2022-03-14T06:55:40.2908296Z   File "C:\actions-runner\_work\pytorch\pytorch\build\win_tmp\build\torch\testing\_internal\common_device_type.py", line 761, in test_wrapper
2022-03-14T06:55:40.2908965Z     return test(*args, **kwargs)
2022-03-14T06:55:40.2909670Z   File "C:\actions-runner\_work\pytorch\pytorch\build\win_tmp\build\torch\testing\_internal\common_device_type.py", line 796, in dep_fn
2022-03-14T06:55:40.2910338Z     return fn(slf, *args, **kwargs)
2022-03-14T06:55:40.2911024Z   File "C:\actions-runner\_work\pytorch\pytorch\build\win_tmp\build\torch\testing\_internal\common_device_type.py", line 954, in only_fn
2022-03-14T06:55:40.2911685Z     return fn(self, *args, **kwargs)

linux-xenial-py3.7-gcc5.4 / test (default, 1, 2, linux.2xlarge) (8/8)

Step: "Test" (full log | diagnosis details | 🔁 rerun)

2022-03-14T06:26:40.6180564Z FAIL [0.079s]: tes...ctional_instance_norm_cpu (__main__.TestCommonCPU)

2022-03-14T06:26:40.6177817Z     self.assertEqual(supported_dtypes, claimed_supported, msg=msg)
2022-03-14T06:26:40.6178232Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_internal/common_utils.py", line 2154, in assertEqual
2022-03-14T06:26:40.6178476Z     msg=msg,
2022-03-14T06:26:40.6178810Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_comparison.py", line 1074, in assert_equal
2022-03-14T06:26:40.6179077Z     raise error_metas[0].to_error()
2022-03-14T06:26:40.6179348Z AssertionError: The supported dtypes for nn.functional.batch_norm on cpu according to its OpInfo are
2022-03-14T06:26:40.6179732Z         {torch.float64, torch.float32}, but the detected supported dtypes are {torch.float64, torch.float32, torch.bfloat16}.
2022-03-14T06:26:40.6180058Z         The following dtypes should be added to the OpInfo: {torch.bfloat16}. 
2022-03-14T06:26:40.6180220Z 
2022-03-14T06:26:40.6180309Z ======================================================================
2022-03-14T06:26:40.6180564Z FAIL [0.079s]: test_dtypes_nn_functional_instance_norm_cpu (__main__.TestCommonCPU)
2022-03-14T06:26:40.6180905Z ----------------------------------------------------------------------
2022-03-14T06:26:40.6181148Z Traceback (most recent call last):
2022-03-14T06:26:40.6181533Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_internal/common_device_type.py", line 376, in instantiated_test
2022-03-14T06:26:40.6181827Z     result = test(self, **param_kwargs)
2022-03-14T06:26:40.6182276Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_internal/common_device_type.py", line 761, in test_wrapper
2022-03-14T06:26:40.6182546Z     return test(*args, **kwargs)
2022-03-14T06:26:40.6182918Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_internal/common_device_type.py", line 796, in dep_fn
2022-03-14T06:26:40.6183182Z     return fn(slf, *args, **kwargs)
2022-03-14T06:26:40.6183552Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_internal/common_device_type.py", line 954, in only_fn
2022-03-14T06:26:40.6183833Z     return fn(self, *args, **kwargs)

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

ghstack-source-id: c66611d Pull Request resolved: #61015

[ghstack-poisoned]

ghstack-source-id: 72c07f7 Pull Request resolved: #61015

[ghstack-poisoned]

ghstack-source-id: ef88790 Pull Request resolved: #61015

mingfeima · 2021-07-08T01:08:16Z

TODO:

add mix precision support for weight and bias, aka. input/output(bf16) + weight/bias(fp32).
refactor the code
fix the compiler warning

[ghstack-poisoned]

ghstack-source-id: cc15ab7 Pull Request resolved: #61015

[ghstack-poisoned]

ghstack-source-id: 293858f Pull Request resolved: #61015

[ghstack-poisoned]

ghstack-source-id: d09feff Pull Request resolved: #61015

[ghstack-poisoned]

ghstack-source-id: 8cc5018 Pull Request resolved: #61015

[ghstack-poisoned]

ghstack-source-id: b605815 Pull Request resolved: #61015

[ghstack-poisoned]

ghstack-source-id: 78b0d8d Pull Request resolved: #61015

VitalyFedyunin · 2021-12-29T19:44:28Z

@VitalyFedyunin has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Differential Revision: [D30476323](https://our.internmc.facebook.com/intern/diff/D30476323) [ghstack-poisoned]

ghstack-source-id: 8e18490 Pull Request resolved: #61015

VitalyFedyunin · 2022-01-12T17:15:29Z

@VitalyFedyunin has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Differential Revision: [D30476323](https://our.internmc.facebook.com/intern/diff/D30476323) [ghstack-poisoned]

ghstack-source-id: ab44b55 Pull Request resolved: #61015

mingfeima · 2022-01-13T02:43:58Z

TODO: update the mixed dtype logic.

Differential Revision: [D30476323](https://our.internmc.facebook.com/intern/diff/D30476323) [ghstack-poisoned]

ghstack-source-id: 3462f8c Pull Request resolved: #61015

VitalyFedyunin · 2022-01-13T20:33:34Z

@VitalyFedyunin has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Differential Revision: [D30476323](https://our.internmc.facebook.com/intern/diff/D30476323) [ghstack-poisoned]

ghstack-source-id: 7c7d642 Pull Request resolved: #61015

Differential Revision: [D30476323](https://our.internmc.facebook.com/intern/diff/D30476323) [ghstack-poisoned]

VitalyFedyunin · 2022-01-25T21:22:19Z

@VitalyFedyunin has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Differential Revision: [D30476323](https://our.internmc.facebook.com/intern/diff/D30476323) [ghstack-poisoned]

add BFloat16 support for BatchNorm2d on CPU

15b8c71

[ghstack-poisoned]

mingfeima mentioned this pull request Jun 30, 2021

SumKernel (BFloat16): use float as accumulation type #55217

Closed

facebook-github-bot added the cla signed label Jun 30, 2021

mingfeima mentioned this pull request Jun 30, 2021

optimize BFloat16 elemwise operators CPU: sigmoid, sigmoid_backward, tanh_backward, addcmul, addcdiv #55221

Closed

mingfeima added a commit that referenced this pull request Jun 30, 2021

add BFloat16 support for BatchNorm2d on CPU

918409d

ghstack-source-id: c66611d Pull Request resolved: #61015

pytorchbot added the open source label Jun 30, 2021

Update on "add BFloat16 support for BatchNorm2d on CPU"

674f4b5

[ghstack-poisoned]

mingfeima added a commit that referenced this pull request Jun 30, 2021

add BFloat16 support for BatchNorm2d on CPU

745b65b

ghstack-source-id: 72c07f7 Pull Request resolved: #61015

Update on "add BFloat16 support for BatchNorm2d on CPU"

eb5282e

[ghstack-poisoned]

mingfeima added a commit that referenced this pull request Jul 8, 2021

add BFloat16 support for BatchNorm2d on CPU

6cc40de

ghstack-source-id: ef88790 Pull Request resolved: #61015

Update on "add BFloat16 support for BatchNorm2d on CPU"

6053740

[ghstack-poisoned]

mingfeima added a commit that referenced this pull request Jul 26, 2021

add BFloat16 support for BatchNorm2d on CPU

f5bf112

ghstack-source-id: cc15ab7 Pull Request resolved: #61015

Update on "add BFloat16 support for BatchNorm2d on CPU"

c61b530

[ghstack-poisoned]

mingfeima added a commit that referenced this pull request Jul 26, 2021

add BFloat16 support for BatchNorm2d on CPU

a972f9a

ghstack-source-id: 293858f Pull Request resolved: #61015

Update on "add BFloat16 support for BatchNorm2d on CPU"

e0a7e7c

[ghstack-poisoned]

mingfeima added a commit that referenced this pull request Jul 28, 2021

add BFloat16 support for BatchNorm2d on CPU

6ce52ed

ghstack-source-id: d09feff Pull Request resolved: #61015

Update on "add BFloat16 support for BatchNorm2d on CPU"

4362ece

[ghstack-poisoned]

mingfeima added a commit that referenced this pull request Aug 4, 2021

add BFloat16 support for BatchNorm2d on CPU

d42f72e

ghstack-source-id: 8cc5018 Pull Request resolved: #61015

mingfeima added the intel priority matters to intel architecture from performance wise label Aug 4, 2021

Update on "add BFloat16 support for BatchNorm2d on CPU"

4be8b57

[ghstack-poisoned]

mingfeima added a commit that referenced this pull request Aug 4, 2021

add BFloat16 support for BatchNorm2d on CPU

6fce66d

ghstack-source-id: b605815 Pull Request resolved: #61015

Update on "add BFloat16 support for BatchNorm2d on CPU"

16345f4

[ghstack-poisoned]

mingfeima added a commit that referenced this pull request Dec 22, 2021

add BFloat16 support for BatchNorm2d on CPU

5fc03f7

ghstack-source-id: 78b0d8d Pull Request resolved: #61015

VitalyFedyunin approved these changes Dec 29, 2021

View reviewed changes

Update on "add BFloat16 support for BatchNorm2d on CPU"

a662484

Differential Revision: [D30476323](https://our.internmc.facebook.com/intern/diff/D30476323) [ghstack-poisoned]

mingfeima added a commit that referenced this pull request Jan 11, 2022

add BFloat16 support for BatchNorm2d on CPU

d002977

ghstack-source-id: 8e18490 Pull Request resolved: #61015

Update on "add BFloat16 support for BatchNorm2d on CPU"

33a71e3

Differential Revision: [D30476323](https://our.internmc.facebook.com/intern/diff/D30476323) [ghstack-poisoned]

mingfeima added a commit that referenced this pull request Jan 13, 2022

add BFloat16 support for BatchNorm2d on CPU

3f12d94

ghstack-source-id: ab44b55 Pull Request resolved: #61015

Update on "add BFloat16 support for BatchNorm2d on CPU"

2249fd1

Differential Revision: [D30476323](https://our.internmc.facebook.com/intern/diff/D30476323) [ghstack-poisoned]

mingfeima added a commit that referenced this pull request Jan 13, 2022

add BFloat16 support for BatchNorm2d on CPU

271be85

ghstack-source-id: 3462f8c Pull Request resolved: #61015

mingfeima marked this pull request as draft January 13, 2022 06:08

Update on "add BFloat16 support for BatchNorm2d on CPU"

e7cb818

Differential Revision: [D30476323](https://our.internmc.facebook.com/intern/diff/D30476323) [ghstack-poisoned]

mingfeima added a commit that referenced this pull request Jan 17, 2022

add BFloat16 support for BatchNorm2d on CPU

cddca65

ghstack-source-id: 7c7d642 Pull Request resolved: #61015

mingfeima mentioned this pull request Jan 17, 2022

improve LayerNorm bfloat16 performance on CPU #71376

Closed

mingfeima added 3 commits January 24, 2022 12:45

Update on "add BFloat16 support for BatchNorm2d on CPU"

a57ba03

Differential Revision: [D30476323](https://our.internmc.facebook.com/intern/diff/D30476323) [ghstack-poisoned]

Update on "add BFloat16 support for BatchNorm2d on CPU"

7c10ab0

Differential Revision: [D30476323](https://our.internmc.facebook.com/intern/diff/D30476323) [ghstack-poisoned]

Update on "add BFloat16 support for BatchNorm2d on CPU"

c04d542

Differential Revision: [D30476323](https://our.internmc.facebook.com/intern/diff/D30476323) [ghstack-poisoned]

XiaobingSuper removed the intel priority matters to intel architecture from performance wise label Jan 26, 2022

Update on "add BFloat16 support for BatchNorm2d on CPU"

8e64b9b

Differential Revision: [D30476323](https://our.internmc.facebook.com/intern/diff/D30476323) [ghstack-poisoned]

This was referenced Mar 7, 2022

add mixed data type mode for LayerNorm forward path #73844

Closed

add native kernel for weight_norm on CPU #73845

Closed

add simd horizantal reduce to improve log_softmax and softmax performance on CPU #73953

Closed

Update on "add BFloat16 support for BatchNorm2d on CPU"

d596dde

Differential Revision: [D30476323](https://our.internmc.facebook.com/intern/diff/D30476323) [ghstack-poisoned]

mingfeima mentioned this pull request Mar 14, 2022

improve group_norm channels last performance on CPU #69067

Closed

suo removed the ciflow/default label Mar 22, 2022

mingfeima closed this May 21, 2022

facebook-github-bot deleted the gh/mingfeima/36/head branch June 20, 2022 14:17

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

add BFloat16 support for BatchNorm2d on CPU #61015

add BFloat16 support for BatchNorm2d on CPU #61015

Uh oh!

mingfeima commented Jun 30, 2021 •

edited

Loading

Uh oh!

facebook-github-bot commented Jun 30, 2021 •

edited

Loading

Uh oh!

mingfeima commented Jul 8, 2021 •

edited

Loading

Uh oh!

VitalyFedyunin commented Dec 29, 2021

Uh oh!

VitalyFedyunin commented Jan 12, 2022

Uh oh!

mingfeima commented Jan 13, 2022

Uh oh!

VitalyFedyunin commented Jan 13, 2022

Uh oh!

VitalyFedyunin commented Jan 25, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

add BFloat16 support for BatchNorm2d on CPU #61015

add BFloat16 support for BatchNorm2d on CPU #61015

Uh oh!

Conversation

mingfeima commented Jun 30, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

facebook-github-bot commented Jun 30, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful links

💊 CI failures summary and remediations

🕵️ 8 new failures recognized by patterns

linux-xenial-py3.7-gcc7 / test (default, 1, 2, linux.2xlarge) (1/8)

linux-xenial-py3.7-clang7-asan / test (default, 1, 3, linux.2xlarge) (2/8)

linux-bionic-py3.7-clang9 / test (noarch, 1, 1, linux.2xlarge) (3/8)

win-vs2019-cuda11.3-py3 / test (default, 1, 2, windows.8xlarge.nvidia.gpu) (4/8)

win-vs2019-cuda11.3-py3 / test (force_on_cpu, 1, 1, windows.4xlarge) (5/8)

linux-bionic-py3.7-clang9 / test (default, 1, 2, linux.2xlarge) (6/8)

win-vs2019-cpu-py3 / test (default, 1, 2, windows.4xlarge) (7/8)

linux-xenial-py3.7-gcc5.4 / test (default, 1, 2, linux.2xlarge) (8/8)

Uh oh!

mingfeima commented Jul 8, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

VitalyFedyunin commented Dec 29, 2021

Uh oh!

VitalyFedyunin commented Jan 12, 2022

Uh oh!

mingfeima commented Jan 13, 2022

Uh oh!

VitalyFedyunin commented Jan 13, 2022

Uh oh!

VitalyFedyunin commented Jan 25, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

mingfeima commented Jun 30, 2021 •

edited

Loading

facebook-github-bot commented Jun 30, 2021 •

edited

Loading

mingfeima commented Jul 8, 2021 •

edited

Loading