CUDA BFloat16 signal windows #45155

zasdfgbnm · 2020-09-22T18:35:24Z

Looks like this op is never tested for the support of different dtypes?

…r_window

dr-ci · 2020-10-14T18:38:14Z

💊 CI failures summary and remediations

As of commit cdf7c49 (more details on the Dr. CI page):

1/1 failures introduced in this PR

🕵️ 1 new failure recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

pytorch_linux_backward_compatibility_check_test (1/1)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

Oct 22 00:20:08 The PR is introducing backward incompatible changes to the operator library. Please contact PyTorch team to confirm whether this change is wanted or not.

Oct 22 00:20:08 processing existing schema:  __setstate__(__torch__.torch.classes.quantized.EmbeddingPackedParamsBase _0, (int, Tensor[], float[], int[]) _1) -> (None _0) 
Oct 22 00:20:08 processing existing schema:  bit_rate(__torch__.torch.classes.quantized.EmbeddingPackedParamsBase _0) -> (int _0) 
Oct 22 00:20:08 processing existing schema:  version(__torch__.torch.classes.quantized.EmbeddingPackedParamsBase _0) -> (int _0) 
Oct 22 00:20:08 processing existing schema:  __getstate__(__torch__.torch.classes.xnnpack.LinearOpContext _0) -> ((Tensor, Tensor?, Scalar?, Scalar?) _0) 
Oct 22 00:20:08 processing existing schema:  __setstate__(__torch__.torch.classes.xnnpack.LinearOpContext _0, (Tensor, Tensor?, Scalar?, Scalar?) _1) -> (None _0) 
Oct 22 00:20:08 processing existing schema:  __getstate__(__torch__.torch.classes.xnnpack.Conv2dOpContext _0) -> ((Tensor, Tensor?, int[], int[], int[], int, Scalar?, Scalar?) _0) 
Oct 22 00:20:08 processing existing schema:  __setstate__(__torch__.torch.classes.xnnpack.Conv2dOpContext _0, (Tensor, Tensor?, int[], int[], int[], int, Scalar?, Scalar?) _1) -> (None _0) 
Oct 22 00:20:08 processing existing schema:  __getstate__(__torch__.torch.classes.xnnpack.TransposeConv2dOpContext _0) -> ((Tensor, Tensor?, int[], int[], int[], int[], int, Scalar?, Scalar?) _0) 
Oct 22 00:20:08 processing existing schema:  __setstate__(__torch__.torch.classes.xnnpack.TransposeConv2dOpContext _0, (Tensor, Tensor?, int[], int[], int[], int[], int, Scalar?, Scalar?) _1) -> (None _0) 
Oct 22 00:20:08 processing existing schema:  __init__(__torch__.torch.classes.dist_rpc.WorkerInfo _0, str _1, int _2) -> (None _0) 
Oct 22 00:20:08 The PR is introducing backward incompatible changes to the operator library. Please contact PyTorch team to confirm whether this change is wanted or not.  
Oct 22 00:20:08  
Oct 22 00:20:08 Broken ops: [ 
Oct 22 00:20:08 	prim::is_vulkan(Tensor a) -> (bool) 
Oct 22 00:20:08 ] 
Oct 22 00:20:08 + cleanup 
Oct 22 00:20:08 + retcode=1 
Oct 22 00:20:08 + set +x 
Oct 22 00:20:08 =================== sccache compilation log =================== 
Oct 22 00:20:08 =========== If your build fails, please take a look at the log above for possible reasons =========== 
Oct 22 00:20:08 Compile requests                 0

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker or post in the (internal) Dr. CI Users group.

See how this bot performed.

This comment has been revised 16 times.

ngimel · 2020-10-15T04:47:23Z

it is tested in test_signal_window_functions, please update the test accordingly.

…r_window

zasdfgbnm · 2020-10-20T19:21:31Z

test added for all dtypes, and arange is required for hann. arange will be tested in #44848

ngimel · 2020-10-21T01:37:58Z

aten/src/ATen/native/cuda/UnaryOpsKernel.cu

-      });
+    const scalar_t alpha = static_cast<scalar_t>((window_length - 1) / 2.0);
+    gpu_kernel(iter, [=]GPU_LAMBDA(scalar_t a) -> scalar_t {
+      return calc_i0(static_cast<scalar_t>(beta) * ::sqrt(1 - ::pow((a - alpha) / alpha, static_cast<scalar_t>(2.0)))) / calc_i0(static_cast<scalar_t>(beta));


the intermediate computations of i0 args here should still be in accscalar_t? I can merge as though.

Is i0 bw bound or compute bound? If it is not bw bound, does it still make sense to compute on accscalar_t?

I don't know, but it's doing internal computations in fp32 anyway, and here

static_cast<scalar_t>(beta) * ::sqrt(1 - ::pow((a - alpha) / alpha

is still doing repeated conversions and truncations

pytorch/aten/src/ATen/native/Math.h

Line 542 in 719d29d

inline c10::BFloat16 calc_i0(c10::BFloat16 a) { return calc_i0(static_cast<float>(a)); }

OK, calc_i0 is already computing on float32 anyway

facebook-github-bot

@ngimel has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

zasdfgbnm · 2020-10-26T18:00:07Z

ping @ngimel

facebook-github-bot

@ngimel has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

facebook-github-bot · 2020-10-27T00:18:07Z

@ngimel merged this pull request in 99cf3b1.

CUDA BFloat16 kaiser_window

83f52db

zasdfgbnm requested a review from ngimel September 22, 2020 18:36

pytorchbot added the open source label Sep 22, 2020

ailzhang added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Sep 22, 2020

Merge branch 'master' of github.com:pytorch/pytorch into bfloat-kaise…

57b6602

…r_window

zasdfgbnm added 2 commits October 20, 2020 11:22

Merge branch 'master' of github.com:pytorch/pytorch into bfloat-kaise…

c3b0af5

…r_window

Add test

c83a102

zasdfgbnm changed the title ~~CUDA BFloat16 kaiser_window~~ CUDA BFloat16 signal windows Oct 20, 2020

ngimel reviewed Oct 21, 2020

View reviewed changes

ngimel approved these changes Oct 21, 2020

View reviewed changes

facebook-github-bot reviewed Oct 21, 2020

View reviewed changes

zasdfgbnm added 5 commits October 21, 2020 12:23

do computation in T_ACC

6f3ff43

update

9aef8fb

save

25ad846

save

98c0c04

fix

cdf7c49

ngimel approved these changes Oct 26, 2020

View reviewed changes

facebook-github-bot reviewed Oct 26, 2020

View reviewed changes

facebook-github-bot closed this in 99cf3b1 Oct 26, 2020

zasdfgbnm deleted the bfloat-kaiser_window branch October 26, 2020 23:06

facebook-github-bot added the Merged label Oct 27, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CUDA BFloat16 signal windows #45155

CUDA BFloat16 signal windows #45155

zasdfgbnm commented Sep 22, 2020

dr-ci bot commented Oct 14, 2020 •

edited

ngimel commented Oct 15, 2020

zasdfgbnm commented Oct 20, 2020

ngimel Oct 21, 2020

zasdfgbnm Oct 21, 2020

ngimel Oct 21, 2020

zasdfgbnm Oct 21, 2020

facebook-github-bot left a comment

zasdfgbnm commented Oct 26, 2020

facebook-github-bot left a comment

facebook-github-bot commented Oct 27, 2020

CUDA BFloat16 signal windows #45155

CUDA BFloat16 signal windows #45155

Conversation

zasdfgbnm commented Sep 22, 2020

dr-ci bot commented Oct 14, 2020 • edited

💊 CI failures summary and remediations

🕵️ 1 new failure recognized by patterns

pytorch_linux_backward_compatibility_check_test (1/1)

ngimel commented Oct 15, 2020

zasdfgbnm commented Oct 20, 2020

ngimel Oct 21, 2020

Choose a reason for hiding this comment

zasdfgbnm Oct 21, 2020

Choose a reason for hiding this comment

ngimel Oct 21, 2020

Choose a reason for hiding this comment

zasdfgbnm Oct 21, 2020

Choose a reason for hiding this comment

facebook-github-bot left a comment

Choose a reason for hiding this comment

zasdfgbnm commented Oct 26, 2020

facebook-github-bot left a comment

Choose a reason for hiding this comment

facebook-github-bot commented Oct 27, 2020

dr-ci bot commented Oct 14, 2020 •

edited