Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added cast op oneDNN kernel for bf16/fp32 datatypes casting(FWD/BWD) #33056

Merged
merged 9 commits into from May 26, 2021

Conversation

jakpiase
Copy link
Contributor

@jakpiase jakpiase commented May 22, 2021

PR types

New features

PR changes

OPs

Describe

Added cast op for 4 types of casts (fp32->fp32, fp32->bf16, bf16->fp32, bf16->bf16) for enabling Word2Vec model. OneDNN kernel can be used both in FWD and BWD.

@paddle-bot-old
Copy link

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@jakpiase jakpiase requested a review from jczaja May 24, 2021 11:57
jczaja
jczaja previously approved these changes May 24, 2021
Copy link
Contributor

@jczaja jczaja left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jczaja jczaja requested a review from arlesniak May 24, 2021 14:44
@lidanqing-intel
Copy link
Contributor

@jakpiase Hi Please rename the PR title or make it more clear because I will use these titles for release note.
I notice another cast op FP32/BF16 PR has been merged. What are the difference

@jakpiase jakpiase changed the title Added cast op bf16/fp32 Added cast op oneDNN kernel for bf16/fp32 datatypes casting May 25, 2021
@jakpiase jakpiase changed the title Added cast op oneDNN kernel for bf16/fp32 datatypes casting Added cast op oneDNN kernel for bf16/fp32 datatypes casting(FWD/BWD) May 25, 2021
jczaja
jczaja previously approved these changes May 25, 2021
@jakpiase jakpiase requested review from arlesniak and jczaja May 25, 2021 10:45
jczaja
jczaja previously approved these changes May 25, 2021
arlesniak
arlesniak previously approved these changes May 25, 2021
Copy link
Contributor

@arlesniak arlesniak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

paddle/fluid/operators/cast_op.cc Outdated Show resolved Hide resolved
paddle/fluid/operators/cast_op.cc Outdated Show resolved Hide resolved
paddle/fluid/platform/mkldnn_reuse.h Show resolved Hide resolved
@jakpiase jakpiase dismissed stale reviews from arlesniak and jczaja via 5af7681 May 25, 2021 12:01
Copy link
Contributor

@arlesniak arlesniak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jakpiase
Copy link
Contributor Author

@luotao1 Could you please start your review?

@luotao1 luotao1 merged commit a2a45d8 into PaddlePaddle:develop May 26, 2021
self.check_grad_with_place(
core.CPUPlace(), ["X"],
"Out",
check_dygraph=False,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

def check_output_with_place(self,
place,
atol=0,
no_check_set=None,
equal_nan=False,
check_dygraph=True,
inplace_atol=None):
self.infer_dtype_from_inputs_outputs(self.inputs, self.outputs)
if self.dtype == np.float64 and \
self.op_type not in op_threshold_white_list.NEED_FIX_FP64_CHECK_OUTPUT_THRESHOLD_OP_LIST:
atol = 0
if self.is_bfloat16_op():
check_dygraph = False
if hasattr(self, 'force_fp32_output') and getattr(

since you add check_dygraph = False when bf16_op in check_output_with_place, could you add same code in check_grad_with_place? Thus, you can decrease one approve
图片

@jakpiase @lidanqing-intel @jczaja

@wzzju Please note this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants