Add native impl of gelu for QuantizedCPU #69968
Conversation
CI Flow Status⚛️ CI FlowRuleset - Version:
You can add a comment to the PR and tag @pytorchbot with the following commands: # ciflow rerun, "ciflow/default" will always be added automatically
@pytorchbot ciflow rerun
# ciflow rerun with additional labels "-l <ciflow/label_name>", which is equivalent to adding these labels manually and trigger the rerun
@pytorchbot ciflow rerun -l ciflow/scheduled -l ciflow/slowFor more information, please take a look at the CI Flow Wiki. |
🔗 Helpful links
💊 CI failures summary and remediationsAs of commit 30f052a (more details on the Dr. CI page): 💚 💚 Looks good so far! There are no failures yet. 💚 💚 This comment was automatically generated by Dr. CI (expand for details).Please report bugs/suggestions to the (internal) Dr. CI Users group. |
|
The failure does not seem to be caused by this patch |
Xia-Weiwen
changed the title
mruberry
added
the
triaged
| const auto kAlphaVec = Vectorized<float>(M_SQRT1_2); | ||
| const auto kOneVec = Vectorized<float>(1); | ||
| const auto kPointFiveVec = Vectorized<float>(0.5); |
There was a problem hiding this comment.
nit: maybe initialize outside the loop?
| float output_scale = scale; | ||
|
|
||
| AT_DISPATCH_QINT_TYPES(qx.scalar_type(), "qgelu", [&]() { | ||
| float inv_output_scale = 1.0 / output_scale; |
There was a problem hiding this comment.
nit: init outside the macro?
| dtype=torch_type) | ||
| qY_hat = op(qX) | ||
| self.assertEqual(qY.dequantize(), qY_hat.dequantize(), | ||
| msg="F.leaky_relu failed ({} vs {})".format(qY, qY_hat)) |
There was a problem hiding this comment.
nit: "gelu" instead of "leaky_relu"?
|
@vkuzo has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
could you take a look at the minor comments, and I can import again after those? |
…ments out of the dispatch macro and loop
Hi. I have modified the code according to your comments. Please take a look. |
|
@vkuzo has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
Hi @Xia-Weiwen , internal CI caught one more issue: "dos-newlines: You must use ONLY Unix linebreaks |
Hi @vkuzo. I have removed those '\r'. Please check again. Thanks. |
|
@vkuzo has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
1 similar comment
|
@vkuzo has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
any chance you could rebase and upload again? I'm seeing a rebase conflict in Meta CI. |
|
@vkuzo has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
Add native implementation of gelu for quantized CPU.
cc @jerryzh168 @jianyuh @raghuramank100 @jamesr66a @vkuzo