New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
TANH/Sigmoid 16-bit activation functions using LUT #34589
Changes from all commits
a858c19
279f926
eaac6ea
38eeb4f
e8ea83a
00879a5
9140684
913a787
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -741,18 +741,22 @@ TEST_P(TanhOpTest, TanhInt16) { | |
const float kMax = 32767.f / 32768.f; | ||
QuantizedActivationsOpModel m( | ||
GetRegistration(), BuiltinOperator_TANH, | ||
/*input=*/{TensorType_INT16, {1, 2, 4, 1}, 8 * kMin, 8 * kMax}, | ||
/*output=*/{TensorType_INT16, {1, 2, 4, 1}, kMin, kMax}); | ||
/*input=*/{TensorType_INT16, {1, 2, 8, 1}, 8 * kMin, 8 * kMax}, | ||
/*output=*/{TensorType_INT16, {1, 2, 8, 1}, kMin, kMax}); | ||
m.SetInput<int16_t>({ | ||
0, -6, 2, 4, // | ||
-4, -2, 8, 1, // | ||
7, -8, 3, -5, // | ||
6, -1, -3, 5 | ||
}); | ||
m.Invoke(); | ||
EXPECT_THAT(m.GetDequantizedOutput<int16_t>(), | ||
ElementsAreArray(ArrayFloatNear( | ||
{ | ||
0.0, -0.999987, 0.964027, 0.999329, // | ||
-0.999329, -0.96402, 0.99999, 0.76159, // | ||
0.999998337, -0.99999, 0.995054754, -0.999909204, // | ||
0.999999996, -0.76159, -0.995054754, 0.999909204 | ||
}, | ||
kQuantizedToleranceInt16))); | ||
} | ||
|
@@ -882,18 +886,20 @@ TEST_P(LogisticOpTest, SigmoidInt16) { | |
const float kMax = 32767.f / 32768.f; | ||
QuantizedActivationsOpModel m( | ||
GetRegistration(), BuiltinOperator_LOGISTIC, | ||
/*input=*/{TensorType_INT16, {1, 2, 4, 1}, 8 * kMin, 8 * kMax}, | ||
/*output=*/{TensorType_INT16, {1, 2, 4, 1}, kMin, kMax}); | ||
/*input=*/{TensorType_INT16, {1, 2, 6, 1}, 8 * kMin, 8 * kMax}, | ||
/*output=*/{TensorType_INT16, {1, 2, 6, 1}, kMin, kMax}); | ||
m.SetInput<int16_t>({ | ||
0, -6, 2, 4, // | ||
3, -2, 10, 1, // | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. why delete this line? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @renjie-liu this line has been modified by replacing 10 with 8, so the input number is in the given range of the input tensor and i added 4 more numbers. |
||
3, -2, 8, 1, // | ||
5, -8, 7, -3 | ||
}); | ||
m.Invoke(); | ||
EXPECT_THAT(m.GetDequantizedOutput<int16_t>(), | ||
ElementsAreArray(ArrayFloatNear( | ||
{ | ||
0.5, 0.002473, 0.880797, 0.982014, // | ||
0.952574, 0.119203, 0.999955, 0.731059, // | ||
0.952574, 0.119203, 0.9995, 0.731059, // | ||
0.993307, 0.0003535, 0.999089, 0.047426 // | ||
}, | ||
kQuantizedToleranceInt16))); | ||
} | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we intended to keep the reference ones
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks like we did not fully align on this. From the discussion preceding this change, we understood that we would keep the new LUT-based implementation as a default one, and assumed that "default" == "kReference".
It also appears that, if GEMMLOWP_NEON is not defined, optimized_ops::Tanh() called in the case of kFixedPointOptimize is the same as reference_ops::Tanh() that used to be called for kReference.
So, we just followed the same code pattern as is used for Int8 and UInt8 versions.
I appreciate now that later in this file kGenericOptimized version of kernel_type seems to be registered as the default one for Tanh and Logistic (in Register_TANH()).
Do you think we should change to calling different implementations for different kernel_type? I.e.
kReference -> reference_ops::Tanh()
kFixedPointOptimize -> optimized_ops::Tanh()
kGenericOptimized -> EvalUsingLookupTableTanh16Bit() (default)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@renjie-liu Could you please clarify how the current implementation and the suggested one using LUT should co-exist ? Should we call the suggested as a reference, because it is more accurate ? The current one looks like a good fit for the kernel kFixedPointOptimize. Which one should be when the kernel type is not specified ?
Thanks !