src: cpu: aarch64: re-enable fp16 post-ops for aarch64 #1588
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Perform the eltwise post-ops in fp32 instead of fp16 by casting up (to fp32) before executing the eltwise op and then casting back down (to fp16) after the operation completes. With this change, all fp16 benchdnn tests pass on aarch64.
Description
The oneDNN API specifies that post-ops need to (always) be executed in fp32, but the Arm Compute Library (ACL) executes them in f16 (when running in fp16 mode) which caused a lot of conv+relu benchdnn tests to fail.
To mitigate this, we submitted this RP (#1506) which disables ACL post-ops in fp16. As a result, all primitives which are followed by a post-op in fp16 started falling to the reference kernels.
Hence, a dramatic drop in performance for fp16.
This PR fixes the inaccuracies resulting from ACL executing the eltwise post-ops in fp16 by:
With this change:
we run 76x faster on Neoverse-V1.
Checklist
General
make test
andmake test_benchdnn_*
) pass locally for each commit?Performance improvements
New features
Bug fixes
RFC PR