You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Current variance kernel converts all data type to double before calculation. It's sub-optimal for integers. Integer arithmetic is much faster than floating points, e.g., summation is 4x faster [1].
A quick test for calculating int32 variance shows up to 3x performance gain. Another benefit is that integer arithmetic is accurate.
Current variance kernel converts all data type to
double
before calculation. It's sub-optimal for integers. Integer arithmetic is much faster than floating points, e.g., summation is 4x faster [1].A quick test for calculating int32 variance shows up to 3x performance gain. Another benefit is that integer arithmetic is accurate.
[1] https://quick-bench.com/q/_Sz-Peq1MNWYwZYrTtQDx3GI7lQ
Reporter: Yibo Cai / @cyb70289
Assignee: Yibo Cai / @cyb70289
PRs and other links:
Note: This issue was originally created as ARROW-10304. Please see the migration documentation for further details.
The text was updated successfully, but these errors were encountered: