Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix overflow in sparkbar function #48121

Merged
merged 5 commits into from
Apr 2, 2023

Conversation

vdimir
Copy link
Member

@vdimir vdimir commented Mar 28, 2023

Changelog category (leave one):

  • Bug Fix (user-visible misbehavior in an official stable release)

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Documentation entry for user-facing changes

  • Documentation is written (mandatory for new features)

Information about CI checks: https://clickhouse.com/docs/en/development/continuous-integration/

@davenger davenger self-assigned this Mar 28, 2023
@qoega
Copy link
Member

qoega commented Mar 28, 2023

My memory tells it is like 3rd time we try to fix this function...

@@ -43,7 +43,16 @@ struct AggregateFunctionSparkbarData

auto [it, inserted] = points.insert({x, y});
if (!inserted)
it->getMapped() += y;
{
if (std::numeric_limits<Y>::max() - it->getMapped() > y)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use arithmeticOverflow.h, it should be more efficient.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In fact, it is ok to annotate it with NO_SANITIZE_UNDEFINED, because the signed integer overflow is UB from the C++ compiler's standpoint but not from the CPU standpoint. We use it in other arithmetic operations, such as plus

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alexey-milovidov Unsigned overflow is also possible. In such case, saturating is a more expected result than losing values with passing through zero, imho. The arithmeticOverflow.h allows only overflow. Should we introduce functions for saturating arithmetic?
Another approach is to use UInt64 as a counter type for all smaller types (and handle only UInt64 overflow).

Copy link
Member Author

@vdimir vdimir Mar 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

UPD: switched implementation to common:addOverflow and common::mulOverflow with checking the result and saturating

@@ -161,7 +171,7 @@ class AggregateFunctionSparkbar final
}

PaddedPODArray<Y> histogram(width, 0);
PaddedPODArray<UInt64> fhistogram(width, 0);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@vdimir vdimir force-pushed the vdimir/sparkbar-overflow-fix branch from c2a7735 to ee3a840 Compare March 31, 2023 11:52
@vdimir vdimir force-pushed the vdimir/sparkbar-overflow-fix branch from ee3a840 to 2d18689 Compare March 31, 2023 11:59
@alexey-milovidov alexey-milovidov merged commit 67d65a9 into master Apr 2, 2023
@alexey-milovidov alexey-milovidov deleted the vdimir/sparkbar-overflow-fix branch April 2, 2023 16:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ubsan: AggregateFunctionSparkbar: signed integer overflow
4 participants