Skip to content

Add note about aggregations on floats #10285

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
suneet-s merged 3 commits into from
Aug 17, 2020
Merged

Add note about aggregations on floats #10285

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
faba138
Add note about aggreations on floats
suneet-s Aug 14, 2020
363feb6
Also talk about doubles
suneet-s Aug 14, 2020
e0deb1e
Apply suggestions from code review
suneet-s Aug 17, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions docs/development/extensions-core/stats.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,13 @@ This algorithm was proven to be numerically stable by J.L. Barlow in
"Error analysis of a pairwise summation algorithm to compute sample variance"
Numer. Math, 58 (1991) pp. 583--590

> As with all [aggregators](../../querying/sql.md#aggregation-functions), the order of operations across segments is
> non-deterministic. This means that if this aggregator operates with an input type of "float" or "double", the result
> of the aggregation may not be precisely the same across multiple runs of the query.
>
> To produce consistent results, round the variance to a fixed number of decimal places so that the results are
> precisely the same across query runs.

### Pre-aggregating variance at ingestion time

To use this feature, an "variance" aggregator must be included at indexing time.
Expand Down
7 changes: 7 additions & 0 deletions docs/querying/sql.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -232,6 +232,13 @@ possible for two aggregators in the same SQL query to have different filters.

Only the COUNT aggregation can accept DISTINCT.

> The order of aggregation operations across segments is not deterministic. This means that non-commutative aggregation
> functions can produce inconsistent results across the same query.
>
> Functions that operate on an input type of "float" or "double" may also see these differences in aggregation
> results across multiple query runs because of this. If precisely the same value is desired across multiple query runs,
> consider using the `ROUND` function to smooth out the inconsistencies between queries.

|Function|Notes|
|--------|-----|
|`COUNT(*)`|Counts the number of rows.|
Expand Down