-
Notifications
You must be signed in to change notification settings - Fork 24.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merge histograms without losing precision #93704
Merged
salvatore-campagna
merged 19 commits into
elastic:main
from
salvatore-campagna:fix/92822-hdrpercentiles-precision-default
Feb 13, 2023
Merged
Merge histograms without losing precision #93704
salvatore-campagna
merged 19 commits into
elastic:main
from
salvatore-campagna:fix/92822-hdrpercentiles-precision-default
Feb 13, 2023
Commits on Feb 10, 2023
-
fix: use max value 3 for numberOfSignificantValueDigits
For unmapped histogram fields we instantiate by default a DoubleHistogram with 3 significant digits precision. The test generates a random value for the number of significant digits that is in the range [0, 5]. As a result, if the test runs with 4 or 5 significant value digits but the HdrHistogram sketch only uses 3, checking errors on results will fail. Here we change the maximum value for the significant value digits to 3 if the query involves an index with unmapped fields.
Configuration menu - View commit details
-
Copy full SHA for 98ff032 - Browse repository at this point
Copy the full SHA 98ff032View commit details -
Configuration menu - View commit details
-
Copy full SHA for fb6ac0d - Browse repository at this point
Copy the full SHA fb6ac0dView commit details -
Configuration menu - View commit details
-
Copy full SHA for f49036f - Browse repository at this point
Copy the full SHA f49036fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 39b1c42 - Browse repository at this point
Copy the full SHA 39b1c42View commit details -
Configuration menu - View commit details
-
Copy full SHA for 1f5fc39 - Browse repository at this point
Copy the full SHA 1f5fc39View commit details -
fix: always merge histograms using the higher precision
Normallt we would create a histogram with the correct number of digits, nut for empty histograms we end up serializing and deserializing large arrays for empty aggregations. Here we have a kind of workaroud were we use just 3 digits for empty histograms and, at reduce time, we always merge usign the larger number of digits among all histograms.
Configuration menu - View commit details
-
Copy full SHA for 00fa30b - Browse repository at this point
Copy the full SHA 00fa30bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 8721bc9 - Browse repository at this point
Copy the full SHA 8721bc9View commit details -
Configuration menu - View commit details
-
Copy full SHA for 18b79a6 - Browse repository at this point
Copy the full SHA 18b79a6View commit details -
Configuration menu - View commit details
-
Copy full SHA for da68312 - Browse repository at this point
Copy the full SHA da68312View commit details -
Configuration menu - View commit details
-
Copy full SHA for 7c2f22b - Browse repository at this point
Copy the full SHA 7c2f22bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 4fc598a - Browse repository at this point
Copy the full SHA 4fc598aView commit details -
Configuration menu - View commit details
-
Copy full SHA for 6683480 - Browse repository at this point
Copy the full SHA 6683480View commit details -
Configuration menu - View commit details
-
Copy full SHA for c1be8c2 - Browse repository at this point
Copy the full SHA c1be8c2View commit details -
Configuration menu - View commit details
-
Copy full SHA for 44e739e - Browse repository at this point
Copy the full SHA 44e739eView commit details -
Configuration menu - View commit details
-
Copy full SHA for 9f8907f - Browse repository at this point
Copy the full SHA 9f8907fView commit details
Commits on Feb 13, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 7eac3f8 - Browse repository at this point
Copy the full SHA 7eac3f8View commit details -
fix: prevent serialization of empty tdigest objects
When an empty aggregations uses a TDigest object the udnelrying arrays used by the AvlTree TDigest implementation is eagerly allocated. If the aggreation produces no result, we serialize and deserialize an array which might be large if the comrpession value is large (about 5 * compression centroids are tracked). Here we use a null value for empty aggreations while building the result and later on use a static empty TDigest object at reduce time and merge it with the non-empty results.
Configuration menu - View commit details
-
Copy full SHA for 5b13951 - Browse repository at this point
Copy the full SHA 5b13951View commit details -
fix: undo changes to TDigest percentiles
This change will end up in another PR focused on improving memory usage for TDigest percentiles aggregations.
Configuration menu - View commit details
-
Copy full SHA for 30126e7 - Browse repository at this point
Copy the full SHA 30126e7View commit details -
Configuration menu - View commit details
-
Copy full SHA for c57b68a - Browse repository at this point
Copy the full SHA c57b68aView commit details
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.