Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bin collisions in histogram_from_sketch possible #518

Closed
jamie256 opened this issue Apr 6, 2022 · 1 comment
Closed

bin collisions in histogram_from_sketch possible #518

jamie256 opened this issue Apr 6, 2022 · 1 comment
Labels
bug Something isn't working

Comments

@jamie256
Copy link
Contributor

jamie256 commented Apr 6, 2022

Summary

If a dataset profile has logged values in a relatively tight range but are large in absolute value, and there are enough bins, the bin edge calculations can hit collisions because of numeric precision on the float values used.

This affects the conversion of the profile to a summary.

@jamie256
Copy link
Contributor Author

jamie256 commented Apr 7, 2022

mitigation in place to prevent errors due to bin width collisions, will follow up in next version to support calling a double version of the kll_sketch so we have closer to the expected precision coming from python floats.

@jamie256 jamie256 closed this as completed Apr 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant