Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Invalid percentiles calculated #4

Closed
sporty81 opened this issue Aug 7, 2020 · 3 comments
Closed

Invalid percentiles calculated #4

sporty81 opened this issue Aug 7, 2020 · 3 comments
Labels
bug Something isn't working

Comments

@sporty81
Copy link

sporty81 commented Aug 7, 2020

For some reason with this tdigest, some percentiles are wrong.

flags 0 count 14 compression 100 centroids 14 (3.249696, 1) (8.566236, 1) (10.157393, 1) (12.757881, 1) (12.882931, 1) (38.585662, 1) (48.822837, 1) (55.881150, 1) (60.400374, 1) (69.476553, 1) (77.267742, 1) (83.499431, 1) (86.896817, 1) (91.439196, 1)

For some reason the 93-96 percentiles report higher than 97-99

92% 90.8941102381796
93% 91.5300433030352
94% 92.1659763678908
95% 92.8019094327465
96% 93.4378424976021
97% 91.4391957223415
98% 91.4391957223415
99% 91.4391957223415
@tvondra
Copy link
Owner

tvondra commented Aug 9, 2020

Thanks for the report. I can reproduce it and yeah - that seems like a bug.

@tvondra tvondra added the bug Something isn't working label Aug 9, 2020
@tvondra
Copy link
Owner

tvondra commented Aug 9, 2020

Should be fixed by 4eb5bff. A stupid rounding issue with int64 and double arithmetics.

@tvondra
Copy link
Owner

tvondra commented Aug 13, 2020

Should be fixed by 4eb5bff, and I've added a simple regression test in 0be29a2.

@tvondra tvondra closed this as completed Aug 13, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@tvondra @sporty81