-
Notifications
You must be signed in to change notification settings - Fork 68
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Incorrect histogram_ bounds check #163
Comments
Using |
Floating point strikes again. Here is reproducer:
Problem is when upper limit of histogram is set to maximum value of sample latter could go to N+1 bin which out of range. I'm not sure how to fix this. |
If it's just a floating point issue, then we can assume it's N+1 for this case always? If so, can we just clamp it to the max number of bins? |
Not quite.
In histogram-fill I had special under/overflow bins. Here original semantics should be kept. Anything out of specified range should throw exception. Only question is how to calculate bins |
How about clamp if it's within floating point precision error? |
I think that's what have been tried. It didn't quite work out: I think it's viable but requires very accurate implementation |
Not to mention that this approach is plain wrong. It pretends to work for any |
What about a special case of the final bin if the element is equal to the upper bound? |
The data is too large for me to narrow down, but I can try. However, I get
with a bin size of 10 for data ranging from 0.0 to 747.0564541606117 with the custom set range of those values (I set the range equal to the minimum and maximum of the list). Is there a rounding issue here?
The text was updated successfully, but these errors were encountered: