New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Strange rounding issue when aggregating as sum #3223
Comments
Seems to be a floating-point rounding issue.
or
or
or
or
If some implementations return the "expected" result with two decimal digits, then they probably have a smarter procedure for rounding floating point values for display/return purposes. |
@jsteemann I thought arango would use double precision in general for its queries? See the docs: All numeric values are treated as 64-bit double-precision values internally |
Yes, and that is the source of the problem. Double-precision floating point values cannot safely represent every possible value without loss.
|
ok, I thought this would only happen with normal floats, because Javas double type handled this calculation just as expected. I helped myself with rounding it to the 6th decimal. |
I think Java will use the same double-precision implementation under the hood, but potentially its rounding and/or display procedure for doubles is a bit different (read: smarter). |
my environment running ArangoDB
I'm using the latest ArangoDB of the respective release series:
3.1 on Single-Server with mmfiles on Windows 10
I'm issuing AQL via:
I have the following result from my query:
The grossPrice is am aggregation using this:
I expected to get
23.53
instead I got23.529999999999998
. I thought this should not happen using double precision as explained in the docs? Even without the multiplication I get a strange value.The text was updated successfully, but these errors were encountered: