New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Scaled Decimal multiplication loses precision #8668
Comments
Hello @guillep. I have been thinking about this issue in order to propose a solution. However, after implementing the mentioned change I saw a consequence of it and I am not sure how desirable it is:
Although the solution from above is right, the trailing zero in 1.006s4 (1.0060) has an effect in the result.
Should the trailing zeros problem be fixed as part of another GitHub issue? What do you think? Thanks in advance. |
…ation-loses-precision Fixes #8668: Scaled Decimal multiplication loses precision
…ect#8668 (ScaledDecimal) Fix pharo-project#9413 related to pharo-project#8668
…t-test-case-for-the-Fix-of-issue-8668-ScaledDecimal Provide additional explicit test case for the Fix of issue #8668 (ScaledDecimal)
Describe the bug
1.003s3 * 1.006s4.
=>
1.0090s4
instead of something like:
1.009018
Expected behavior
This happens because scaled decimal multiplication is just taking the largest scale between the two scales. One possibility to keep precision is to multiply the scales instead (and taking care of the cases where scale = 0)
The text was updated successfully, but these errors were encountered: