Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scaled Decimal multiplication loses precision #8668

Closed
guillep opened this issue Mar 1, 2021 · 1 comment · Fixed by #9408
Closed

Scaled Decimal multiplication loses precision #8668

guillep opened this issue Mar 1, 2021 · 1 comment · Fixed by #9408

Comments

@guillep
Copy link
Member

guillep commented Mar 1, 2021

Describe the bug
1.003s3 * 1.006s4.

=>

1.0090s4

instead of something like:

1.009018

Expected behavior

This happens because scaled decimal multiplication is just taking the largest scale between the two scales. One possibility to keep precision is to multiply the scales instead (and taking care of the cases where scale = 0)

@pablo-s
Copy link

pablo-s commented Mar 19, 2021

Hello @guillep. I have been thinking about this issue in order to propose a solution.
As you correctly indicated with one of the tags, the problem is easy: the scale of the result (of a multiplication) is the addition of the scales of multiplicands.

However, after implementing the mentioned change I saw a consequence of it and I am not sure how desirable it is:

1.003s3 * 1.006s4
=>
1.0090180s7

Although the solution from above is right, the trailing zero in 1.006s4 (1.0060) has an effect in the result.
In other cases the effect is more evident:

1.0 asScaledDecimal * 1.0 asScaledDecimal.
=>
1.0000000000000000000000000000s28

Should the trailing zeros problem be fixed as part of another GitHub issue? What do you think?

Thanks in advance.

MarcusDenker added a commit that referenced this issue Jun 7, 2021
…ation-loses-precision

Fixes #8668: Scaled Decimal multiplication loses precision
astares added a commit to astares/pharo that referenced this issue Jun 7, 2021
Ducasse added a commit that referenced this issue Jun 7, 2021
…t-test-case-for-the-Fix-of-issue-8668-ScaledDecimal

Provide additional explicit test case for the Fix of issue #8668 (ScaledDecimal)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants