Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bigdecimal] Loss of precision with different execution order #2524

Closed
emboss opened this Issue Jan 27, 2015 · 7 comments

Comments

Projects
None yet
5 participants
@emboss
Copy link

emboss commented Jan 27, 2015

Depending on the order of calculation I get different results that should be equal:

require 'bigdecimal'

fraction = BigDecimal.new("0.0095") / 365 * BigDecimal.new(50_000)
# => #<BigDecimal:4f654cee,'0.130136986301369863013698630136986301369863013698630136986301369863013698630136986301369863013698630136986301369863013698630136986301369863013698630136986301369863013698630136986301369863013698630136985E1',201(208)>
r1 = fraction * BigDecimal.new(50_000) / BigDecimal.new(100_000)
# => #<BigDecimal:7134b8a7,'0.65068493150684931506849315068493150684931506849315068493150684931506849315068493150684931506849315068493150684931506849315068493150684931506849315068493150684931506849315068493150684931506849315068493E0',200(204)>
r2 = fraction * (BigDecimal.new(50_000) / BigDecimal.new(100_000)
# => #<BigDecimal:441fbe89,'0.650684931506849315068493150684931506849315068493150684931506849315068493150684931506849315068493150684931506849315068493150684931506849315068493150684931506849315068493150684931506849315068493150684925E0',201(208)>
r1 == r2
# => false

I could reproduce this with both

$ jruby -v
jruby 9.0.0.0.pre1 (2.2.0p0) 2015-01-20 d537cab Java HotSpot(TM) 64-Bit Server VM 25.0-b70 on 1.8.0-b132 +jit [linux-amd64]

and

$ jruby -v
jruby 1.7.18 (1.9.3p551) 2014-12-22 625381c on Java HotSpot(TM) 64-Bit Server VM 1.7.0_67-b01 +jit [linux-amd64]

Thanks for your great work on 9000 btw!

@Ch4s3

This comment has been minimized.

Copy link

Ch4s3 commented Jan 27, 2015

That looks like an error with how many decimal places are being stored.

I tried:

r3 = r1.round(200)
 => #<BigDecimal:761257ea,'0.65068493150684931506849315068493150684931506849315068493150684931506849315068493150684931506849315068493150684931506849315068493150684931506849315068493150684931506849315068493150684931506849315068493E0',200(204)>
r4 = r2.round(200)
 => #<BigDecimal:acaf0ac,'0.65068493150684931506849315068493150684931506849315068493150684931506849315068493150684931506849315068493150684931506849315068493150684931506849315068493150684931506849315068493150684931506849315068493E0',200(204)>
r3 == r4
 => true

The 201st decimal place seems to be where it breaks.

@emboss

This comment has been minimized.

Copy link
Author

emboss commented Jan 30, 2015

Thank you, awesome!

@enebo enebo added this to the 9.0.0.0.pre2 milestone Jan 30, 2015

@k77ch7

This comment has been minimized.

Copy link
Contributor

k77ch7 commented Feb 1, 2015

@enebo this still doesn't work on current jruby-1_7. Should I open a different PR against jruby-1_7? Thank you.

@enebo

This comment has been minimized.

Copy link
Member

enebo commented Feb 2, 2015

@k77ch7 If you are unable to make a new PR I can try and cherry-pick this but it is simpler if you make a new PR

@enebo

This comment has been minimized.

Copy link
Member

enebo commented Feb 2, 2015

@k77ch7 1.9+ bugs are a little simpler for us to merge if the PR is submitted to jruby-1_7 first.

@k77ch7

This comment has been minimized.

Copy link
Contributor

k77ch7 commented Feb 3, 2015

@enebo I just submitted the PR #2554 to jruby-1_7.

@k77ch7

This comment has been minimized.

Copy link
Contributor

k77ch7 commented Feb 3, 2015

@enebo Thanks for your quick response.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.