Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Division with BigDecimal not correct #648

Closed
michaelgpearce opened this Issue Apr 18, 2013 · 4 comments

Comments

Projects
None yet
4 participants
@michaelgpearce
Copy link

commented Apr 18, 2013

Division with BigDecimal is not performing correctly. Notice the incorrect result in JRuby and the correct result in MRI with the same code.

Here is a JRuby example:

java -jar bin/jruby-complete-1.7.3.jar -e 'require "bigdecimal"; puts (BigDecimal.new(1.05, 10) / (0.48 + 1.0)).to_s'
0.105E1

Here is the same example on MRI:

m-mpearce [with_javascript_and_jruby] ~/projects/main/backend/pricing/pws$ ruby -e 'require "bigdecimal"; puts (BigDecimal.new(1.05, 10) / (0.48 + 1.0)).to_s'
0.7094594594594595
@michaelgpearce

This comment has been minimized.

Copy link
Author

commented Apr 19, 2013

Working around with:

class BigDecimal
  alias :/ :fdiv
end
@vfrride

This comment has been minimized.

Copy link

commented Apr 19, 2013

Duplicate of #644

Reported in JIRA

Thanks for adding the workaround!

@ghost ghost assigned tychobrailleur May 28, 2013

@tychobrailleur

This comment has been minimized.

Copy link
Contributor

commented Jun 10, 2013

When dividing BigDecimal by a float, the float is converted into a rational, and then we re-divide the numerator by the denominator of the rational until it gets a “good precision” (see getVpRubyObjectWithPrec19Inner). The original problem was caused by the use of div19 , which does a floor on the resulting float.

I have a partial fix here: tychobrailleur@737a388, which also takes care of return types, and I have added some rubyspecs here: https://github.com/rubyspec/rubyspec/pull/220

However, I am still having rounding issues caused by getVpRubyObjectWithPrec19Inner’s logic which tries to mimic MRI’s logic, but doesn’t fully implement it (esp. call to BigDecimal_div2).

@enebo Do you have any objection with replacing the recursive logic of getVpRubyObjectWithPrec19Inner with Java’s java.math.BigDecimal? It seems to be it would be a lot cleaner, and potentially more efficient.

@BanzaiMan

This comment has been minimized.

Copy link
Member

commented Jun 17, 2013

Fixed with b5069f8.

@BanzaiMan BanzaiMan closed this Jun 17, 2013

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.