Division with BigDecimal not correct #648

Closed
michaelgpearce opened this Issue Apr 18, 2013 · 4 comments

Projects

None yet

4 participants

@michaelgpearce

Division with BigDecimal is not performing correctly. Notice the incorrect result in JRuby and the correct result in MRI with the same code.

Here is a JRuby example:

java -jar bin/jruby-complete-1.7.3.jar -e 'require "bigdecimal"; puts (BigDecimal.new(1.05, 10) / (0.48 + 1.0)).to_s'
0.105E1

Here is the same example on MRI:

m-mpearce [with_javascript_and_jruby] ~/projects/main/backend/pricing/pws$ ruby -e 'require "bigdecimal"; puts (BigDecimal.new(1.05, 10) / (0.48 + 1.0)).to_s'
0.7094594594594595
@michaelgpearce

Working around with:

class BigDecimal
  alias :/ :fdiv
end
@vfrride
vfrride commented Apr 19, 2013

Duplicate of #644

Reported in JIRA

Thanks for adding the workaround!

@tychobrailleur
Contributor

When dividing BigDecimal by a float, the float is converted into a rational, and then we re-divide the numerator by the denominator of the rational until it gets a “good precision” (see getVpRubyObjectWithPrec19Inner). The original problem was caused by the use of div19 , which does a floor on the resulting float.

I have a partial fix here: tychobrailleur@737a388, which also takes care of return types, and I have added some rubyspecs here: https://github.com/rubyspec/rubyspec/pull/220

However, I am still having rounding issues caused by getVpRubyObjectWithPrec19Inner’s logic which tries to mimic MRI’s logic, but doesn’t fully implement it (esp. call to BigDecimal_div2).

@enebo Do you have any objection with replacing the recursive logic of getVpRubyObjectWithPrec19Inner with Java’s java.math.BigDecimal? It seems to be it would be a lot cleaner, and potentially more efficient.

@BanzaiMan
Member

Fixed with b5069f8.

@BanzaiMan BanzaiMan closed this Jun 17, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment