Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Division with BigDecimal not correct #648

michaelgpearce opened this issue Apr 18, 2013 · 4 comments

Division with BigDecimal not correct #648

michaelgpearce opened this issue Apr 18, 2013 · 4 comments


Copy link

@michaelgpearce michaelgpearce commented Apr 18, 2013

Division with BigDecimal is not performing correctly. Notice the incorrect result in JRuby and the correct result in MRI with the same code.

Here is a JRuby example:

java -jar bin/jruby-complete-1.7.3.jar -e 'require "bigdecimal"; puts (, 10) / (0.48 + 1.0)).to_s'

Here is the same example on MRI:

m-mpearce [with_javascript_and_jruby] ~/projects/main/backend/pricing/pws$ ruby -e 'require "bigdecimal"; puts (, 10) / (0.48 + 1.0)).to_s'
Copy link

@michaelgpearce michaelgpearce commented Apr 19, 2013

Working around with:

class BigDecimal
  alias :/ :fdiv
Copy link

@vfrride vfrride commented Apr 19, 2013

Duplicate of #644

Reported in JIRA

Thanks for adding the workaround!

@ghost ghost assigned tychobrailleur May 28, 2013
Copy link

@tychobrailleur tychobrailleur commented Jun 10, 2013

When dividing BigDecimal by a float, the float is converted into a rational, and then we re-divide the numerator by the denominator of the rational until it gets a “good precision” (see getVpRubyObjectWithPrec19Inner). The original problem was caused by the use of div19 , which does a floor on the resulting float.

I have a partial fix here: tychobrailleur@737a388, which also takes care of return types, and I have added some rubyspecs here:

However, I am still having rounding issues caused by getVpRubyObjectWithPrec19Inner’s logic which tries to mimic MRI’s logic, but doesn’t fully implement it (esp. call to BigDecimal_div2).

@enebo Do you have any objection with replacing the recursive logic of getVpRubyObjectWithPrec19Inner with Java’s java.math.BigDecimal? It seems to be it would be a lot cleaner, and potentially more efficient.

Copy link

@BanzaiMan BanzaiMan commented Jun 17, 2013

Fixed with b5069f8.

@BanzaiMan BanzaiMan closed this Jun 17, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants