Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Reduce the chance of a rounding error when converting Float to BigDecimal. #53

Open
wants to merge 1 commit into from

1 participant

@pnomolos

On Ruby 1.9.3 on OS X Lion, doing 67.15.to_d.to_s('F') will return 67.15000000000001. By passing the precision to to_d it will avoid the likelihood of a slight rounding error.

@pnomolos pnomolos On Ruby 1.9.3 on OS X Lion, doing 67.15.to_d.to_s('F') will return 67…
….15000000000001. By passing the precision to to_d it will avoid the likelihood of a slight rounding error.
61f9bba
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Feb 6, 2012
  1. @pnomolos

    On Ruby 1.9.3 on OS X Lion, doing 67.15.to_d.to_s('F') will return 67…

    pnomolos authored
    ….15000000000001. By passing the precision to to_d it will avoid the likelihood of a slight rounding error.
This page is out of date. Refresh to see the latest.
Showing with 1 addition and 1 deletion.
  1. +1 −1  lib/data_mapper/validation/rule/numericalness.rb
View
2  lib/data_mapper/validation/rule/numericalness.rb
@@ -68,7 +68,7 @@ def valid?(resource)
def value_as_string(value)
case value
# Avoid Scientific Notation in Float to_s
- when Float then value.to_d.to_s('F')
+ when Float then value.to_d(options.fetch(:precision, nil)).to_s('F')
when BigDecimal then value.to_s('F')
else value.to_s
end
Something went wrong with that request. Please try again.