Strange dump behavior for floats #78

Closed
austinthecoder opened this Issue Mar 19, 2013 · 5 comments

Comments

Projects
None yet
2 participants
@austinthecoder

Whenever the value is a specific float, it gets changed. Here's some examples:

> Oj.dump({'a' => 70.35})
=> "{\"a\":70.34999999999999}"

> Oj.dump({'a' => 60.35})
=> "{\"a\":60.35}"
@ohler55

This comment has been minimized.

Show comment Hide comment
@ohler55

ohler55 Apr 11, 2013

Owner

You will probably find that Ruby does something similar when printing. Floats are inherently imprecise. Others have noticed this and use BigDecimals instead when precision is important. There is an option in Oj to force the use of BigDecimal when loading. It is :big decimal_load.

Does that answer the question?

Owner

ohler55 commented Apr 11, 2013

You will probably find that Ruby does something similar when printing. Floats are inherently imprecise. Others have noticed this and use BigDecimals instead when precision is important. There is an option in Oj to force the use of BigDecimal when loading. It is :big decimal_load.

Does that answer the question?

@austinthecoder

This comment has been minimized.

Show comment Hide comment
@austinthecoder

austinthecoder Apr 12, 2013

Thanks for the response. I understand that Floats are imprecise, but I would expect that since 70.35.to_s is "70.35" then Oj.dump(70.35) would be the same. Does it not make sense for Oj.dump to just call to_s on the object? Is this because it's done in C?

Thanks for the response. I understand that Floats are imprecise, but I would expect that since 70.35.to_s is "70.35" then Oj.dump(70.35) would be the same. Does it not make sense for Oj.dump to just call to_s on the object? Is this because it's done in C?

@ohler55

This comment has been minimized.

Show comment Hide comment
@ohler55

ohler55 Apr 12, 2013

Owner

Oj uses the c sprintf() approach. Let me look a little deeper at it. Most of the time the results are the same. I'll see what I can do to make it more reasonable.

Owner

ohler55 commented Apr 12, 2013

Oj uses the c sprintf() approach. Let me look a little deeper at it. Most of the time the results are the same. I'll see what I can do to make it more reasonable.

@ohler55

This comment has been minimized.

Show comment Hide comment
@ohler55

ohler55 Apr 23, 2013

Owner

Changed the number of decimals to 15 instead of 16 and that seems to fix the problem. It will be in the next release.

Owner

ohler55 commented Apr 23, 2013

Changed the number of decimals to 15 instead of 16 and that seems to fix the problem. It will be in the next release.

@austinthecoder

This comment has been minimized.

Show comment Hide comment
@austinthecoder

austinthecoder Apr 23, 2013

Excellent! Thank you.

Excellent! Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment