Use BigDecimal and Integer where appropriate to prevent floating point accuracy loss #43

joshwlewis · 2017-07-02T15:24:52Z

This partially addresses the concerns in #35 about losing accuracy with Floats. Most notably, this gives much greater accuracy for Pi.

joshwlewis changed the title ~~Use BigDecimal where appropriate to prevent floating point accuracy loss~~ Use BigDecimal and Integer where appropriate to prevent floating point accuracy loss Jul 2, 2017

joshwlewis added 6 commits July 2, 2017 18:36

Extract number simplification

fdefba9

Drop some redundant tests

38ae5e3

Prevent potential float accuracy loss by using appropriate Numeric type

729c03b

Also avoid precision loss with prefixes

75080d6

Regenerate base data with higher accuracy

a15c837

Update changelog with base unit type change

1535325

joshwlewis force-pushed the data-precision branch from bf23210 to 1535325 Compare July 2, 2017 23:37

joshwlewis added 2 commits July 2, 2017 18:41

Add a test to cover #inspect.

aac968b

Additional test cases for Number#simplify

d9c7907

joshwlewis merged commit b048507 into master Jul 3, 2017

joshwlewis deleted the data-precision branch July 3, 2017 00:18

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use BigDecimal and Integer where appropriate to prevent floating point accuracy loss #43

Use BigDecimal and Integer where appropriate to prevent floating point accuracy loss #43

joshwlewis commented Jul 2, 2017 •

edited

Loading

Use BigDecimal and Integer where appropriate to prevent floating point accuracy loss #43

Use BigDecimal and Integer where appropriate to prevent floating point accuracy loss #43

Conversation

joshwlewis commented Jul 2, 2017 • edited Loading

joshwlewis commented Jul 2, 2017 •

edited

Loading