Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Number Filter Precision has weird rounding errors #1208

Closed
troynt opened this Issue · 3 comments

2 participants

@troynt

Hello,

Number Filter Precision has weird rounding errors.

http://jsfiddle.net/troynt/qzdR8/

number = 3.141592653589793

{{number | number: i}}

produces

3
3.1
3.14
3.142
3.1416
3.14159
3.141593
3.1415927
3.14159265
3.141592654
3.1415926536
3.14159265359
3.141592653590
3.1415926535898
3.14159265358979
3.141592653589793
3.1415926535897930
3.14159265358979300
3.141592653589793000
3.1415926535897930000
3.14159265358979360000 Where did that 6 come from?
3.141592653589793000000
3.1415926535897930000000
3.14159265358979270000000 Where did that 27 come from?
3.141592653589793000000000

@btford
Owner

I suspect that this is a JavaScript problem (arising from the IEEE 754 Double Precision floating point implementation), and not an AngularJS problem. See: http://yuiblog.com/blog/2009/03/10/when-you-cant-count-on-your-numbers/

You might want to look into: https://github.com/dtrebbien/BigDecimal.js (you will have to write your own filter to interact with this type)

I'll take a closer look into it as well. Is there some case where you need this level of precision?

@troynt

Yea, I think this level of precision shouldn't be expected in JavaScript.

I initially thought this was an issue with Angular, but I think JS just can't handle these numbers.

e.g.
Math.PI.toFixed(20)
"3.14159265358979311600"

Math.PI.toFixed(21)
"RangeError: toFixed() digits argument must be between 0 and 20"

@btford
Owner

I reviewed the code, and I'm confident that AngularJS itself isn't causing the issue.

@btford btford closed this
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.