Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

33.3333除以100 在浏览器上计算出现问题 #19

Closed
UnlimitedGalaxy opened this issue Aug 29, 2019 · 2 comments
Closed

33.3333除以100 在浏览器上计算出现问题 #19

UnlimitedGalaxy opened this issue Aug 29, 2019 · 2 comments

Comments

@UnlimitedGalaxy
Copy link

NP.divide(33.3333, 100) = 0.33333300000000005

@lxfriday
Copy link
Collaborator

lxfriday commented Oct 21, 2019

@UnlimitedGalaxy @camsong Chrome 表现不太一致

console.log('divide', divide(33.3333, 100), 33.3333 / 100);

Chrome浏览器 异常

divide 0.33333300000000005 0.333333

Firefox、IE、Edge 正常

divide 0.333333 0.333333

node v10.16.3 正常

divide 0.333333 0.333333

@lxfriday
Copy link
Collaborator

@camsong 问题的环节定位到了

image

image

原因是 Math.pow(10, 0 - 4) => 在 Chrome 上 0.00009999999999999999,在 FF、IE、Edge、node 上都是 0.0001

divide 最后一步的计算就变成了 times(3333.33, 0.00009999999999999999)

接着 float2Fixed(num2) => 10000000000000000 这个数已经超过了 Number.MAX_SAFE_INTEGER

image

后面的计算就没有意义了, num1Changed * num2Changed => 会变得更大

@lxfriday lxfriday mentioned this issue Oct 21, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants