Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Calc: calculation error with decimal number #98

Closed
jrlacharnay opened this issue Jul 23, 2016 · 5 comments
Closed

Calc: calculation error with decimal number #98

jrlacharnay opened this issue Jul 23, 2016 · 5 comments

Comments

@jrlacharnay
Copy link

@jrlacharnay jrlacharnay commented Jul 23, 2016

Hello,

When entering the following calculation: "500.10-400", the displayed result is not "100.10" as expected, but "100.10000000000002".

My version:

10:16:14.390 Keypirinha 2.8 (246eefb) for x64
10:16:14.390 Portable mode.
10:16:14.390 System: WinNT-x64 10.0.10586-ws-0x0300

Thanks!
Jean-Rémy

@jrlacharnay
Copy link
Author

@jrlacharnay jrlacharnay commented Jul 23, 2016

calc issue

@polyvertex
Copy link
Member

@polyvertex polyvertex commented Jul 23, 2016

A way to deal with that would be for the Calc package to offer a "rounding" option so the user can specify the number of digits displayed after the separator by asking Calc to round it. This is something that can be added to a future release.

FYI and if that is of any interest to you, for now the displayed value is computed by the embedded Python interpreter and returned as-is by the Calc plugin. Python itself is dependent, at least partly, on the way the CPU deals with floating point arithmetic.

@ThaSiouL
Copy link

@ThaSiouL ThaSiouL commented Aug 3, 2016

here is an updated calc version by myself that deals with the issue (among others)
Calc.zip
(Updated version from the version I posted in #70 and #93 )

EDIT by @polyvertex: @ThaSiouL has published its contribution in a GitHub repository

This was referenced Aug 3, 2016
@ThaSiouL
Copy link

@ThaSiouL ThaSiouL commented Aug 3, 2016

2016-08-03 20_18_52-program manager
including .ini decimal sign

@polyvertex
Copy link
Member

@polyvertex polyvertex commented Aug 20, 2016

The Calc package has been greatly improved and released with v2.9.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.