Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rpc: Avoid unnecessary parsing roundtrip in number formatting, fix locale issue #6456

Merged
merged 3 commits into from
Jul 24, 2015

Conversation

laanwj
Copy link
Member

@laanwj laanwj commented Jul 18, 2015

Three weakly related fixes to number handling after the univalue switch. These came to the surface while troubleshooting #6443.

Make ValueFromAmount always return 8 decimals

This is the format that was always returned to JSON clients. The difference was not noticed before, because VREAL values are post-processed by univalue.

By implementing the functionality directly it breaks the dependency of rpcserver on utilmoneystr. FormatMoney is now only used for debugging purposes.

To test, port over the formatting tests from util_tests.cpp to rpc_tests.cpp.

univalue: Avoid unnecessary roundtrip through double for numbers

JSON makes no distinction between numbers and reals, and our code doesn't need to do so either.

This removes VREAL, as well as its specific post-processing in UniValue::write. Non-monetary amounts do not need to be forcibly formatted with 8 decimals, so the extra roundtrip was unnecessary (and potentially loses precision).

util: use locale-independent parsing in ParseDouble

Use locale-indepent C++ based parsing instead of C's strtod, which checks for different input based on the user's locale.
Fixes #6443.

This is the format that was always returned to JSON clients.
The difference was not noticed before, because VREAL values
are post-processed by univalue.

By implementing the functionality directly it breaks the dependency
of rpcserver on utilmoneystr. FormatMoney is now only used for debugging
purposes.

To test, port over the formatting tests from util_tests.cpp to
rpc_tests.cpp.
JSON makes no distinction between numbers and reals, and our code
doesn't need to do so either.

This removes VREAL, as well as its specific post-processing in
`UniValue::write`. Non-monetary amounts do not need to be forcibly
formatted with 8 decimals, so the extra roundtrip was unnecessary
(and potentially loses precision).
Use locale-indepent C++ based parsing instead of C's strtod,
which checks for different input based on the user's locale.
Fixes bitcoin#6443.
@laanwj laanwj changed the title rpc: Avoid unnecessary roundtrips in number parsing, fix locale issue rpc: Avoid unnecessary parsing roundtrip in number formatting, fix locale issue Jul 18, 2015
@laanwj laanwj added the Bug label Jul 18, 2015
@dexX7
Copy link
Contributor

dexX7 commented Jul 18, 2015

I can confirm this resolves #6443, thanks!

As far as I can see the output is as expected, except for the difficulty, returned by "getinfo", which is shown in e notation ("difficulty": 4.656542373906925e-10 vs. 0.00000000, as returned by Core 11.0). Edit: I wouldn't say this is bad though, as it offers a greater precision.

@laanwj
Copy link
Member Author

laanwj commented Jul 20, 2015

as returned by Core 11.0). Edit: I wouldn't say this is bad though, as it offers a greater precision

Exactly - the fixed 8-digit formatting is for clarity in monetary amounts.
Earlier it was applied to all numbers because JSON spirit had no other way.
But now we have full control over how values are rendered, and other floating point quantities such as bits, difficulty can be represented in full precision.

@jonasschnelli
Copy link
Contributor

Tested ACK.

If i understand this right, then there is a slightly API change. Every double/float (non-monetary) comes now with a precision of 16 instead of 8 (because of https://github.com/bitcoin/bitcoin/pull/6456/files#diff-0f1b401041a14398229cf7e31b6db7eeR89). Because we are using oss << std::setprecision(16) << val it will rewrite things like 0.00000000046565423739069247 into scientific notation. That is what @dexX7 has detected.

@laanwj
Copy link
Member Author

laanwj commented Jul 20, 2015

Right. All numbers were 'cut' to %.8f notation, so it would have reported 0.00000000046565423739069247 as 0.00000000 instead of going to scientific notation.
E.g. a difficulty smaller than a "satoshi" was reported as 0. This makes no sense at all for non-monetary numbers. This is all within the limits of number representation in the JSON protocol.

@jgarzik
Copy link
Contributor

jgarzik commented Jul 23, 2015

ACK. Happy to see VREAL go away.

@laanwj laanwj merged commit ec249d4 into bitcoin:master Jul 24, 2015
laanwj added a commit that referenced this pull request Jul 24, 2015
ec249d4 util: use locale-independent parsing in ParseDouble (Wladimir J. van der Laan)
7650449 univalue: Avoid unnecessary roundtrip through double for numbers (Wladimir J. van der Laan)
e061e27 rpc: Make ValueFromAmount always return 8 decimals (Wladimir J. van der Laan)
zkbot added a commit to zcash/zcash that referenced this pull request Feb 10, 2017
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Sep 8, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ParseDouble() is faulty, depending on locale
4 participants