-
Notifications
You must be signed in to change notification settings - Fork 94
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversion of double to string can result in a loss of precision #408
Comments
Hi @btasker, It looks like that the generated output depends on the platform. On my laptop the generated output is:
My platform:
What platform does the customer use? According to the
Regards |
Hey @bednar Agreed - it looks like I can't repro in Docker on Linux either
As MS have the tags, I've tested with SDK 5.0 and can't repro there either. I believe the customer is on Windows - I'll check which version (and SDK etc). |
My earlier Repro was done using this - https://dotnetfiddle.net/srx9kM Looking a bit closer at that, by default it's using .NET 4.7.2. If I switch it to using .NET 6 the results are as expected - so this probably does only affect older versions |
I tested on |
Do we have a confirmed version of ".NET" from the customer? |
Not currently, I asked last week but haven't heard back. Will chase up later today. |
@bednar The customer is running 4.6.1 However, they've noted that the Net 7 docs refer to this http://msdn.microsoft.com/en-us/library/kfsatb94.aspx
|
When the client starts building LP, it converts double and float values to a string:
However,
CultureInfo.InvariantCulture
may not preserve full precision.Whereas, if that's replaced with
"R"
the precision will be maintained:For example
Generates
However, this isn't a full solution, because the use of
R
may also add unexpected precision. If we redefinea
to be45.29587181323112
in the example above, then we get45.295871813231123
because of the way that C# handles conversion.Steps to reproduce:
List the minimal actions needed to reproduce the behavior.
Expected behavior:
The full precision should be sent
Actual behavior:
The value will have lost precision
Specifications:
The text was updated successfully, but these errors were encountered: