Incorrect values returned by `Real.fromString` on strings with many digits

[shwestrick]

Version

110.99.8 (Latest)

Operating System

• Any
• Linux
• macOS
• Windows
• Other Unix

OS Version

macOS Sequoia 15.5

Processor

• Any
• Arm (using Rosetta)
• PowerPC
• Sparc
• x86 (32-bit)
• x86-64 (64-bit)
• Other

System Component

Basis Library

Severity

Major

Description

Real.fromString returns incorrect values. I've found a few examples, see below.

My best guess at the moment: the bug seems to occur when there are 18 or more decimal digits past the decimal point. All of the examples below have 18 decimal digits. Increasing the number of decimal digits even further causes the outputs to change.

Transcript

A few examples of incorrect outputs from Real.fromString:

Standard ML of New Jersey [Version 110.99.8; 64-bit; April 25, 2025]
- Real.fromString "0.501782303180000055";
val it = SOME 502.812494993 : real option
- Real.fromString "0.211111111111111111";
val it = SOME 9.46111993270 : real option
- Real.fromString "0.311111111111111111";
val it = SOME 418.928139547 : real option
- Real.fromString "0.411111111111111111";
val it = SOME 316.528139547 : real option

Expected Behavior

Not sure about exact values (this will depend on printing conventions), but the above are clearly wrong.

FWIW, here is output from MLton:

$ cat foo.sml
fun dump x =
  print (Real.fmt StringCvt.EXACT (valOf (Real.fromString x)) ^ "\n")
val () = dump "0.501782303180000055"
val () = dump "0.211111111111111111"
val () = dump "0.311111111111111111"
val () = dump "0.411111111111111111"

$ mlton foo.sml

$ ./foo
0.50178230318
0.2111111111111111
0.3111111111111111
0.4111111111111111

Steps to Reproduce

See above.

Additional Information

No response

Email address

github username at gmail

[JohnReppy]

Definitely an issue with the number of digits

- Real.fromString "0.50178230318000005";
val it = SOME 0.50178230318 : real option
- Real.fromString "0.501782303180000055";
val it = SOME 502.812494993 : real option

[JohnReppy]

I've done a bit more digging into this problem and I think that the problem is that the input has too many digits, which violates some of the assumptions in the Real/frep-to-real.sml code. I have to think about where we should trim the digit list, but I think that doing so will fix the issue.

[JohnReppy]

I have pushed a fix for this issue, which will be included in the 110.99.9 release. There were two parts to the fix. First, we need to normalize the results of scanning a floating-point literal in a string (this process also handles numbers that are too big, etc.) and, second, normalization needs to trim excess digits (more than 17 for double-precision).

There is a remaining minor problem in that the result of

Real.fromString "0.501782303180000059";

should probably be the same as

Real.fromString "0.50178230318000006";

(i.e., we should round when trimming the excess digits). I have left a note in the source code that rounding should be added.