SExp: parsing the output of the printer produces a different SExp

[JohnReppy]

Version

No response

Operating System

• All
• Linux
• macOS
• Windows
• Other Unix

OS Version

No response

Processor

Any

Component

SML/NJ Library

Severity

Major

Description of the problem

"SExpParser.parse" from the SML/NJ library does not produce the same
s-expression that was written using "SExpPrinter.print" when strings
containing multibyte characters are involved. How to reproduce:

(* 1. Print str. )
val str = "∞"; ( Infinity character. *)
print str;

(* 2. Write str to a file as an s-expression. *)
val out = TextIO.openOut "infinity.txt";
SExpPrinter.print (out, SExp.STRING str);
TextIO.closeOut out;

(* 3. Attempt to read back str from the file. *)
case SExpParser.parseFile "infinity.txt" of
[SExp.STRING str'] => print str'
| _ => ();

In step 1, "∞" is displayed, but in step 3, "\226\136\158" is displayed.

This problem occurs because "SExpPrinter.print" escapes all strings
using "String.toString" but "SExpParser.parseFile" does not unescape
strings using "String.fromString".

Transcript

No response

Expected Behavior

No response

Steps to Reproduce

No response

Additional Information

No response

Email address

No response

Comments from smlnj-gforge

Original smlnj-gforge bug number 309

Submitted on 2022-05-17 at 06:03:00

comment by @JohnReppy on 2022-05-31 14:28:00 +000 UTC

Fixed for 110.99.3 and 2022.1. The SExp library now uses Scheme syntax for string values.