LWG4087 Standard exception messages have unspecified encoding

[jwakely]

https://cplusplus.github.io/LWG/issue4087

LWG would like SG16 to discuss this issue please.

[tahonermann]

SG16 discussed this issue during its 2024-06-12 and 2024-07-31 meetings. No polls were taken. SG16 will review again in the new year.

[inbal2l]

@tahonermann - please update once SG16 is done with it.

[tahonermann]

Now that I've finally published a meeting summary for the 2024-06-12 SG16 meeting, I can detail the next steps for resolving this issue.

The SG16 discussion concluded that the what() member function has the following requirements:

• It must return a string that is constructed at the time the exception object is constructed.
• It must return a NTBS (which may be a NTMBS).
• The string must contain the exact sequence of code units provided for the what_arg (if any) of the exception class constructor; no transcoding of that argument is permitted.
• The returned string must be suitable for conversion to a wide string using the std::codecvt<char, wchar_t> specialization that converts from the encoding of the execution character set to the encoding of the execution wide-character set.

Satisfying these requirements implies the following:

• The string provided as the what_arg (if any) for the exception class constructor must be encoded in the encoding of the execution character set.
• When the exception class constructor builds the string to be returned by what(), it must convert any content it adds to the encoding of the execution character set. This potentially requires converting literal text (in the ordinary literal encoding), file system paths (in the native ordinary encoding), error messages from std::error_category (in the system/environment encoding), etc... to the encoding of the execution character set.

Failure to produce a consistently encoded string would violate the wide string convertibility requirements (and result in mojibake generally).

If the encoding of the execution character set is changed (e.g., via a call to std::setlocale()), after constructing an exception object, but before what() is called on that object, then the string that is returned will satisfy the NTBS requirement (all valid NTMBSs are valid NTBSs for all conforming execution character set encodings), but might not be suitable for loss-less error-free conversion to a wide string.

There is no way to query or reliably infer what the current execution character set was at the time an exception object was constructed, nor is there a reasonable way to store that information to make it available to callers of what() without an ABI break. Normative encouragement should be provided for use of the encoding of the current execution character set at the time what() is called to interpret the returned string with additional guidance to appropriately process ill-formed code unit sequences if it is possible that the encoding of the execution character set has changed compared to when the exception object was created. That encouragement should then be applied to std::format(); it should use the encoding of the current execution character set for any transcoding needs and format any ill-formed code unit sequences using the escaped string representation ([format.string.escaped]).

A proposed resolution is needed to make further progress on this issue. I'll ask SG16 participants for a volunteer to draft wording. Once a PR materializes, I'll have SG16 confirm it matches the intent.