StringComparison.OrdinalIgnoreCase does not consistently match ToUpperInvariant

[GrabYourPitchforks]

Per the following two documents, comparing strings using StringComparison.OrdinalIgnoreCase is explicitly documented as equivalent to calling ToUpperInvariant on each string, then performing an ordinal comparison against the contents.

• https://docs.microsoft.com/dotnet/api/system.stringcomparer.ordinalignorecase#remarks
• https://docs.microsoft.com/dotnet/standard/base-types/best-practices-strings#ordinal-string-operations

These statements within the docs imply that ToUpperInvariant and ToLowerInvariant are ordinal case conversions ("simple case mapping"), not linguistic case conversions. However, it looks like we're not consistenly following this pattern.

string s1 = "s";
string s2 = "\u017f"; // Latin Sharp S, which uppercase-maps to a normal ASCII "S"
Console.WriteLine(s1.Equals(s2, StringComparison.OrdinalIgnoreCase)); // False
Console.WriteLine(s1.ToUpperInvariant() == s2.ToUpperInvariant()); // True

This has collateral impact. For example, recent PRs like #67758 assume that non-ASCII characters cannot case-map to ASCII characters, which is not a guarantee offered by Unicode, but which might be a guarantee we'd be willing to make separately within the runtime by munging the Unicode tables.

See also #30960 for further discussion on case mapping as a more general Unicode concept.

/cc @tarekgh, who had thoughts on this offline.

[ghost]

Tagging subscribers to this area: @dotnet/area-system-globalization
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Per the following two documents, comparing strings using StringComparison.OrdinalIgnoreCase is explicitly documented as equivalent to calling ToUpperInvariant on each string, then performing an ordinal comparison against the contents.

• https://docs.microsoft.com/dotnet/api/system.stringcomparer.ordinalignorecase#remarks
• https://docs.microsoft.com/dotnet/standard/base-types/best-practices-strings#ordinal-string-operations

These statements within the docs imply that ToUpperInvariant and ToLowerInvariant are ordinal case conversions ("simple case mapping"), not linguistic case conversions. However, it looks like we're not consistenly following this pattern.

string s1 = "s";
string s2 = "\u017f"; // Latin Sharp S, which uppercase-maps to a normal ASCII "S"
Console.WriteLine(s1.Equals(s2, StringComparison.OrdinalIgnoreCase)); // False
Console.WriteLine(s1.ToUpperInvariant() == s2.ToUpperInvariant()); // True

This has collateral impact. For example, recent PRs like #67758 assume that non-ASCII characters cannot case-map to ASCII characters, which is not a guarantee offered by Unicode, but which might be a guarantee we'd be willing to make separately within the runtime by munging the Unicode tables.

See also #30960 for further discussion on case mapping as a more general Unicode concept.

/cc @tarekgh, who had thoughts on this offline.

Author:	GrabYourPitchforks
Assignees:	-
Labels:	area-System.Globalization
Milestone:	-