Regular expressions: "invalid bound for character class range" despite "\x{...}" formulation

[stefanspringer1]

Describe the bug

Regular expressions may result in the compilation error "invalid bound for character class range" despite using the "\x{...}" formulation.

That happens for combining characters, but also happens for some letter like symbols.

To Reproduce

 _ = "abc".contains(/[\x{33D}-\x{344}]/) // compilation error: Cannot parse regular expression: invalid bound for character

Expected behavior
For codepoints referenced by the "\x{...}" syntax, the mentioned error should not occur (at least some other regex tools accept them in the example above).

Configuration (please complete the following information):

• Swift Version: swiftlang-6.1.2.1.2 clang-1700.0.13.5)
• OS: macOS
• OS Version: 15.5

Regression information:
Always have been a problem.

Additional context
If the codepoints in the range are all listed separately, the regex expression is accepted.

[stefanspringer1]

...It is not only about an isolated range, but:

 _ = "abc".contains(#/[\x{31C}-\x{333}]/#) // OK
 _ = "abc".contains(#/[\x{339}-\x{33C}\x{347}-\x{349}]/#) // error: '̼͇' is an invalid bound for character class range
 _ = "abc".contains(#/[\x{347}-\x{349}]/#) // OK

Note that in the middle the ranges from the first and the last expression are both used sequentially.

[itingliu]

The function is in fact func contains(_ regex: some RegexComponent) -> Bool.

[jmschonfeld]

Apologies for the second transfer, but after speaking with @stephentyrone it sounds like this belongs in the Swift repo because it's an issue with regex literals which are parsed by the compiler (cc @milseman @natecook1000)

[natecook1000]

@stefanspringer1 Thanks for filing this issue! I opened #82535 for the behavior in your follow-up comment, which exposes a different problem.

In the original issue, what the compiler is identifying is that the upper bound of the character class range is not in NFC form. (U+0344 decomposes to U+308 U+301 and does not re-compose.) Currently, the parser requires that the bounds of a character class are both single-scalar characters and in NFC form.

Given that it seems like this kind of construction should be allowed, we could look at modifying those requirements, perhaps specifically for cases like this when both bounds are specified as Unicode scalar literals. At the same time, there's interplay between this kind of character class and matching in character mode vs scalar mode – when matching characters against character classes, we only match single-scalar characters, which implies the limitations on the range boundaries. For example:

let s = "e\u{301}"
s.contains(#/[a-f]/#) // false
s.contains(#/[a-f]/#.matchingSemantics(.unicodeScalar)) // true

@milseman what do you think here? Does it make sense to modify the parser to allow any single scalar bounds when they're specified using this syntax, since it seems like we're requiring single-scalar comparisons at the matching site?

[stefanspringer1]

In the original issue, what the compiler is identifying is that the upper bound of the character class range is not in NFC form. (U+0344 decomposes to U+308 U+301 and does not re-compose.) Currently, the parser requires that the bounds of a character class are both single-scalar characters and in NFC form.

Generally and from a user’s standpoint, U+0344 Is just a Unicode codepoint and normalization is something “extra” that might be desired or not. E.g. parsing an XML 1.0 document which is still the most used XML version would keep U+0344 and you then have no chance to find that character by using the mentioned range. As a simple solution you could break those ranges into a list of all characters in that range which makes the application of such a search way more inefficient.