lexer: lowercase-only delimiters for quoted strings

[Octachron]

This PR fixes #13626 by limiting quoted string delimiters to alphabetical lowercase identifiers (straße but not x1 nor x').

[gasche]

Naive questions:

• The bug report appeared to also be about single-quote ' being supported that previously was not, right? What is the impact of your PR on single quotes? Where are they hidden in the code?
• Is this conservative with respect to the lexing rules of 5.2? (How do I review this property?)

[Octachron]

1. The single quotes (and digits) are hidden in the definition of ident_ext:

let identchar = ['A'-'Z' 'a'-'z' '_' '\'' '0'-'9']
let identstart_ext = identstart | utf8
let identchar_ext = identchar | utf8
let identstart = lowercase | uppercase
let ident_ext = identstart_ext  identchar_ext*

compared to the new delim_ext:

let delim_ext = ( lowercase | uppercase | utf_8 ) *

(where uppercase is only added to benefits from the improved error messages).
2. You can compare with 5.2 with git diff 5.2.. -- parsing/lexer.mll . The difference with 5.2 is essentially that lowercase* is replaced by delim_ext. The absence of uppercase letters is then checked in the delimiter validation function, to either emit a syntax error or to continue lexing inside a comment.

[gasche]

Note to self: in the diff from 5.2, the _ext prefix generally means "extended" and that means "also with a restricted choice of extra utf8 characters", except for validate_ext where ext means "extension (point)".

[gasche]

Thanks for the extra comments. I think that I now understand the change and agree that it is safe -- the set of quoted string delimiters of the PR is closer to the one of 5.2, so there are less risks of regressions similar to #13626.

[gasche]

Maybe you could have a comment on the fact that uppercase is in fact rejected by the validate_delim check, we intentionally accept more to provide clearer / more specific error messages?

[Octachron]

I have added an error message to point out that we want the same error message for {A| and {À|.

[gasche]

Looks good to me. By default I assume that you will merge and cherry-pick in 5.3.

[Octachron]

Indeed.

[Octachron]

Cherry-picked on 5.3 as 66b424d .