UTF-8 characters are not managed properly

[mingodad]

Testing this Javascript grammar https://github.com/pegjs/pegjs/blob/master/examples/javascript.pegjs converted to peglib I've noticed that peglib doesn't managed single char unicode properly (like shown bellow) because it stores it internally as char without any warning/error, see here

cpp-peglib/peglib.h

Line 1306 in 527422a

char ch_;

where it should be char32_t like here

cpp-peglib/peglib.h

Line 1286 in 527422a

std::vector<std::pair<char32_t, char32_t>> ranges_;

LineTerminatorSequence #"end of line"
  <- "\n"
  / "\r\n"
  / "\r"
  / "\u2028" #silently truncated
  / "\u2029" #silently truncated

[mingodad]

Today I was converting this grammar https://github.com/GuntherRademacher/rex-parser-generator/blob/main/rex/src/parser/EbnfParser.ebnf to use on https://mingodad.github.io/cpp-peglib/ and found that peglib also silently accept \x123456 hex codes bigger than \xFF but silently trunc it without any warning/error what can make find/debug a grammar a bit difficult.
You can see it on https://mingodad.github.io/cpp-peglib by selecting rex-parser-generator parser from Examples then clicking Parse to see that it's parsing the original grammar without problems but if we change any of the '\uXXXXvalues to\xXXXX` like in the original grammar it start showing parse errors on the input but no warning/error for the grammar:

To see the problem change grammar line 160 from:

/ [\u203F-\u2040]

to:

/ [\x203F-\x2040]

[mingodad]

My bad !
The grammar only accept 1 or 2 hexadecimal digits for '\xXX, anyway would be nice to have an option on peglint` to warn about possible mistakes like the one I was facing due to not properly pay attention to details.

[yhirose]

[\x203F-\x2040] is equivalent to [ 03F- 40], and it's ok in syntax wise. We could report "F- isn't a correct character range", and "0 and duplicates".