Characters higher than \x7f are being changed to \xff during parser generation

[deschutron]

Hi, I had a rule in my grammar that referred to characters higher than 127 ('\x7f'), and it changed all the character escape codes in my rule to \xff.
This is the rule I started out with:

id_uc <-
	[\xc0-\xdf][\x80-\xbf] /
	[\xe0-\xef][\x80-\xbf][\x80-\xbf] /
	[\xf0-\xff][\x80-\xbf][\x80-\xbf][\x80-\xbf]

As I kept simplifying it to track down the problem, the generator kept changing the escape codes to \xff, and leaving lower ones (e.g. \x41) as they are.

I found the cause of the problem too:
On line 487, in the function escape_character, you have a bit that says (unsigned)ch instead of (unsigned char)ch.
Here is the code block that's in:

        if (ch >= '\x20' && ch < '\x7f')
            snprintf(*buf, 5, "%c", ch);
        else
            snprintf(*buf, 5, "\\x%02x", (unsigned)ch);

In your code, it seems you are using the unsigned cast to make sure the higher char values are being fed properly to snprintf, but the C type unsigned is equal to unsigned int, so snprintf is writing strings like \xffffffc0 into the string buffer.
Changing the cast to (unsigned char) makes write them do it correctly (e.g. \xc0).

So can you change that line to this for me?:

            snprintf(*buf, 5, "\\x%02x", (unsigned char)ch);

(Actually it would be for other users, because I've already fixed it in my downloaded copy.)

And thanks for the great tool.

[arithy]

Thank you for your report.
I'll fix it with other issues.