Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

empty string with non-empty trailing context consumes code units #116

Closed
skvadrik opened this issue Jul 30, 2015 · 1 comment

Comments

@skvadrik
Copy link
Owner

commented Jul 30, 2015

Given the following code:

/*!re2c
    "" / "a" {}
*/

re2c generates:

{
        YYCTYPE yych;
        YYCTXMARKER = YYCURSOR + 1;
        if (YYLIMIT <= YYCURSOR) YYFILL(1);
        yych = *YYCURSOR;
        switch (yych) {
        case 'a':       goto yy3;
        default:        goto yy2;
        }
yy2:
yy3:
        ++YYCURSOR;
        YYCURSOR = YYCTXMARKER;
        {}
}

On strings starting with "a" this lexer will consume single code unit ('a') and advance YYCURSOR to code unit next to 'a'. This is clearly not what was intended.

Lookahead is a useful feature and this should be fixed.

@skvadrik skvadrik self-assigned this Jul 30, 2015

skvadrik added a commit that referenced this issue Oct 4, 2015

Fixed bug #116: "empty string with non-empty trailing context consume…
…s code units".

Prior to this commit backup of trailing context position was done
before advancing input position and re2c either had to emit
    YYCTXMARKER = YYCURSOR + 1;
(with default input API), or
    YYRESTORECTX ();
    YYSKIP ();
(with custom input API).

The problem is that sometimes initial state doesn't sdvance input position
at all. Now re2c emits context backup after advancing input position and it
no longer needs '+1' or 'YYSKIP' hacks. It always backups the correct position.
@skvadrik

This comment has been minimized.

Copy link
Owner Author

commented Oct 4, 2015

Fixed in this commit.

@skvadrik skvadrik closed this Oct 4, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
1 participant
You can’t perform that action at this time.