-
-
Notifications
You must be signed in to change notification settings - Fork 112
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Token or regex not matched as expected when it's prefix of an another, larger regex #227
Comments
I think longer regexs are preferred, see similar issue here: #255 |
This is not about longest match rule, logos just doesn't support backtracking. If you try my example with any other lexer generator with the longest match rule (alex, ocamllex, lexgen, ...) you'll see that they all accept the last case. |
@osa1 hmmm... I assumed longer I wonder as a worked around if (presuming spaces get #[regex("a+b")]
APlusB,
// make it longer.... or add higher priority...
#[regex("a *")]
A, These are the sorts of hacks I often employ. I understand it's adding cruft and stink... |
Just ran into this myself. Regex priority is irrelevant here--if a regex fails to match, Logos backtracks to a lower priority regex/token anyway. Or rather, is supposed to--it seems Logos fails to backtrack correctly when encountering The
|
I believe this to be related to #279. I don't think this is fixable until the derive macro is rewritten (see 0.13 release notes). |
I'm using current git master branch (925c49e).
Repro:
The first two tests are fine. In the third case, I would expect the lexer to yield two
Token::A
s, but instead it yieldsError
.Is this expected?
The text was updated successfully, but these errors were encountered: