-
Notifications
You must be signed in to change notification settings - Fork 96
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Simple tokenizer question #58
Comments
Hi! The tokenizer will consider the whole span matched by the parser to be the token, regardless of the value that's returned, but, you can use For this to work you need to drop off the static TextParser<Unit> HashIdent { get; } = Character.EqualTo('#').IgnoreThen(Identifier.CStyle) HTH! |
Thanks, thats very helpful (I was just wondering what the apply was doing in your example code)! When I do that, I get a compile error... do I need to change the generic type to TextSpan? |
Also, how do I handle the opposite case, where I want to ignore the last character? static TextParser<Unit> Actor { get; } =
from name in Character.LetterOrDigit.Many()
from last in Character.EqualTo(':')
select Unit.Value; |
Yes, dropping As for your second question something like this would probably work:
|
I guess this water is long under the bridge - hope you found a good solution, @dhowe - closing as stale, but keep us posted :-) |
I'm trying to match c-style identifiers that start with a '#' character, but to discard the hash character and keep the rest. In my attempts below the '#' is always included in the token:
I know I'm missing something simple...
The text was updated successfully, but these errors were encountered: