Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Match a string with tokenizer only if it is first token in line #61

Closed
dhowe opened this issue Oct 7, 2018 · 2 comments
Closed

Match a string with tokenizer only if it is first token in line #61

dhowe opened this issue Oct 7, 2018 · 2 comments

Comments

@dhowe
Copy link

dhowe commented Oct 7, 2018

When tokenizing, how to match a token only if it is the first thing in a line?

Take the following example where I am trying to match any ident followed by a colon, ONLY if it is the first thing in the line (thus the ^):

static TextParser<Unit> Actor { get; } =
  from start in Span.Regex(@"^[A-Za-z][A-Za-z0-9_]+:")
  select Unit.Value;

It seems that if the line is "1 abc:" and Ignore(Span.WhiteSpace) is set, then the tokenizer consumes the first token ('1'), then ignores the white space as directed, then sees "abc:" as starting from position 0, thus matching.

But what I want is to only match "abc:" if it is the first token ... Is this possible ?

@dhowe dhowe changed the title Match a string only if it is the first thing in a line? Match a string only if it is starting a line Oct 7, 2018
@dhowe dhowe changed the title Match a string only if it is starting a line Match a string with tokenizer only if it begins a line Oct 7, 2018
@dhowe dhowe changed the title Match a string with tokenizer only if it begins a line Match a string with tokenizer only if it is first token in line Oct 9, 2018
@dhowe
Copy link
Author

dhowe commented Oct 28, 2018

@AndrewSav
Copy link
Contributor

It looks like the answer is given to you on StackOverflow. Basically it boils down to the fact that if TokenizerBuilder does not do what you want you need to build you Tokenizer yourself. Is there anything else with this question that is unresolved?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants