Handle line continuation in Lexer::peek_char #43
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The existing lexer implementation requires that the
Lexer::line_continuations
function be called every time before peeking or skipping a character where line continuation is allowed. This is error-prone because it can only covered by tests if we forget to call the function.This PR modifies the lexer to automatically skip line continuation before peeking so that we no longer need to call the function, except in a few contexts where line continuation should not be recognized, i.e., single quotes, literal here-document contents, and comments.
LexerCore
Lexer::peek_char