Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle line continuation in Lexer::peek_char #43

Merged
merged 7 commits into from
May 22, 2021
Merged

Handle line continuation in Lexer::peek_char #43

merged 7 commits into from
May 22, 2021

Conversation

magicant
Copy link
Owner

@magicant magicant commented May 22, 2021

The existing lexer implementation requires that the Lexer::line_continuations function be called every time before peeking or skipping a character where line continuation is allowed. This is error-prone because it can only covered by tests if we forget to call the function.

This PR modifies the lexer to automatically skip line continuation before peeking so that we no longer need to call the function, except in a few contexts where line continuation should not be recognized, i.e., single quotes, literal here-document contents, and comments.


  • Extract LexerCore
  • Skip line continuation in Lexer::peek_char
  • Remove all calls to Lexer::line_continuations
  • Remove unnecessary tests about line continuation
  • Remove unnecessary mentions to line continuation in doc comments

@magicant magicant self-assigned this May 22, 2021
@magicant magicant added this to In progress in Main via automation May 22, 2021
@magicant magicant marked this pull request as ready for review May 22, 2021 14:46
@magicant magicant merged commit a78f66a into master May 22, 2021
Main automation moved this from In progress to Done May 22, 2021
@magicant magicant deleted the lexer_core branch May 22, 2021 14:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Main
  
Done
Development

Successfully merging this pull request may close these issues.

None yet

1 participant