Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove support for 1-token lookahead from the lexer #62329

Merged
merged 9 commits into from
Jul 6, 2019
Merged

Remove support for 1-token lookahead from the lexer #62329

merged 9 commits into from
Jul 6, 2019

Commits on Jul 4, 2019

  1. remove StringReader::peek 

    The reader itself doesn't need ability to peek tokens, so it's better
if clients implement this functionality.

This hopefully becomes especially easy once we use iterator interface
for lexer, but this is not too easy at the moment, because of buffered
errors.
    @matklad
    matklad committed Jul 4, 2019
    Configuration menu
    Copy the full SHA
    830ff4a View commit details
    Browse the repository at this point in the history

  2. remove peek_token from StringReader

    @matklad
    matklad committed Jul 4, 2019
    Configuration menu
    Copy the full SHA
    e9dc95c View commit details
    Browse the repository at this point in the history

  3. remove peek_span_src_raw from StringReader

    @matklad
    matklad committed Jul 4, 2019
    Configuration menu
    Copy the full SHA
    256df83 View commit details
    Browse the repository at this point in the history

  4. cleanup lexer constructors

    @matklad
    matklad committed Jul 4, 2019
    Configuration menu
    Copy the full SHA
    601bad8 View commit details
    Browse the repository at this point in the history

  5. move constructors to top

    @matklad
    matklad committed Jul 4, 2019
    Configuration menu
    Copy the full SHA
    30fa99e View commit details
    Browse the repository at this point in the history

  6. slightly comment lexer API

    @matklad
    matklad committed Jul 4, 2019
    Configuration menu
    Copy the full SHA
    1c6eb19 View commit details
    Browse the repository at this point in the history

  7. don't rely on spans when checking tokens for jointness

    @matklad
    matklad committed Jul 4, 2019
    Configuration menu
    Copy the full SHA
    8bea334 View commit details
    Browse the repository at this point in the history

  8. remove unused mk_sp_and_raw

    @matklad
    matklad committed Jul 4, 2019
    Configuration menu
    Copy the full SHA
    3035a05 View commit details
    Browse the repository at this point in the history

  9. make unwrap_or_abort non-generic again

    @matklad
    matklad committed Jul 4, 2019
    Configuration menu
    Copy the full SHA
    3e362a4 View commit details
    Browse the repository at this point in the history