Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rewrite the parser to be fully incremental #68

Closed
wants to merge 74 commits into from
Closed

Commits on Jan 3, 2014

Commits on Apr 21, 2014

Commits on Jul 21, 2014

  1. The semicolon token should not be respected.

    See: http://dev.w3.org/csswg/css-syntax/#consume-a-declaration0
    After consuming the colon token, it states the declaration's value is the consumption of every subsequent component value, until we reach an EOF token.
    ezequiel committed Jul 21, 2014

Commits on Aug 16, 2014

  1. Merge pull request #3 from ezequiel/ident

    Identifiers may now begin with "--".
    SimonSapin committed Aug 16, 2014
  2. Merge pull request #4 from ezequiel/semicolon

    The semicolon token should not be respected.
    SimonSapin committed Aug 16, 2014

Commits on Oct 8, 2014

  1. Merge pull request #5 from rgrove/url-functions

    Quoted URLs are now tokenized as functions
    SimonSapin committed Oct 8, 2014

Commits on Dec 28, 2014

  1. Rewrite ALL THE THINGS!

    The tokenizer is now fully incremental, rather than yielding a block/function at a time.
    
    The convention is now to take `&mut Parser` as input for parsing, and only consume as necessary.
    SimonSapin committed Dec 28, 2014

Commits on Dec 29, 2014

  1. Use `while let` instead of `loop` + `match` + `break`.

    SimonSapin committed Dec 29, 2014
  2. Remove the Tokenizer::length field.

    Let’s assume that str::len() calls are optimized away.
    SimonSapin committed Dec 29, 2014
  3. Encapsulate input/position access in tokenizer.

    This would help taking input incrementally (rather than having a single `&str`) if we want to do that.
    SimonSapin committed Dec 29, 2014

Commits on Jan 5, 2015

  1. Merge branch 'rustup_20141221' into incremental

    Conflicts:
    	Cargo.toml
    	src/color.rs
    	src/parser.rs
    	src/serializer.rs
    	src/tests.rs
    	src/tokenizer.rs
    SimonSapin committed Jan 5, 2015
  2. Fix Rust-upgrade warnings.

    SimonSapin committed Jan 5, 2015
  3. Add and use a match_ignore_ascii_case! macro.

    SimonSapin committed Jan 5, 2015
  4. Fix incorrect doc-comment.

    SimonSapin committed Jan 5, 2015
  5. Change all `String` fields in `Token` to `CowString`.

    (Most values are still `Owned`, so we don’t take much advantage this yet.)
    SimonSapin committed Jan 5, 2015

Commits on Jan 7, 2015

  1. Add DeclarationListParser::run and RuleListParser::run.

    SimonSapin committed Jan 7, 2015

Commits on Jan 8, 2015

  1. Add Parser::expect_{ident_matching,delim,function}

    SimonSapin committed Jan 8, 2015

Commits on Jan 9, 2015

  1. Rename AtRulePrelude to AtRuleType

    SimonSapin committed Jan 9, 2015

Commits on Jan 13, 2015

  1. Refactor parser.rs to simplify it and propably fix multiple bugs.

    "Nested" or "delimited" parsers can now only be used in a closure given
    to the corresponding `Parser` method. This allows doing cleanup after
    calling the closure, rather than in a `Drop` implementation.
    
    This more local reasoning makes thing easier to understand
    and less likely to be buggy.
    SimonSapin committed Jan 13, 2015
  2. Implement Clone for Parser.

    A cloned parser is "detatched" from its parent and will progress independently,
    but it keeps referencing the same input.
    SimonSapin committed Jan 13, 2015
  3. Make source location methods take &self instead of &mut self. (Use a …

    …std::cell::Cell.)
    SimonSapin committed Jan 13, 2015

Commits on Jan 14, 2015

Commits on Jan 15, 2015

  1. Fix some parsing bugs.

    SimonSapin committed Jan 15, 2015
  2. Remove ast.rs file left over from 41fa928.

    Its content is now merged into tokenizer.rs.
    SimonSapin committed Jan 15, 2015

Commits on Jan 16, 2015

  1. Internal refactor: have "nested" parsers not consume their end token.

    This simplifies the code by having less special cases.
    SimonSapin committed Jan 16, 2015
  2. Internal refactor: remove the Parser::exhausted boolean flag.

    It wasn’t really necessary, and confusigly had a different meaning than Parser::is_exhausted().
    SimonSapin committed Jan 16, 2015
  3. Remove peek/push_back/unexpect, add Parser::reset instead.

    The new convention is that any parsing function returning `Err(())`
    may or may not have consumed any number of tokens from the input.
    
    Therefore, any the caller wishes to recover and try parsing something else,
    it should call `parser.reset(position)` where `position` is the result of
    `parser.position()` that was saved beforehand.
    
    The Parser::try helper does this for you by taking a closure returns Result,
    and resetting the position if the result is Err.
    SimonSapin committed Jan 16, 2015

Commits on Jan 20, 2015

  1. Keep in tokens the origin content of whitespace and comments.

    It’s cheap, and could be useful for tools such as pre-processors.
    SimonSapin committed Jan 20, 2015
You can’t perform that action at this time.