New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Token parsing #93

Open
ptal opened this Issue Aug 2, 2016 · 0 comments

Comments

Projects
None yet
1 participant
@ptal
Owner

ptal commented Aug 2, 2016

Initially, #80 was opened to be able to directly parsed on the rust token stream and, taking advantages of techniques already used in OMeta, being able to parse on arbitrary tree structure. However we are in a typed context and it seems more difficult than what I first thought. Similarly to #92, we propose to use the string literal to parse on enum's variants.

Design

  • Provide a stream structure encapsulating the Rust parser, it will also be useful to interact with external parser.
  • Implement the trait ConsumePrefix such that it consumes one or more tokens depending on the prefix.
  • A literal, for example "=" will correspond to the token Eq. If the literal cannot be mapped, then an assert will be triggered at runtime. General parsing on tree proposed by #80 would avoid this runtime error, however, for now, we believe it's easy enough to find and fix.
  • Tokens containing data must be parsed with an external parser (Ident(..), Literal(..), Lifetime(..),....) if you want to extract the contained data. Otherwise, the token data will be compared against the string literal. Note that Ident("a_variable") does not match "a_var" "iable" but only "a_variable".

@ptal ptal added the runtime-lib label Aug 2, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment