Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Lexical modes #14

Open
oxc opened this issue Aug 21, 2018 · 0 comments
Open

[Feature] Lexical modes #14

oxc opened this issue Aug 21, 2018 · 0 comments

Comments

@oxc
Copy link

oxc commented Aug 21, 2018

It would be nice if the library supported lexical modes (at least that's what ANTLR4 calls them: https://github.com/antlr/antlr4/blob/master/doc/lexer-rules.md#lexical-modes), where the lexer keeps a stack of modes and produces different tokens based on the current mode.

For example, I have a small mini language that basically specifies boolean expressions. Now I would like to use these expressions inside a larger document language, where they can occur only in certain positions in the header of the document (after a "@require" token until the end of the line). It would easily be possible to reference the root parser of the mini-language inside the document-grammar, but since the tokens are not compatible, this fails.
I can workaround this by specifying a token that matches the whole line, and parse this separately in a second step. This, of course, is very hacky, and does not work in all situations.

It would be much nicer if I could switch the lexer to the mini-language mode when encountering a @require token, and then popping back to the default mode when encountering the EOL token.
Ideally, I could "import" the mode from another Grammar. Perhaps it might even make sense to make this the preferred way to specify different modes.

Let me know if I should create a minimal example.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant