-
Notifications
You must be signed in to change notification settings - Fork 57
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Replace Slime.Parser with parser generated from PEG (using neotoma) #113
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Thank you for this @Rakoth, this is awesome! Sorry for the delay getting back to you, I've been swapped with work and family things. Could we sync up on Slack and you give me a rough run down of things? This is a new approach for me. |
@Rakoth once the merge conflicts are resolved is this safe to merge? Will we retain the same level of functionality we currently have? |
This was referenced May 4, 2017
This introduces new compile-time dependency: neotoma parser
Also with this commit slime preprocessor will raise an error on inconsistent indentation as ruby slim does.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is experimental implementation of a parser using PEG
First thing to look at is grammar src/slime_parser.peg
Then take a look at new preprocessor lib/slime/parser/preprocessor.ex which main goal is to add indents/dedents where needed (I get the ideas from here)
Finally take a look at transform callback module lib/slime/parser/transform.ex This is where parsed AST gets transformed into useful one. Its current goal is to produce tokens exactly as our current parser do. But it can be easily modified to produce nested structures as
Slime.Tree
do.Things to discuss:
The bummer is that this parser is ~10x slower than our current parser. But I believe grammar might be optimized to reduce the performance gap. See benchmark bench/example_parse_bench.exs.
There is currently a pending test case about this issue. The possible solution is to make preprocessor smart enough to skip embedded engine lines and do not add indents/dedents.
:attr_list_delims
work again.There is penging test case
The possible solutions are:
transform(:wrapped_attributes, ...)
based on config options chose ether transformed attributes or the original input.(When there is a space separating tag head and wrapped attributes list). Pending test. While it is currently supported and is supported by original slim, I found it hard to manage using PEG, such attributes now parsed as inline text after wrapped attributes check is failed.
Things to do next:
Closes #106 #32 #123
@doomspork, @henrik, @lpil, @bozydar I need your thoughts on this, guys.