Join GitHub today
GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.Sign up
WIP chore: drop twing for twig-lexer #17
When using Twing, we had some issues about whitespace control modifier (
But with twig-lexer everything is preserved as tokens, example:
And also use FastPath instead of our own print function because of (prettier/plugin-php#12):
Glad to see that this will help you. It will be the lexer used by the next major version of Twing.
Just a remark about the "WHITESPACE" tokens. They may seem redundant - after all they can easily be resolved by simple comparison of value and line/column properties of the tokens - but I felt they would make life easier for tools like yours - less code from your side is always better.
The drawback is that they have to be ignored by eventual parsers (like Twing parser) but this is trivial.
Another remark : I see that you implemented a token stream which is very good. I have been torned between having one included in twig-lexer or not. Actually, Twing will also need one so maybe it could be beneficial to have it included in twig-lexer and tested there instead of having it spread in multiple projects. It would come as a separate module that export a constructor that takes an array of tokens as parameter. With tree shaking, it would add zero byte to consumer packages if not used.
Don't worry for the
For the token stream, it would be really nice since the one I wrote is like 80% copy/pasted from Twing.