Remove Tokens from AST #403
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
These are now only used for showing Keyword nodes, so it gets replaced with showKw(). There's also no need to use isReserved() anymore, as all keyword nodes only contain data from reserved token types.
Without this, many of these type names are too generic (like Parameter) and can't be intuitively identified as being part of AST data structure.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Refactored AST data structure so we no more have to deal with raw tokens inside it.
ClauseNode.namecould previously contain any type of Token, but now it can only contain a KeywordNode.The TokenNode is now eliminated and replaced with IdentifierNode, LiteralNode, ...
There's still a some token-data left inside the new KeywordNode, which has a
tokenTypefield. This will hopefully get eliminated in the process of extending the parser.The old
show()method inside ExpressionFormatter is now replaced withshowKw(), which only deals with showing KeywordNodes.It's quite a big change in the amount of code affected, but functionality-wise nothing has changed.