Merged
Conversation
This commit removes the dependency on `pest` and `pest-ast`. A custom lexer is now implemented using `logos`; this allows for easy parsing of doc comments at any point in the token stream. The blackbox of `pest-ast` has been removed in favor of implementing a `Parse` and `Peek` trait on the AST types. AST types no longer have to mirror the pest grammar, so they have been simplified. Top-level resource declarations have been removed from the grammar as they must appear inside of interface and world declarations. `bail!` has been removed in parsing and resolution in favor of custom `thiserror`-derived error types. Error reporting has been improved with more colors and removing redundant information. No new tests were added; parsing test baselines have substantial changes as a result of the AST simplification. Resolution failure test baselines have changed as a result of the error formatting changes, but the error messages themselves should remain the same.
Member
Author
|
While this is a substantial change, a large chunk of the changes come from updated test baselines. |
Just use an array of `Token` now that tokens are only a single byte (they used to be much larger before a previous refactoring). Additionally, add test coverage for multiple expected tokens errors. Fix a bug where a failed lookahead used the current token position instead of the last position in the source where the next token is EOI.
Collaborator
fibonacci1729
left a comment
fibonacci1729
left a comment
There was a problem hiding this comment.
Nice, this is much easier to understand than pest!!
fibonacci1729
approved these changes
Nov 7, 2023
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR removes the dependency on
pestandpest-ast.A custom lexer is now implemented using
logos; this allows for easy parsing of doc comments at any point in the token stream.The blackbox of
pest-asthas been removed in favor of implementing aParseandPeektrait on the AST types. AST types no longer have to mirror the pest grammar, so they have been simplified.Top-level resource declarations have been removed from the grammar as they must appear inside of interface and world declarations.
bail!has been removed in parsing and resolution in favor of customthiserror-derived error types.Error reporting has been improved with more colors and removing redundant information.
No new tests were added; parsing test baselines have substantial changes as a result of the AST simplification.
Resolution failure test baselines have changed as a result of the error formatting changes, but the error messages themselves should remain the same.