Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a new ParserExtra::State: Inspector trait #681

Merged
merged 1 commit into from
Oct 21, 2024

Conversation

jyn514
Copy link
Contributor

@jyn514 jyn514 commented Oct 18, 2024

This allows using imperative concrete syntax tree parsers like rowan and cstree. In particular those libraries want to know every token that is parsed, and need to know when chumsky backtracks and reparses the same tokens again.

This adds the following new API surface:

pub trait Inspector<'a, I: Input<'a>>: Default {
    type SaveMarker: Copy + Clone;
    fn on_token(&mut self, token: &I::Token);
    fn on_save<'parse>(&self, offset: I::Offset) -> Self::SaveMarker;
    fn on_rewind<'parse>(&mut self, marker: Marker<'a, 'parse, I, Self::SaveMarker>);
}
pub struct SimpleState<T>(pub T);
impl<'a, T, I: Input<'a>> Inspector<'a, I> for SimpleState<T>;
impl<T> DerefMut<Target = T> for SimpleState<T>;
impl<T> From<T> for SimpleState<T>;

and additionally now requires ParserExtra::State: Inspector.

i didn't write any of my own tests yet but my own parser that hooks chumsky up to cstree does work: spreadsheet-lang/spreadsheet@1b97313#diff-14962dfdd56397d99e07041d6337cb5b0327f4d7433d80fd3383955415014910R111

Fixes #96

@jyn514 jyn514 force-pushed the recorder branch 2 times, most recently from a91768f to 8baf215 Compare October 19, 2024 03:41
@jyn514 jyn514 changed the title wip: Add [ParserExtra::State::Recorder] Add a new ParserExtra::State: Recorder trait Oct 19, 2024
@jyn514 jyn514 marked this pull request as ready for review October 19, 2024 03:42
src/recorder.rs Outdated Show resolved Hide resolved
src/recorder.rs Outdated Show resolved Hide resolved
src/recorder.rs Outdated Show resolved Hide resolved
src/recorder.rs Outdated Show resolved Hide resolved
@jyn514 jyn514 changed the title Add a new ParserExtra::State: Recorder trait Add a new ParserExtra::State: Inspector trait Oct 19, 2024
@zesterer
Copy link
Owner

Looks like there are a few minor CI failures

This allows using imperative concrete syntax tree parsers like `rowan` and `cstree`.
In particular those libraries want to know every token that is parsed, and need to know when chumsky backtracks and reparses the same tokens again.

This adds the following new API surface:
```rust
pub trait Inspector<'a, I: Input<'a>>: Default {
    type SaveMarker: Copy + Clone;
    fn on_token(&mut self, token: &I::Token);
    fn on_save<'parse>(&self, offset: I::Offset) -> Self::SaveMarker;
    fn on_rewind<'parse>(&mut self, marker: Marker<'a, 'parse, I, Self::SaveMarker>);
}
pub struct SimpleState<T>(pub T);
impl<'a, T, I: Input<'a>> Inspector<'a, I> for SimpleState<T>;
impl<T> DerefMut<Target = T> for SimpleState<T>;
impl<T> From<T> for SimpleState<T>;
```
and additionally now requires `ParserExtra::State: Inspector`.
@zesterer zesterer merged commit d611f42 into zesterer:main Oct 21, 2024
3 of 4 checks passed
@zesterer
Copy link
Owner

Thanks so much for your work on this, it's very appreciated!

@jyn514 jyn514 deleted the recorder branch October 21, 2024 12:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support integration with cstree
2 participants