Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parsing Error need to point to the code #4

Closed
6 tasks done
thi8v opened this issue May 14, 2023 · 4 comments · Fixed by #43
Closed
6 tasks done

Parsing Error need to point to the code #4

thi8v opened this issue May 14, 2023 · 4 comments · Fixed by #43
Assignees
Labels
A-Parser Area: Parser, in the zom_fe crate enhancement New feature or request
Milestone

Comments

@thi8v
Copy link
Member

thi8v commented May 14, 2023

Is your feature request related to a problem? Please describe.
Add error that clearly point to the code.

Describe the solution you'd like

  • rename the actual Token enum into TokenType
  • Create a struct Token that stores a TokenType and a range, named the span.
  • The lexer will still put the token type and add a span for every token, the start and the end.

After that the token stream has the location in the code.

  • Create a trait CodeLocation that will have a function span() that return the span of the AST.
  • All AST nodes will store it's own span, (and so implement CodeLocation)
  • When an error occurs, you can use the span of the most coherent AST node and spawn an error. -> Replace anyhow error with custom errors #30
@thi8v thi8v added the enhancement New feature or request label May 14, 2023
@thi8v
Copy link
Member Author

thi8v commented May 14, 2023

The new struct shouldn't pass a Position to the new struct but instead a much lighter one without the filename and filetext parameters. But instead put those parameters in the new struct.

@thi8v thi8v self-assigned this May 17, 2023
@thi8v
Copy link
Member Author

thi8v commented May 18, 2023

I'm definitely not happy with the result, I think I need to found a new implementation, it's too bulky. I've two ideas how to do it more efficiently :

  1. Create a new method (or function) to the lexer that retokenize the text and return the position of the token that as issue with. The prototype looks like that get_pos_token(&mut self, tok_idx: usize) -> Option<(Position, Position)> with the first Position be the start and the second, the end.
  2. Or a function, with a vec of tokens recreate the text an then, because it recreates the text, it knows where the token was and return a tuple of Position (like the first idea) with the text that it recreates and the filename doesn't change.
    Both function / method will only be call when there is an error.

@thi8v thi8v linked a pull request May 18, 2023 that will close this issue
@thi8v thi8v mentioned this issue May 18, 2023
@thi8v thi8v added this to the 0.1.0 milestone Jun 4, 2023
@thi8v thi8v added the A-Parser Area: Parser, in the zom_fe crate label Jun 4, 2023
@thi8v thi8v linked a pull request Jun 14, 2023 that will close this issue
@thi8v thi8v closed this as completed in #33 Jun 14, 2023
@thi8v thi8v reopened this Jun 14, 2023
@thi8v
Copy link
Member Author

thi8v commented Jun 21, 2023

This is done.

@thi8v thi8v closed this as completed Jun 21, 2023
@thi8v
Copy link
Member Author

thi8v commented Jul 10, 2023

The "reverse lexer" is just dumb, we can instead store a span in tokens, and in the AST

@thi8v thi8v reopened this Jul 10, 2023
@thi8v thi8v removed a link to a pull request Jul 10, 2023
@thi8v thi8v linked a pull request Jul 22, 2023 that will close this issue
@thi8v thi8v closed this as completed in #43 Jul 26, 2023
@thi8v thi8v mentioned this issue Aug 12, 2023
10 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-Parser Area: Parser, in the zom_fe crate enhancement New feature or request
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

1 participant