Unfeeding Tokens #895
-
I am using lark-parser for a project I am working on. While parsing the input text, if an unexpected token is received, the token needs to be fixed with heuristic methods (that is, the token is converted to another token and/or additional tokens are feeded). To accomplish this, I coded custom lexer. It works really good. To carry my work to one upper level, I need to unfeed tokens. But I couldn't find any functions in api to accomplish this. Is unfeeding tokens possible? Example: One way is to extend the language to parse ambiguous inputs. But I don't want to use this method because the language is already very complex. |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 5 replies
-
No, this is not really possible, primarily because it is not always unambiguous what exactly that means, and sometimes there is even a loss of information. You can instead copy the state after every successfully fed token and then manually roll back. (using the new Interactive Parser interface) |
Beta Was this translation helpful? Give feedback.
No, this is not really possible, primarily because it is not always unambiguous what exactly that means, and sometimes there is even a loss of information. You can instead copy the state after every successfully fed token and then manually roll back. (using the new Interactive Parser interface)