Lexer: keep the lines in lex #14
Assignees
Labels
enhancement
New feature or request
Comments
Merged
evenyag
pushed a commit
that referenced
this issue
Dec 21, 2022
* feat(lex): skeleton * refactor(lexer): use enum instead of trait object * lex: keyword_or_identifier * fix(ctx): pos, start, peek logic * refactor: fn -> Fn * refactor: move shift into Lexer * lexer: accept_string * lexer: duration, lable_selector * example: make example work with lexer * test: common test done * test(lex): more test cases * test(lex): unbalanced parentheses * test(lexer): subquery * test(lexer): more cases * test: token and production fn * misc(lex): modification based on review comments - don't expose some public methods - refactor test cases Reviewer: @evenyag Refs: #14 #15 * misc(lex): revision based on review comment - func in token mod, directly use match arms - use char default method, instead of a hashmap - use for loop to asset_eq!(left, right) instead of assert!(all) reviewer: @evenyag * misc(lex): use const instead of lazy_static * perf(token): use hashmap instead of match on &str * fix(lex): potential panic try to dec depth when depth is less than 1 * misc(lex): use usize instead of u8 for paren_depth * feat(lex): to check if backup is at the beginning of input * comment(lex): typo
waynexia
pushed a commit
that referenced
this issue
Feb 13, 2023
* feat(lex): skeleton * refactor(lexer): use enum instead of trait object * lex: keyword_or_identifier * fix(ctx): pos, start, peek logic * refactor: fn -> Fn * refactor: move shift into Lexer * lexer: accept_string * lexer: duration, lable_selector * example: make example work with lexer * test: common test done * test(lex): more test cases * test(lex): unbalanced parentheses * test(lexer): subquery * test(lexer): more cases * test: token and production fn * misc(lex): modification based on review comments - don't expose some public methods - refactor test cases Reviewer: @evenyag Refs: #14 #15 * misc(lex): revision based on review comment - func in token mod, directly use match arms - use char default method, instead of a hashmap - use for loop to asset_eq!(left, right) instead of assert!(all) reviewer: @evenyag * misc(lex): use const instead of lazy_static * perf(token): use hashmap instead of match on &str * fix(lex): potential panic try to dec depth when depth is less than 1 * misc(lex): use usize instead of u8 for paren_depth * feat(lex): to check if backup is at the beginning of input * comment(lex): typo
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
LRNonStreamingLexer needs to keep the lines information.
The text was updated successfully, but these errors were encountered: