[bug] Tokenizer line/char is not always accurate #10
Labels
A-Compiler
Dealing with the compiler (analyzer, tokenizer, parser, transpiler)
P-Low
T-Bug
Something isn't working
Describe the bug
Sometimes the line / char is inaccurate when parsing something with the tokenizer. This is very annoying with error traces.
Note this is a shortcoming from using lua patterns as the tokenizer, which gives less control as to what happens for more performance. There is probably an elegant solution to this that maintains the pattern matching tokenizer, but it is definitely a lot more complex than if it looped over all chars.
To Reproduce
Nothing right now
Expected behaviour
Get proper line/char of the token.
The text was updated successfully, but these errors were encountered: