Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rewrite lexer #12

Closed
maanlamp opened this issue Jan 10, 2019 · 3 comments
Closed

Rewrite lexer #12

maanlamp opened this issue Jan 10, 2019 · 3 comments
Assignees
Labels
bug Something isn't working

Comments

@maanlamp
Copy link
Member

The lexer works but I cannot remember what anything does. That's bad. Please rewrite your horrible mess you dummy.

@maanlamp maanlamp added Lexer Cleanup This requires cleaning labels Jan 10, 2019
@maanlamp maanlamp self-assigned this Jan 10, 2019
@maanlamp
Copy link
Member Author

Keep in mind issue #4 when doing this!

@maanlamp
Copy link
Member Author

maanlamp commented Jan 18, 2019

Done.

Rewrote large blocks of unintelligible expressions and statements into functions with proper names.
Also as per #4 , I've learnt how to use streams and utilised them for this rewrite.

See the modified Lexer.js in maanlamp@801c438e65cf0d0eab0f8d028d7e045e286a32fe.

There's a problem now though, when a grammar has limitless quantifiers, it will register false negative matches, resulting in errors in places where it should not error.

this.limitlessQuantifiers = /[*+]|.\{\d,\}/;

I truly am not experienced enough in using streams to know a solution to this. I'm leaving the issue open and attaching new labels.

@maanlamp maanlamp added bug Something isn't working and removed Cleanup This requires cleaning labels Jan 18, 2019
@maanlamp
Copy link
Member Author

Lexer is now v1.0 5a74377

No streams are implemented in the official release because of severly faulty lexeme recognition errors.
Lexer will be decoupled into a separate repository later on.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant