Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rewrite the lexer. #264

Merged
merged 1 commit into from Nov 9, 2018

Conversation

@dvander
Copy link
Member

dvander commented Nov 6, 2018

lex() is pretty unreadable, it has duplicated code and weird state transitions, and it's too long. This patch breaks it up into separate methods that each perform a clear task.

The operator lookup code is now inlined into a switch statement (similar to how the v2 lexer works), removing a loop+strcmp over a sizable chunk of the token table. Compiling SourceMod's plugins looks about 5% faster with this change.

After this change, it will be much easier to implement a token map and remove the sc_token loop.

lex() is pretty unreadable, it has duplicated code and weird state
transitions, and it's too long. This patch breaks it up into separate
methods that each perform a clear task.

The operator lookup code is now inlined into a switch statement (similar
to how the v2 lexer works), removing a loop+strcmp over a sizable chunk
of the token table. Compiling SourceMod's plugins looks about 5% faster
with this change.

After this change, it will be much easier to implement a token map and
remove the sc_token loop.
@dvander dvander merged commit 6d4c561 into master Nov 9, 2018
2 checks passed
2 checks passed
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@dvander dvander deleted the rewrite-lex branch Nov 9, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
1 participant
You can’t perform that action at this time.