Skip to content

Rewrite the lexer.#264

Merged
dvander merged 1 commit intomasterfrom
rewrite-lex
Nov 9, 2018
Merged

Rewrite the lexer.#264
dvander merged 1 commit intomasterfrom
rewrite-lex

Conversation

@dvander
Copy link
Copy Markdown
Member

@dvander dvander commented Nov 6, 2018

lex() is pretty unreadable, it has duplicated code and weird state transitions, and it's too long. This patch breaks it up into separate methods that each perform a clear task.

The operator lookup code is now inlined into a switch statement (similar to how the v2 lexer works), removing a loop+strcmp over a sizable chunk of the token table. Compiling SourceMod's plugins looks about 5% faster with this change.

After this change, it will be much easier to implement a token map and remove the sc_token loop.

lex() is pretty unreadable, it has duplicated code and weird state
transitions, and it's too long. This patch breaks it up into separate
methods that each perform a clear task.

The operator lookup code is now inlined into a switch statement (similar
to how the v2 lexer works), removing a loop+strcmp over a sizable chunk
of the token table. Compiling SourceMod's plugins looks about 5% faster
with this change.

After this change, it will be much easier to implement a token map and
remove the sc_token loop.
@dvander dvander merged commit 6d4c561 into master Nov 9, 2018
@dvander dvander deleted the rewrite-lex branch November 9, 2018 05:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant