refactor(tokenizer): Re-use object for current location #397
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Previously, `ctLoc` was overwritten on every new character. This change should lead to a great reduction in allocations when using location information.
wooorm
reviewed
Feb 9, 2022
| @@ -925,7 +953,7 @@ export class Tokenizer { | |||
| //------------------------------------------------------------------ | |||
| private _stateData(cp: number): void { | |||
| this.preprocessor.dropParsedChunk(); | |||
| this.ctLoc = this._getCurrentLocation(); | |||
| this.updateStartLocation(); | |||
There was a problem hiding this comment.
feels a bit weird having to do this book-keeping — its existence seems to indicate that the data might be out of date and before you access it you’d have to call this function?
There was a problem hiding this comment.
I played around with this some more. Turns out we can actually get rid of location updates in the *DATA states altogether: #402
fb55
added a commit
to parse5/parse5-fork
that referenced
this pull request
Feb 11, 2022
Supersedes inikulin#397 Previously, `ctLoc` was overwritten on every new character. inikulin#397 attempted to improve this by maintaining the locations in an updated object, and creating copies when needed. With this PR, we instead use explicit offsets when creating tokens. There is no more need to update locations; instead, we exploit the fact that a *DATA state is entered only after emitting the previous token. This change should lead to a great reduction in allocations when using location information.
|
Closing in favour of #402 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Previously,
ctLocwas overwritten on every new character. This change should lead to a great reduction in allocations when using location information.