[diffs/edit] avoid full retokenization on newline inserts - #1057
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ed2fcc4db3
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| const canReuseShiftedStates = | ||
| hostRealignsRows && !canReuseCachedStates && dirtyStart >= startingLine; |
There was a problem hiding this comment.
When one applied change both inserts and deletes lines so the deltas cancel, change.lineDelta is 0 but canReuseCachedStates is false. This branch now lets non-diff editors settle after the first changed range and jump to the next one, omitting the shifted lines between ranges from dirtyLines; #rerender only runs applyDocumentChange/row realignment when the net line count changes, so those intervening rendered rows and cache entries keep old text after a net-zero multi-edit such as inserting a newline above line 10 and deleting one above line 20 in the same applyEdits batch.
Useful? React with 👍 / 👎.
2 participants
Fixes high memory usage when inserting newlines by reusing shifted syntax states after tokenization reconverges.
The testing shows reducing the Enter-key path from 100 dirty lines and 6.9 ms of tokenization to 2 dirty lines and 0.5 ms.