This repository has been archived by the owner on Mar 3, 2023. It is now read-only.
Fix difference in CRLF line ending handling between tree-sitter and text-buffer #16738
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #16642
Fixes #16643
Tree-sitter allows tokens to end between CR and LF characters. For exampe, in JavaScript, line comment tokens always end at a LF character; if the file uses CRLF line endings, the comment token will include the the preceding CR character.
Text-buffer on the other hand, does not consider the position between a CR and and LF to be a valid position. It does not let you read or write changes between those characters.
To compensate for this difference in position semantics, I've had to add a special case to the
TreeSitterTextBufferInput
for the situation where the Tree-sitter parser decides toseek
to the position right between a CR and an LF.