Fix incorrect mask in BufferChunks::next - #57544
Merged
Merged
Conversation
|
Thank you for your pull request and welcome to our community. We could not parse the GitHub identity of the following contributors: Galen Elias.
|
I noticed while working on a personal project which derives some logic from the core Zed stack that I was getting very unexpected results out of BufferChunks::next when walking through formatted chunks. The code calculates the mask for the `tabs`, `chars`, and `newlines` masks incorrectly, making the mask far too large when we are extracting chunks when chunk_start != 0. I believe the reason that this isn't a problem in Zed is that the InlayMap ends up re-masks all the bitmasks before passing things up the stack, so covers up the problem, preventing it from causing any damage. It appears this isn't caught in tests because this only happens when extracting chunks with formatting, during which we offset our retrieval by next_capture_start, and the current chunks tests don't use formatting. I think the fix is sound. The test is maybe not super ideal, but I wanted to demonstrate the issue in the pull request. If we process the following text by chunks: ```rust use std::cmp::Eq; ``` The first chunk is 'use ' with the chars mask being 0b1111. The second chunk is 'std' with the chars mask being 0b1111111 when it should just be 0b111.
galenelias
force-pushed
the
fixBufferChunksMask
branch
from
May 22, 2026 23:55
8ef93b9 to
ce10a1f
Compare
cla-bot
Bot
added
the
cla-signed
smitbarmase
added
the
area:editor
Veykril
enabled auto-merge
June 15, 2026 08:08
Contributor
This was referenced Jun 18, 2026
Closed
This was referenced Jul 1, 2026
This was referenced Jul 10, 2026
jolutz
pushed a commit
to jolutz/zed
that referenced
this pull request
Aug 8, 2026
I noticed while working on a personal project which derives some logic from the core Zed stack that I was getting very unexpected results out of BufferChunks::next when walking through formatted chunks. The code calculates the mask for the `tabs`, `chars`, and `newlines` masks incorrectly, making the mask far too large when we are extracting chunks when chunk_start != 0. I believe the reason that this isn't a problem in Zed is that the InlayMap ends up re-masks all the bitmasks before passing things up the stack, so covers up the problem, preventing it from causing any damage. It appears this isn't caught in tests because this only happens when extracting chunks with formatting, during which we offset our retrieval by next_capture_start, and the current chunks tests don't use formatting. I think the fix is sound. The test is maybe not super ideal, but I wanted to demonstrate the issue in the pull request. If we process the following text by chunks: ```rust use std::cmp::Eq; ``` The first chunk is 'use ' with the chars mask being 0b1111. The second chunk is 'std' with the chars mask being 0b1111111 when it should just be 0b111. Self-Review Checklist: - [x] I've reviewed my own diff for quality, security, and reliability - [x] Unsafe blocks (if any) have justifying comments - [x] The content is consistent with the [UI/UX checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist) - [x] Tests cover the new/changed behavior - [x] Performance impact has been considered and is acceptable Release Notes: - N/A Co-authored-by: Lukas Wirth <lukas@zed.dev>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I noticed while working on a personal project which derives some logic from the core Zed stack that I was getting very unexpected results out of BufferChunks::next when walking through formatted chunks.
The code calculates the mask for the
tabs,chars, andnewlinesmasks incorrectly, making the mask far too large when we are extracting chunks when chunk_start != 0.I believe the reason that this isn't a problem in Zed is that the InlayMap ends up re-masks all the bitmasks before passing things up the stack, so covers up the problem, preventing it from causing any damage. It appears this isn't caught in tests because this only happens when extracting chunks with formatting, during which we offset our retrieval by next_capture_start, and the current chunks tests don't use formatting.
I think the fix is sound. The test is maybe not super ideal, but I wanted to demonstrate the issue in the pull request.
If we process the following text by chunks:
The first chunk is 'use ' with the chars mask being 0b1111. The second chunk is 'std' with the chars mask being 0b1111111 when it should just be 0b111.
Self-Review Checklist:
Release Notes: