Add command-backed path stream walkers#705
Open
tobim wants to merge 7 commits into
Open
Conversation
Document that treefmt does not support paths containing newlines before introducing command-backed path stream walkers.
Introduce a shared reader for walker commands that emit paths on stdout. Parse newline- and NUL-delimited records, normalize emitted paths inside the tree root, post-filter requested path filters, and convert existing regular files into walk.File values. Own the stdout and stderr pipes instead of using Cmd.StdoutPipe so the command can be waited for while Read drains stdout. Close cancels the command to unblock producers when callers stop before EOF, and treats post-cancel signal exits as expected.
Build the uncached reader in a helper and apply the cache wrapper once in NewReader. Keep auto fallback inside uncached construction so trying built-in walkers does not recurse through cache setup.
Store filesystem path filters as a slice and walk them sequentially in the filesystem reader. Use one filesystem reader for multiple requested paths instead of building one reader per path. Add coverage for multi-path filesystem walking.
Run git ls-files from the tree root and parse its NUL-delimited output with the shared path stream reader. Pass requested paths to git as path filters after -- so subtree formatting can prune the git output inside git.
Run jj file list from the tree root and parse its output with the shared path stream reader. Pass requested paths to jj after -- so paths that start with dashes are handled as path filters. Remove the generic CompositeReader now that all built-in walkers consume the same validated path-filter list.
tobim
commented
Jun 7, 2026
| return NewPathStreamReader(root, statz, PathStreamConfig{ | ||
| Name: "git", | ||
| Command: "git", | ||
| Options: []string{"ls-files", "-z", "--cached", "--others", "--exclude-standard", "--full-name", "--"}, |
Author
There was a problem hiding this comment.
Removing --stage here should not pose a problem.
The original reason to pass that option was so submodule paths are omitted, but the PathStreamReader filters out directories by itself. This also speeds up the execution time of git ls-files quite a bit.
Author
There was a problem hiding this comment.
I would also like to remove --others, because those files can't be recovered when they get formatted but the user didn't want that. Imho the "git" walker should not list untracked files.
I didn't do it tho because that would make this PR a behavioral change.
When no explicit path filters are passed, run `jj file list` with `.` as the positional filter. In non-colocated jj workspaces, running from a subdirectory can otherwise emit paths outside the treefmt tree root. Add a regression test for the jj-only workspace shape from numtide#704.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds a command-backed path stream reader interface for walkers that emit paths on stdout. The reader accepts newline- or NUL-delimited records, normalizes them relative to the tree root, filters requested path prefixes, and only returns existing regular files to the formatter pipeline.
The Git and Jujutsu walkers now use this shared reader. Requested file and directory paths are passed to their commands as positional filters, which lets the underlying walker prune output for subtree formatting.
The Jujutsu walker now also passes
.when no explicit path filter is provided. This keepsjj file listscoped to treefmt's resolved tree root in non-colocated Jujutsu workspaces.This also lets the filesystem walker handle multiple path filters directly and documents that paths containing newlines are unsupported.
Fixes #704.