Skip to content

workbench.action.findInFiles: accept file URIs to bypass argv-length limits #319112

Description

@FreHu

Problem

Extensions that want to scope a search to a programmatically-built file set
can only pass filesToInclude: string to workbench.action.findInFiles /
search.action.openNewEditor. The string is parsed back into globs,
expanded by spreadGlobComponents, and emitted as -g args to ripgrep.

On Windows the 8191-character argv limit is hit quickly. With the ~3x
expansion factor from progressive-path globs (srcsrc/xsrc/x/y.ts),
a few hundred files is enough to overflow. Extensions are forced to cap the include pattern length (I use ~4000 chars).

I've gone to some length in devising a workaround and best I can do is batch the file list and open multiple search editors for one logical search — fragmenting results.

Use cases

My extension Fresh File Explorer (https://github.com/FreHu/vscode-fresh-file-explorer) has two:

  • "Search in fresh files" (meaning pending + files last touched in last x days based on git history)
  • "Search in found files" (chain a search scoped to the files found in a previous search)

Existing internal solution

I noticed in the release notes that you added a toggle to search in modified files only, and had Claude look at how that works.

The built-in "Search in Modified Files" toggle builds changedFileUris: URI[] from the SCM
service and hands it to QueryBuilder.commonQueryFromFileList, which:

  • Groups files by workspace root → one folderQuery per root
  • Emits per-folder includePattern as a glob object
    ({ "rel/path": true }) — no brace string, no spreadGlobComponents
  • Sets excludePattern: { '**/*': true }

This gives ripgrep one -g per file and a fresh argv budget
per workspace folder. The argv limit effectively stops being a problem at
realistic file counts.

References:

  • src/vs/workbench/contrib/search/browser/searchView.ts ~L1654 (URI collection)
  • src/vs/workbench/services/search/common/queryBuilder.ts commonQueryFromFileList

Request

Extend IFindInFilesArgs (and the search.action.openNewEditor args) to
accept either:

  • filesToIncludeUris?: URI[] — same path the SCM toggle uses, or
  • filesToInclude: string | URI[] — overload the existing field

Either should route through commonQueryFromFileList instead of the
string-parsing path.

Workaround (current)

I cap the generated filesToInclude string at ~4000 chars (well under
8191 to leave room for the spreadGlobComponents 3x expansion) and, when
the file set doesn't fit, split it into batches that each open in their
own search editor tab.

Concretely, for a single user action ("search in my N fresh files") my
extension may open 3, 5, sometimes 10+ search editor tabs. This breaks
the search UX in ways that are hard to paper over:

1. The search viewlet is unreachable once batching kicks in.
The viewlet shows exactly one search. The moment I need >1 batch I'm
forced into search editors, even for users who configured "open in
viewlet". Their config silently loses.

2. Results are fragmented across tabs with no unified view.
The user wants one result list. They get N. No combined match count, no
single "next match" navigation, no cross-batch sorting by file. To see
all hits for foo they alt-tab between editors and mentally merge.

3. Refinement multiplies the pain.
Tweaking the query (toggle case sensitivity, change regex, narrow the
pattern) means re-running it in every tab, one at a time. There is no
"re-run across all batches" affordance and I can't build one — each
search editor is an independent VS Code document.

4. Replace-all is effectively broken.
A replace operation on batch 1 has no idea batches 2..N exist. The user
either replaces per-tab (error-prone, easy to miss files) or gives up
and uses something else.

5. Files with very long paths get silently dropped.
If a single file's relative path exceeds the per-batch budget on its own
— possible in deeply nested monorepos or under long workspace prefixes —
I log a warning and skip it. The user sees "N files searched" but
doesn't know which were excluded unless they read my output channel.

6. The cap is conservatively wrong on multi-root workspaces.
The internal toggle splits argv budget per workspace folder. I can't,
because I go through the single filesToInclude string. A 3-root
workspace with 1500 files would fit comfortably through the URI[] path
(500 per ripgrep invocation, three invocations) but blows my single
~4000-char string and triggers batching.

7. I can't point users at the built-in toggle as a substitute.
The toggle only covers working-tree-dirty files. My use case ("files
changed in the last N days, including committed history") is a
superset. The overlap is partial, so "just use the SCM toggle" is wrong
advice for most of my users.

8. The 3x post-expansion tax forces a conservative budget.
I emit a single comma+brace-optimized include string
(src/{a.ts,b.ts},lib/c.ts). VS Code's
performBraceExpansionForRipgrep un-braces it, then
spreadGlobComponents progressively splits each path into every
prefix: src/components/Button.tsx becomes
-g src -g src/components -g src/components/Button.tsx. A path of
depth N produces N -g args.

The argv limit is on the post-expansion ripgrep invocation, not my
input string. So I have to budget my single string against the
explosion that happens after it crosses the API boundary — which is why
my cap is ~4000 chars rather than something closer to 8191. The URI[]
path skips spreadGlobComponents entirely (it knows the entries are
exact file paths, not globs to be anchored), so one file → one -g
arg. Same files, ~3x less argv pressure, no guessing on my side about
what the expansion will produce.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions