use .gitignore to ignore search inside files and folders

[luisrudge]

use .gitignore to know which files and folders not to search into (think node_modules or packages)

[Tyriar]

I don't think .gitignore should dictate this, in other text editors it's normally done on a project setting or editor setting-level.

[luisrudge]

atom does this. there's no need for project settings or editor settings. If I'm git-ignoring it, there's no need to make this searchable or "autocompletable" (I just invented that word)

[Tyriar]

Huh it does too, I guess I'm thinking of sublime.

[ZombieProtectionAgency]

This is a feature request, not a bug report/issue. Feature suggestions already have a place over at vscodes uservoice.

[luisrudge]

Ops! Sorry. I didn't know there was a uservoice.

[egamma]

@luisrudge reopening. We haven't fully figured out what to discuss on user voice and what to discuss in a feature request issue.

This feature request is better discussed here. In particular since the last comment in user voice is not correct 😞 .

I could imagine a meta property 'USE_GITIGNORE' and that would be a great opportunity for a pull request:

    "files.exclude": {
        "USE_GITIGNORE": true
    },

@bpasero fyi.

[zwacky]

+1

[luisrudge]

I'm not sure I like the idea of having this as a opt-in flag.. This really should be the default behavior, since searching inside node_modules is not a good thing and certainly isn't the most common use case. Maybe we can add a flag to opt-out this behavior?

Also, I'd like to do this! Can you help me understand which parts of vscode would have to change? THanks!

[bpasero]

I think in the example above it should really be search.exlude and not files.exclude because the former will exclude it from search but the latter from everywhere in the UI including the explorer.

We are reading this setting in from https://github.com/Microsoft/vscode/blob/master/src/vs/workbench/parts/search/common/searchQuery.ts#L18 and I think we could add code there to check for another setting that - if enabled - would mix in anything from .gitignore.

[zwacky]

as @luisrudge already said, this should be the default behaviour like in atom and sublime.

[luisrudge]

@bpasero so.. just add code to read from .gitignore and push that files/folders to searchExcludes ?

[bpasero]

@luisrudge well we also have to convert it to the glob pattern format we use in VS Code. If you look at the default settings and how we ignore the .git folder there you can get an idea of the syntax.

[nchammas]

I agree with @luisrudge that this should be the default behavior, but perhaps there is an intermediate solution we can go for first:

Results from tracked files should come first. Results from git-ignored files should come last.

This is a lower-risk change that, IMO, can be implemented immediately and yield a notable user experience benefit for search.

Currently, results from git-ignored files are mixed in with results from tracked files. This is super noisy, for example, if you're developing in Python and have a virtual environment or build folder within your project folder with all kinds of code that matches almost anything you might search for.

If you're searching for a string across your project, I wager in 100% of cases you want results from your tracked files first, perhaps followed by results from untracked but not ignored files.

Would it make sense to implement this behavior first? Basically, just tweaking the order in which in the results are displayed. I don't think we would even need a new setting for that.

[nchammas]

I think the behavior I described should also apply to the behavior of Command + P, when you want to switch to a file and there are multiple possible matches for the user's search term.

[bpasero]

Currently the sorting is based on our scoring algorithm. I do not think it would make sense to change the sorting based on other criteria unless there is a good way to explain this to the user.

[zwacky]

the sorting can stay the same, just ignore the files listed in .gitignore. this is the absolute main thing that keeps me from using code.

[nchammas]

@bpasero - Could you elaborate? I think the change I'm describing is very easy to explain.

Here's take 2 at explaining the change, simplifying what I said earlier:

Search results are divided into 2 sections--tracked files and everything else. Results within each section are sorted according to the current scoring algorithm. The tracked files section always shows up first.

In addition to being easy to explain, this is the behavior I wager most people would expect.

As I commented earlier, Python projects generally have venv, dist, and build subfolders which contain many Python files, some of which are even copies of the main project files from the last packaging build. These are not part of the project and you don't check these files in because they are copied from third-parties or auto-generated.

Treating these files as equal to the main project files severely detracts from the usability of search. No one wants to see results from these files at the top, if even at all.

I'm sure other languages have similar problems as well.

If ignoring these files in search results is too big of a step to take, then I think the tweak I'm describing here yields the same usability benefit while being straightforward to explain.

[bpasero]

@nchammas I do think that the right fix is to just exclude those files when .gitignore is present and the setting to respect it. otherwise one could also argue why we are not doing the same sorting trick for the search/files exclude setting.

The real work here is to translate .gitignore patterns into our ignore patterns and making sure the solution scales. I have seen .gitignore files with 1000+ rules that would need to be applied many times over many paths.

[Tyriar]

Adding my 👍. This would solve one of the issues I encountered in the ember-cli test tour, saving devs from setting up a vscode-specific config file. Related PR: ember-cli/ember-cli#5643

[stubailo]

Hey after waiting for someone else to do it, I ended up putting together a small extension that copies your gitignore into your vscode workspace settings with one command: https://github.com/stubailo/vscode-ignore-gitignore

Let me know if this is useful, and looking forward to feedback and PRs! I've only tried it in a few projects but seems to work so far.

[egamma]

@paulcbetts regarding:

Cmd-T will really show you files in node_modules?

Cmd-T shows you all the symbols and not files in your workspace.

Regarding:

You now have a 50% chance of editing the file in out

The "out" folder includes the typescript source files in "out\trickline-win32-x64\resources\app\src" and the way the TypeScript language service is configured in the top-level "tsconfig.json" these files are part of the project context. They are therefore analyzed by TypeScript. To address this you should add an exclude of "out" to your "tsconfig.json" as shown below:

{
  "compilerOptions": {
    "removeComments": false,
    "preserveConstEnums": true,
    "declaration": true,
    "experimentalDecorators": true,
    "noImplicitAny": true,
    "noImplicitReturns": true,
    "suppressImplicitAnyIndexErrors": true,
    "strictNullChecks": true,
    "noUnusedLocals": true,
    "noImplicitThis": true,
    "noUnusedParameters": true,
    "module": "commonjs",
    "moduleResolution": "node",
    "pretty": true,
    "target": "es2015",
    "jsx": "react",
    "outDir": ".tmp/build"
  },
  "formatCodeOptions": {
    "indentSize": 2,
    "tabSize": 2
  },
  "types" : [
    "node",
    "electron"
  ],
  "exclude": [
    "out"
  ]
}

[luisrudge]

@egamma that's the whole point we're trying to get here... If we already have out in our .gitignore file, why would I have to add it to the tsconfig file again?

[bdbch]

Exactly, there should actually be more like a hierarchy for that.

.gitignore should always be the first ignore configuration to be loaded and should be merged with other vscode specific ignore configs.

[Stanzilla]

Thanks @stubailo, https://github.com/stubailo/vscode-ignore-gitignore is a decent workaround for now but this shouldl really be in core.

[dmitriid]

I think IntelliJ IDEA nailed this:

• .gitignored files and folders are grayed out by default in File explorer
• .gitignored files are automatically added to the projects "excluded" list
  • this means they do not participate in Search, Replace, Open File, Open Recent Files, Find symbols etc.
• however, if you actually spell the file name (or if the file cannot be found in the current non-excluded project), it will switch to looking across all files in the project, including the ignored ones.

Searching for a file, there are matching files in non-ignored files/folders:

mobx.js does not exist in the non-ignored files, switch to full-project search (ignored files are still grayed out)

[bdbch]

Are there actually any plans on ever solving this issue? Is this something way more complicated?
I tried to wrap my head around the src (never worked with ts before) and it seems that hooking into the right place would make it possible to fetch the .gitignore from the projects cwd and add all array elements to the files.exclude and search.exclude.

I couldn't find the right place where this happens, even though I found multiple files loading data from the settings.json.

Is there anyone from Microsoft clarifying the status of this issue and if it's actually wanted by Microsoft?

Anyway looking forward for further discussions here.

[roblourens]

Hoping to get it in for search in this release, via moving search to ripgrep: #19983

Once we have that, it might be possible to use ripgrep's highly optimized gitignore code to support it in the file explorer as well. The problems with putting the patterns in files.exclude is that the syntax is not exactly the same, and that it doesn't support negative patterns, and performance for large gitignore files could also be an issue.

[bdbch]

@roblourens sounds great and I understand the issues. I'm looking forward for the next release.

[roblourens]

Ripgrep search landed in today's Insiders, with .gitignore support enabled by default! Set "search.useRipgrep": true to try it out. There will likely be bugs... .gitignore support will be configurable later but I'm not sure how yet.

[bdbch]

@roblourens thats great. Going to check it out and see if I can spot bugs for now.

[stubailo]

@roblourens this is awesome, thank you!

[bpasero]

@roblourens should we close this one given ripgrep work landed (release notes)?

[therealkenc]

But not file explorer yet. Issues like #13535 have been dupped into this one.

[roblourens]

I wasn't sure since it has a lot of discussion about gitignore for the file explorer too. I think it makes sense to close this and start a new issue for that though.

[perlun]

The Ripgrep support is great, thank you for that! Can it be used for Cmd-P searching also? (It seems not, since I still get noise there if I don't configure the files.exclude setting to exclude them).

(Sorry to pollute this issue with a somewhat unrelated question, please point me in the right direction and I'll shut up. 😛)

[roblourens]

Not yet, I've got an issue for it - #24074

[jamesdixon]

@roblourens just curious: why was this closed? as far as I can tell it was never resolved.

[roblourens]

It was implemented for 1.11 - #78 (comment)

[jhm-ciberman]

@roblourens Why is this closed? It is solved for searchs yes, but it's not solved for explorer and there is not an option to exclude files in the explorer tree from gitignore. Is there any other issue for the explorer topic?

Edit: yes, you can use the ignore-gitignore extension, but I think that this should be integrated in the editor.

[roblourens]

Graying out gitignored files is implemented in 1.18, and hiding them can be tracked by #23804