feature request: prevent Spotlight from indexing node_modules folders

[PierBover]

What's the feature?

On macOS NPM could optionally add a .metadata_never_index file in the node_modules folder to prevent Spotlight from indexing those folders.

What problem is the feature intended to solve?

While indexing those folders with so many files Spotlight consumes valuable CPU resources. A simple way of preventing this in an automated way is by automatically adding a .metadata_never_index file in the node_modules folder.

It's common practice that most users put the node_modules folder in .gitignore so this would not be a problem for Windows or Linux users.

Is the absence of this feature blocking you or your team? If so, how?

In some systems with large projects Spotlight indexing of these folders can clog the CPU affecting the performance of the machine.

Is this feature similar to an existing feature in another tool?

Not that I know of.

Is this a feature you're prepared to implement, with support from the npm CLI team?

Yes.

[EvanLovely]

Apparently adding .noindex to the end of a folder name will prevent Spotlight from indexing it. Not sure exactly how we can have that feature help this issue, but felt it may be relevant to others.

Source

[PierBover]

Hopefully some member from the team will respond...

[pascalwhoop]

There is an answer on apple.stackexchange.com suggesting what @PierBover asks for works
https://apple.stackexchange.com/questions/247019/prevent-spotlight-from-indexing-folders-with-a-certain-name/258791#258791?newreg=7e2b0708cb774a82b0d945061dd735d0
However I cannot confirm this working. @PierBover has this strategy worked for you locally? If so what was the process that worked?

What I tried:

1. Running: find ~/Code -type d \( -name "node_modules" -o -name "bower_modules" \) -exec touch "{}/.metadata_never_index" \;
2. Force rebuild of spotlight index in given folder as described by Apple here
3. all my .js, .ts. .json files are still indexed in spotlight. That is the ones inside and outside the node_modules folders.

What's bugging me even more is that is does not index markdown files, at least not their content.

[PierBover]

Hey @pascalwhoop I honestly haven't tried it.

Maybe Spotlight hasn't finished rebuilding the index?

[mems]

See also ember-cli/ember-cli#2338

[npm-robot]

We're closing this issue as it has gone seven days without activity and without being labeled. If we haven't even labeled in issue in seven days then we're unlikely to ever read it.

If you are still experiencing the issue that led to you opening this or this is a feature request you're still interested in then we encourage you to open a new issue. If this was a support issue, you may be better served by joining package.communty and asking your question there.

For more information about our new issue aging policies and why we've instituted them please see our blog post.

[PierBover]

Thanks @mems for the link!

This is exactly how Ember CLI solved it. ember-cli/ember-cli@f760493

This shouldn't be closed...

Would the NPM team accept a PR for this feature?

[mems]

Chrome (and Opera) use the same mechanism for the user profile directory to exclude it from spotlight index.

See 395300 - Spotlight constantly indexing files in "Profile 2" - chromium - Monorail

[vesper8]

this .metadata_never_index trick does not seem to work at all. I made a script to insert a blank ".metadata_never_index" file in a bunch of folders (node_modules, bower_components, vendor) and verified that the files were created properly. Than I re-indexed with Spotlight and it still took forever.. create a humongous spotlight index (12gb) and basically did not respect these files at all

[EvanLovely]

@vesper8 I've never been able to get it to work either; would love it to.

[PierBover]

Adding the .metadata_never_index does work.

I just added it to my folder of repos, rebuilt Spotlight index with the instructions from Apple, and all my repos are gone from my Spotlight searches.

For some reason the usual mdutil commands don't work and in iTerm I only get the message Indexing enabled. instead of the usual screen.

Edit: I reversed the procedure (deleted .metadata_never_index and rebuilt index) and now I have my repos back in Spotlight.

I'm on High Sierra btw.

Would a PR with this feature be accepted? Can someone point me out where I should start looking to implement this?

[tim-phillips]

The true fix would be if Apple allowed us to blacklist folder names from the Spotlight index.

I've submitted feedback at https://www.apple.com/feedback/, I suggest others do the same.