Exclude directories from workspace for crawling performance reasons

[jevakallio]

@jojanaho reported on Twitter:

I started to experience slowdowns also yesterday, basically the the editor didnt respond to paste events for a while, and then all of a sudden multiple pastes from buffer

Not sure what caused it, but iirc markdown notes/links was showing some status update somewhere
I believe I had foam monorepo loaded, so it was prolly scanning various node_modules

In Foam we use workspace.findFiles, which has an exclude glob pattern option:
https://code.visualstudio.com/api/references/vscode-api#workspace (search for findFiles)

Markdown Notes uses the same:
https://github.com/kortina/vscode-markdown-notes/blob/c6d97c15c665a7f5f189b5a17d59ff808933c1f0/src/NoteWorkspace.ts#L291

Markdown links uses workspace.fs.readDirectory, which does not support an exclude glob:
https://github.com/tchayen/markdown-links/blob/78baab74c0d40a89eadcd3bfda8415ca2269ed6f/src/parsing.ts#L93

Foam and Markdown Notes already support files.exclude via workspace.findFiles, sowe can do this in .vscode/settings.json:

  "files.exclude":{
    "**/node_modules": true
  },

Steps to fix this issue are:

• Submit a PR to Markdown Links to move to using workspace apis instead of workspace.fs apis
• Add node_modules, _layout, and other known "bad" folders to files.exclude in foam-template
  • submit a PR to foam-gatsby-template
  • submit a PR to foam-eleventy-template
• Document how to fix this for existing workspaces

We could also research:

• Change the globbing pattern to be **/*{all markdown extensions here} in Foam -- this is a more of an advanced optimisation, as we already filter out all non-markdown files, so only work we save with this is iterating through a shorter array of paths
  • Submit the same change to Markdown Notes once we've gotten it to work

[jivthesh]

@jevakallio Can i work on this issues I am a beginer to open source community can u guide can until be how to work with this

[jevakallio]

@jivthesh I think @ingalless is already looking to make the code fixes in vscode markdown links, but another useful thing would be add node_modules directories to the exclude lists in the different templates.

Basically, that would involve submitting PRs to the projects, adding this setting:

  "files.exclude": {
    "**/node_modules": true,
  },

To these settings.json files:

• Default Foam template: https://github.com/foambubble/foam-template/blob/master/.vscode/settings.json
• Gatsby template: https://github.com/mathieudutour/foam-gatsby-template/blob/master/.vscode/settings.json
• Eleventy template: https://github.com/juanfrank77/foam-eleventy-template/blob/master/.vscode/settings.json

Here's an article that shows how to make a pull request:
https://www.freecodecamp.org/news/how-to-make-your-first-pull-request-on-github-3/

[ingalless]

Just to add

files.exclude is good, but I'd love to see it respecting .gitignore too!

[Thiruppathi]

@jevakallio I've fixed it for Gatsby template here.
mathieudutour/foam-gatsby-template#11

Can you please have a look at the PR?

[riccardoferretti]

this is now fixed in foam core by #300