Yarn workspaces support improvements

[ehmicky]

For monorepos where each package has its own Netlify Site, we recommend using a different base directory for each of them, and optionally a different netlify.toml. At least that's my assumption, please correct me if I'm wrong.

The problem is that, when using Yarn workspaces, the yarn.lock file will be at the top-level, not inside each package. Because of this, we currently don't detect it and we use npm instead of yarn.

One workaround is to create an empty yarn.lock file inside packages directories, but that's hacky.

This problem was reported by @50bbx here.

[pixelbreaker]

I'm having really slow builds due to this issue. I have a large app in /www and related common UI in /components. When I set up as a monorepo with a base folder of /www in the Netlify UI, and hacked a yarn.lock into /www with a symlink to the one in the root, then Yarn runs and redownloads/compiles all packages every time a build runs.

This is costing us about 95 seconds of build minutes for every single deploy.

[slorber]

Hey

Having an empty yarn.lock in a sub package seems to work to tell netlify to use Yarn, however I encouter the same issue as @pixelbreaker with Yarn downloading each time dependencies.

As dependencies end up being in both root/node_modules + root/packages/mySiteToDeploy/node_modules, it's not totally clear how Netlify handles caching

Also, there is a "yarn install --focus" command that we should be able to leverage so that Yarn download only the dependencies for the package we are trying to build. Currently if you have a very large monorepo with a packages having very different dependencies, all dependencies have to be downloaded each time. (see https://classic.yarnpkg.com/blog/2018/05/18/focused-workspaces/)

Also, on my personal website (opensource gatsby site, no workspace) I had yarn install && yarn build as a build command. I was surprised that it would redownload dependencies on each commit. As adviced by someone from Forestry who asked someone of the Netlify team, moving to just yarn build did solve the problem (as Netlify download deps anyway), but it's not totally clear to me why?

For monorepo support doesn't it make sense to let the user add himself yarn install or yarn install --focus to his build command? Both cases make sense to me, it depends on if the deployed app first need to build extra packages of the monorepo (in which case having all deps seems necessary) or if the extra packages from which the app depends are already build (in which case we can use --focus to make the build faster)

[slorber]

Also something related, but it would be convenient if the netlify support could check parent folders for .nvmrc that specify node version, so that we don't have to put a .nvmrc to each subfolder if the repo use the same node version across all projects (it's more convenient to have one at the monorepo root)

[JGAntunes]

Hey folks 👋

Seems like the original issue reported by @ehmicky has been fixed meanwhile, users can use the NETLIFY_USE_YARN=true environment variable to force yarn usage in the absence of a yarn.lock file in the base directory. For more information on this - https://docs.netlify.com/configure-builds/environment-variables/#netlify-configuration-variables.

Meanwhile, for the cache related issues with yarn workspaces we have the following open #479 and work being done in #526, so we can follow up the discussion in there. Closing this 👍

Also, there is a "yarn install --focus" command that we should be able to leverage so that Yarn download only the dependencies for the package we are trying to build.

Unfortunately seems like it's not a straight forward process, as per - yarnpkg/yarn#6715 - focus assumes all the sibling packages are pushed to npm which is not always the case.

Also something related, but it would be convenient if the netlify support could check parent folders for .nvmrc that specify node version

That's a good call actually. Mind opening up a separate issue for it?

[kaganjd]

I know this is closed but wanted to share an exciting update about Yarn workspaces in case you're still tracking the issue @slorber: we’ve made improvements to how we cache dependencies for these projects. You can try out our new caching by creating an environment variable called NETLIFY_YARN_WORKSPACES and setting it to "true"

We want your feedback before we roll this out more widely- please let us know here or in the forum what your experience is like! You can also read more about this change here: https://answers.netlify.com/t/improved-caching-for-yarn-workspaces/36066