Support for monorepos, including lerna or yarn workspaces

[herkulano]

Currently, you're only caching the node_modules at the root level.
With lerna or yarn workspaces, there are sub-folders with node_modules in them that are not cached. This causes a failure to build. It then builds correctly if it's retried with a clean cache.

[fool]

related internal issue: netlify/buildbot#163

[zanona]

For my use-case (yarn workspaces) I have been able to work around this issue by using:

netlify.toml

[build]
  command = "npm i yarn@1.10.1 && yarn && yarn build"

Of course, this is not ideal as it takes additional time to run npm install and yarn install again, however, the cache dir which the npm installed version of yarn uses is the same so it should be all ready for the next time a build is triggered. Bottom line, it has increased building times by about 1 minute for me.

Hopefully, we will be able to revert to the expected behaviour soon, though :)

Btw, thanks for the help @fool

PS. Perhaps it's good practice to call yarn install with yarn $YARN_FLAGS as it should carry netlify or user defaults for installing the dependencies, such as --production

[eckdanny-osi]

I had a frustrating afternoon playing guess-and-check with netlify builds on my yarn workspaces + lerna repo. I'm 99% sure its todo with netlify's caching scheme for node_modules since my build goes from failing to working if i just clear the cache. I'd don't want to have to do this everytime tho... looking for a config option or ENV var to the effect of NO_CACHE or lerna clean and re-install as part of the build process for now :\

@zanona you can set a build env var YARN_VERSION.

I think why its working for you is actually the && yarn part where you reinstall deps.

UPDATE:

can confirm. I had a good deploy. Then I did a no code change (updated a README). Build failed. Redeployed with the "clear cache" btn and it worked. For now, I guess I'll put a lerna clean && lerna bootstrap into my build routine i dunno what todo

[eckdanny-osi]

Strange. I have a YARN_VERSION env var in my web config (set to 1.9.4) and the build logs seem to indicate its working...

but i changed my package.json#scripts#build to "build": "yarn -v && exit 1" and the result is not expected:

...
6:32:06 PM: Executing user command: yarn build
6:32:06 PM: yarn run v1.9.4
6:32:06 PM: $ yarn -v && exit 1
6:32:06 PM: 1.3.2

The issue I'm seeing in my dead builds:

error An unexpected error occurred: "patterns.map is not a function".

was resolved by yarnpkg/yarn#5114 and should work since yarn v1.4

I'll try @zanona's workaround.

Why do i not have the yarn version i think i should have?

UPDATE:

@zanona's workaround solved for now. I added a toml:

[build]
  command = "npm i -gs yarn && yarn build"

and my pkgJson#scripts:

{
    ...
    "prebuild": "lerna bootstrap",
    "build": "lerna run --stream build"
}

[iddan]

I expect Netlify to be able to look for different Netlify.toml files: So for deploying a monorepo with multiple sites I would create multiple Netlify sites each of them with a different Netlify.toml.

[tsaiDavid]

Just curious if anyone has [since] had luck with caching non-root node_modules?

[praneybehl]

I am in the same boat, even though I try to bootstrap lerna before the build command it still not able to...

[lucashfreitas]

+1

[jaredh159]

I also would really like to drop a netlify.toml file into my base dir, and have multiple of them for my monorepo. I was a bit surprised it didn't work like this already. :(

[marcosscriven]

Any news on this please @fool? In the meantime, please could you update the documentation to make it clear you can only have one netlify.toml for the whole repo, even though most other things work fine by per-site config in the UI.

The big thing missing for me is the greater redirect control one gets with the netlify.toml config, but that's not common to my sites in each of the sub directories.

I don't really even need to share anything, so presumably you could make each site build only aware of a given sub-dir, as if that's the root of the whole thing - then everything else would work as-is?

[backspace]

I’m working on a PR that builds both a static documentation Middleman site and an Ember UI. Part of the script involves checking what branch is being deployed and using a different _redirects file if it’s a UI PR, so maybe that would be useful to people.

But the caching is my concern, it doesn’t seem like it can be hacked around in this manner 😞