Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for monorepos, including lerna or yarn workspaces #196

Open
herkulano opened this issue Sep 4, 2018 · 11 comments

Comments

@herkulano
Copy link

commented Sep 4, 2018

Currently, you're only caching the node_modules at the root level.
With lerna or yarn workspaces, there are sub-folders with node_modules in them that are not cached. This causes a failure to build. It then builds correctly if it's retried with a clean cache.

@fool

This comment has been minimized.

Copy link
Member

commented Sep 4, 2018

related internal issue: netlify/buildbot#163

@zanona

This comment has been minimized.

Copy link

commented Oct 12, 2018

For my use-case (yarn workspaces) I have been able to work around this issue by using:

netlify.toml

[build]
  command = "npm i yarn@1.10.1 && yarn && yarn build"

Of course, this is not ideal as it takes additional time to run npm install and yarn install again, however, the cache dir which the npm installed version of yarn uses is the same so it should be all ready for the next time a build is triggered. Bottom line, it has increased building times by about 1 minute for me.

Hopefully, we will be able to revert to the expected behaviour soon, though :)

Btw, thanks for the help @fool

PS. Perhaps it's good practice to call yarn install with yarn $YARN_FLAGS as it should carry netlify or user defaults for installing the dependencies, such as --production

@eckdanny-osi

This comment has been minimized.

Copy link

commented Oct 12, 2018

I had a frustrating afternoon playing guess-and-check with netlify builds on my yarn workspaces + lerna repo. I'm 99% sure its todo with netlify's caching scheme for node_modules since my build goes from failing to working if i just clear the cache. I'd don't want to have to do this everytime tho... looking for a config option or ENV var to the effect of NO_CACHE or lerna clean and re-install as part of the build process for now :\

@zanona you can set a build env var YARN_VERSION.

I think why its working for you is actually the && yarn part where you reinstall deps.

UPDATE:

can confirm. I had a good deploy. Then I did a no code change (updated a README). Build failed. Redeployed with the "clear cache" btn and it worked. For now, I guess I'll put a lerna clean && lerna bootstrap into my build routine i dunno what todo

@eckdanny-osi

This comment has been minimized.

Copy link

commented Oct 13, 2018

Strange. I have a YARN_VERSION env var in my web config (set to 1.9.4) and the build logs seem to indicate its working...

but i changed my package.json#scripts#build to "build": "yarn -v && exit 1" and the result is not expected:

...
6:32:06 PM: Executing user command: yarn build
6:32:06 PM: yarn run v1.9.4
6:32:06 PM: $ yarn -v && exit 1
6:32:06 PM: 1.3.2

The issue I'm seeing in my dead builds:

error An unexpected error occurred: "patterns.map is not a function".

was resolved by yarnpkg/yarn#5114 and should work since yarn v1.4

I'll try @zanona's workaround.

Why do i not have the yarn version i think i should have?

UPDATE:

@zanona's workaround solved for now. I added a toml:

[build]
  command = "npm i -gs yarn && yarn build"

and my pkgJson#scripts:

{
    ...
    "prebuild": "lerna bootstrap",
    "build": "lerna run --stream build"
}
@iddan

This comment has been minimized.

Copy link

commented Dec 10, 2018

I expect Netlify to be able to look for different Netlify.toml files: So for deploying a monorepo with multiple sites I would create multiple Netlify sites each of them with a different Netlify.toml.

RobinCsl added a commit to kiwicom/margarita that referenced this issue Jan 15, 2019

@tsaiDavid

This comment has been minimized.

Copy link

commented Feb 27, 2019

Just curious if anyone has [since] had luck with caching non-root node_modules?

@praneybehl

This comment has been minimized.

Copy link

commented Apr 3, 2019

I am in the same boat, even though I try to bootstrap lerna before the build command it still not able to...

@fool fool changed the title Support for lerna or yarn workspaces Support for monorepos, including lerna or yarn workspaces Jun 27, 2019

dz0ny added a commit to dz0ny/build-image that referenced this issue Jul 4, 2019

Pre-build steps
I believe this is the way to implement the pre-build changes needed by many issues with the current build process. It allows a bit more flexibility for advanced users and does not break things for existing users.

Here are some examples:

## Skipping unnecessary language installs (netlify#141)

One can create `.netlify/remove_languages.sh` with contents:
```bash
rm Pipfile
rm Pipfile.lock
rm requirements.txt
rm runtime.txt
```

## Specify Composer file (netlify#237)

One can create `.netlify/patch_composer.sh` with contents:
```bash
mv -f composer-netlify.json composer.json
```

## Deploying multiple sites from the same repo (netlify#196)

Add enviroment vairable $NETLIFY_SITE to project
Create a `.netlify/netfily-site.sh` with contents:
```bash
mv -f .netlify/$NETLIFY_SITE netlify.yaml
```
@dz0ny dz0ny referenced this issue Jul 4, 2019
@lucashfreitas

This comment has been minimized.

Copy link

commented Jul 8, 2019

+1

@jaredh159

This comment has been minimized.

Copy link

commented Jul 19, 2019

I also would really like to drop a netlify.toml file into my base dir, and have multiple of them for my monorepo. I was a bit surprised it didn't work like this already. :(

@marcosscriven

This comment has been minimized.

Copy link

commented Aug 19, 2019

Any news on this please @fool? In the meantime, please could you update the documentation to make it clear you can only have one netlify.toml for the whole repo, even though most other things work fine by per-site config in the UI.

The big thing missing for me is the greater redirect control one gets with the netlify.toml config, but that's not common to my sites in each of the sub directories.

I don't really even need to share anything, so presumably you could make each site build only aware of a given sub-dir, as if that's the root of the whole thing - then everything else would work as-is?

@backspace

This comment has been minimized.

Copy link

commented Aug 20, 2019

I’m working on a PR that builds both a static documentation Middleman site and an Ember UI. Part of the script involves checking what branch is being deployed and using a different _redirects file if it’s a UI PR, so maybe that would be useful to people.

But the caching is my concern, it doesn’t seem like it can be hacked around in this manner 😞

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.