New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Yarn workspaces to replace bootstrap command #899

Merged
merged 3 commits into from Jul 6, 2017

Conversation

Projects
None yet
8 participants
@bestander
Copy link
Contributor

bestander commented Jun 27, 2017

Description + Motivation and Context

Yarn has a Workspaces feature that implements Lerna's bootstrap logic in a nice "more atomic" and more native from a package manager point of view way.
The progress and all the links to the feature is here yarnpkg/yarn#3294.

Workspaces are still considered "experimental" but we are moving forward and are ready to turn it on for a couple of projects, e.g. facebook/jest#3906.

This PR integrates Yarn workspaces with Lerna by replacing all the bootstrap/hoisting logic with a simple call to yarn install.
Also it makes Lerna use package.json/workspaces field as Yarn does.

How Has This Been Tested?

  1. Manual:
  • update to Yarn 0.27.2
  • checkout jest with this PR facebook/jest#3906
  • run ../lerna/bin/lerna.js bootstrap
  • verify that workspace symlinks were installed in jest/node_modules
  • verify that unhoistable dependencies got installed in workspaces (jest/packages/jest-runtime/node_modules/strip-bom)
  • run yarn test-ci
  • added integration test

Types of changes

  • New feature (non-breaking change which adds functionality)

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.
@hzoo

This comment has been minimized.

Copy link
Member

hzoo commented Jun 27, 2017

Cool thanks @bestander!

@evocateur

This comment has been minimized.

Copy link
Member

evocateur commented Jun 27, 2017

Thanks @bestander!

@bestander bestander changed the title WIP: Support Yarn workspaces to replace bootstrap command Support Yarn workspaces to replace bootstrap command Jun 29, 2017

@bestander

This comment has been minimized.

Copy link
Contributor Author

bestander commented Jun 29, 2017

@hzoo @evocateur this is ready for review.

@Bnaya

This comment has been minimized.

Copy link

Bnaya commented Jun 30, 2017

@bestander Thanks for the great work!
Where i can see example for this setup?

@bestander

This comment has been minimized.

Copy link
Contributor Author

bestander commented Jun 30, 2017

@Bnaya we are turning it on for jest here facebook/jest#3906.
Waiting for yarn 0.27 to become stable

@bestander bestander force-pushed the bestander:yarn-workspaces branch from 8d93488 to 3d6f874 Jun 30, 2017

@jquense

This comment has been minimized.

Copy link

jquense commented Jul 5, 2017

yarn 27.0 is out yeah? can we merge this? I'd like to give it a shot

@bestander

This comment has been minimized.

Copy link
Contributor Author

bestander commented Jul 5, 2017

Yes, 0.27 is out and marked as stable.
Waiting for a review and merge please

@evocateur

This comment has been minimized.

Copy link
Member

evocateur commented Jul 5, 2017

looking now, thanks for understanding

@evocateur
Copy link
Member

evocateur left a comment

Apologies for the delay!

README.md Outdated
@@ -840,6 +840,23 @@ May also be configured in `lerna.json`:
}
```

#### --yarnWorkspaces

This comment has been minimized.

@evocateur

evocateur Jul 5, 2017

Member

I think this would be clearer as --use-workspaces, with the long-term goal of making it client-independent.

This comment has been minimized.

@bestander

bestander Jul 5, 2017

Author Contributor

ok

README.md Outdated
{
...
"npmClient": "yarn",
"yarnWorkspaces": ["bootstrap"]

This comment has been minimized.

@evocateur

evocateur Jul 5, 2017

Member

I don't think this is an appropriate place to whitelist commands; that's an implementation detail if workspaces are enabled at all. In this PR alone there's already an implicit "ls" member in this array, and that's not very clear. The useWorkspaces (the camelization of --use-workspaces) value should be a boolean.

This comment has been minimized.

@bestander

bestander Jul 5, 2017

Author Contributor

ok

@@ -55,6 +55,9 @@ export default class Repository {
}

get packageConfigs() {
if (this.lernaJson.yarnWorkspaces) {

This comment has been minimized.

@evocateur

evocateur Jul 5, 2017

Member

This isn't sufficient because it doesn't pick up any potential --use-workspaces flag in the CLI args, only the "durable" config from a lerna.json (mostly equivalent to the relationship of yarn to a .yarnrc file, CLI flags always supersede the lerna.json values).

This comment has been minimized.

@bestander

bestander Jul 5, 2017

Author Contributor

ok

const { yarnWorkspaces } = this.options;

if (yarnWorkspaces && (yarnWorkspaces.indexOf("bootstrap") >= 0)) {
this.installRootPackageOnly(callback);

This comment has been minimized.

@evocateur

evocateur Jul 5, 2017

Member

No symlinking happens in this case? Or is that handled by Yarn now? (I dug through yarnpkg/yarn#3294 but my eyes glazed over before I could figure it out, sorry)

This comment has been minimized.

@bestander

bestander Jul 5, 2017

Author Contributor

Yep, workspaces feature in Yarn creates symlinks in node_modules.
It also treats all workspaces as one installation, so it would hoist symlinks to the root node_modules unless there is a conflict, e.g. jest@next might be using jest-cli@stable in the root.

This comment has been minimized.

@evocateur

evocateur Jul 5, 2017

Member

Awesome, thanks for the explanation!

@bestander

This comment has been minimized.

Copy link
Contributor Author

bestander commented Jul 5, 2017

Thanks @evocateur, no need to apologize.

@bestander

This comment has been minimized.

Copy link
Contributor Author

bestander commented Jul 5, 2017

@evocateur, I've switched to --use-workspaces and tested that it works as a CLI config as well.

@bestander bestander force-pushed the bestander:yarn-workspaces branch from 2a663ac to a003740 Jul 5, 2017

@bestander

This comment has been minimized.

Copy link
Contributor Author

bestander commented Jul 6, 2017

I don't think my PR is responsible for appveyor fail

@SimenB

This comment has been minimized.

Copy link
Contributor

SimenB commented Jul 6, 2017

Oh, npm 5 strikes again.

    "lerna info version __TEST_VERSION__
    +package-3: ┌───────────────────────────────────────────────────────────────┐
    +package-3: │                    npm update check failed                    │
    +package-3: │              Try running with sudo or get access              │
    +package-3: │             to the local update config store via              │
    +package-3: │ sudo chown -R $USER:$(id -gn $USER) C:\\Users\\appveyor\\.config │
    +package-3: └───────────────────────────────────────────────────────────────┘

Export NO_UPDATE_NOTIFIER in the tests?

@evocateur

This comment has been minimized.

Copy link
Member

evocateur commented Jul 6, 2017

@SimenB

This comment has been minimized.

Copy link
Contributor

SimenB commented Jul 6, 2017

https://github.com/yeoman/update-notifier#user-settings

I also sent a PR to skip the check on CI automatically. Not sure if it will be accepted, though. yeoman/update-notifier#116

@SimenB SimenB referenced this pull request Jul 6, 2017

Merged

Skip update notifiers on CI #907

4 of 9 tasks complete
@evocateur
Copy link
Member

evocateur left a comment

Thanks @bestander!

@evocateur evocateur merged commit 616e44c into lerna:master Jul 6, 2017

{
...
"npmClient": "yarn",
"use-workspaces": true

This comment has been minimized.

@SimenB

SimenB Jul 6, 2017

Contributor

Weird that this is not camel case

This comment has been minimized.

@evocateur

evocateur Jul 6, 2017

Member

Whoops, I can fix momentarily.

@bestander

This comment has been minimized.

Copy link
Contributor Author

bestander commented Jul 6, 2017

Thanks a lot, guys!

@@ -840,6 +840,22 @@ May also be configured in `lerna.json`:
}
```

#### --use-workspaces

Enables integration with [Yarn Workspaces](https://github.com/yarnpkg/rfcs/blob/master/implemented/0000-workspaces-install-phase-1.md) (available since yarn@0.27+).

This comment has been minimized.

@hzoo

hzoo Jul 6, 2017

Member

Is there a a docs page on yarnpkg.com for workspaces?

This comment has been minimized.

@bestander

bestander Jul 6, 2017

Author Contributor

Not yet, I'll work on it in the next 3-4 weeks.

@hzoo

This comment has been minimized.

Copy link
Member

hzoo commented Jul 6, 2017

Nice work everyone!

@jquense

This comment has been minimized.

Copy link

jquense commented Jul 6, 2017

@bestander sorry to ping you here, but it doesn't seem like bins from workspaces are linked or generated. I've got a workspace packages/foo with a bin, but after an install there isn't anything in node_modules/.bin in the root or any workspaces that depend on foo.

wonderful feature tho, i'm really excited about it !

@bestander

This comment has been minimized.

Copy link
Contributor Author

bestander commented Jul 6, 2017

@jquense, might be a bug in Yarn.
File an issues there, we'll resolve it.

@Bnaya

This comment has been minimized.

Copy link

Bnaya commented Jul 6, 2017

I have it up and running in my project, with my .bin files in the appropriate places

@@ -55,6 +55,9 @@ export default class Repository {
}

get packageConfigs() {
if (this.lernaJson.useWorkspaces) {
return this.packageJson.workspaces;

This comment has been minimized.

@SimenB

SimenB Jul 7, 2017

Contributor

|| [DEFAULT_PACKAGE_GLOB];? This is not mentioned in the readme, so everything went boom when I tried to use this

This comment has been minimized.

@bestander

bestander Jul 7, 2017

Author Contributor

Yarn does not have a default (PR welcome ;), so it would boom there if this setting is missing.
There is a comment about package.json/workspaces being used instead of packages https://github.com/lerna/lerna#--use-workspaces

This comment has been minimized.

@SimenB

SimenB Jul 8, 2017

Contributor

You're correct, I'm just blind! 😀

PR welcome ;)

No doubt

@thomasfr

This comment has been minimized.

Copy link

thomasfr commented Jul 10, 2017

Is it possible to exclude some node modules from beeing hoisted to the root folder? Like the lerna no-hoist flag did before?

But besides that. Awesome work. Cuts the installation time by half with yarn workspaces for us!! From ~2minutes down to 58secs

@jquense

This comment has been minimized.

Copy link

jquense commented Jul 10, 2017

I don't think so regarding no-hoist since that's how yarn workspaces work and lerna is defering to it.

@jquense

This comment has been minimized.

Copy link

jquense commented Jul 11, 2017

So i've done some more testing. It appears like yarns install with workspaces doesn't different things from lerna bootstrap. The big things i've notice are:

  • yarn doesn't link local bins
  • yarn doesn't run local prepublish/prepare scripts in linked packages, which is required for build steps

I've tried a bunch of different configs and setups with the same results. As a replacement for bootstrap it doesn't do all the things :)

@bestander

This comment has been minimized.

Copy link
Contributor Author

bestander commented Jul 11, 2017

Thanks, @jquense.
Feel free to raise issues in https://github.com/yarnpkg/yarn/issues with repro steps and ping me.

@thomasfr

This comment has been minimized.

Copy link

thomasfr commented Jul 12, 2017

Just fyi i created an issue for --no-hoist at yarn: yarnpkg/yarn#3882

@Hypnosphi

This comment has been minimized.

Copy link
Contributor

Hypnosphi commented Aug 7, 2017

So is lerna bootstrap --use-workspaces a "shortcut" to yarn command?

@lock

This comment has been minimized.

Copy link

lock bot commented Dec 27, 2018

This thread has been automatically locked because there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock bot locked as resolved and limited conversation to collaborators Dec 27, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.