Support Yarn workspaces to replace bootstrap command #899

bestander · 2017-06-27T02:41:43Z

Description + Motivation and Context

Yarn has a Workspaces feature that implements Lerna's bootstrap logic in a nice "more atomic" and more native from a package manager point of view way.
The progress and all the links to the feature is here yarnpkg/yarn#3294.

Workspaces are still considered "experimental" but we are moving forward and are ready to turn it on for a couple of projects, e.g. facebook/jest#3906.

This PR integrates Yarn workspaces with Lerna by replacing all the bootstrap/hoisting logic with a simple call to yarn install.
Also it makes Lerna use package.json/workspaces field as Yarn does.

How Has This Been Tested?

Manual:

update to Yarn 0.27.2
checkout jest with this PR facebook/jest#3906
run ../lerna/bin/lerna.js bootstrap
verify that workspace symlinks were installed in jest/node_modules
verify that unhoistable dependencies got installed in workspaces (jest/packages/jest-runtime/node_modules/strip-bom)
run yarn test-ci

added integration test

Types of changes

New feature (non-breaking change which adds functionality)

Checklist:

My code follows the code style of this project.
My change requires a change to the documentation.
I have updated the documentation accordingly.
I have read the CONTRIBUTING document.
I have added tests to cover my changes.
All new and existing tests passed.

hzoo · 2017-06-27T02:46:07Z

Cool thanks @bestander!

evocateur · 2017-06-27T16:46:09Z

Thanks @bestander!

bestander · 2017-06-29T03:57:35Z

@hzoo @evocateur this is ready for review.

Bnaya · 2017-06-30T09:49:19Z

@bestander Thanks for the great work!
Where i can see example for this setup?

bestander · 2017-06-30T18:39:59Z

@Bnaya we are turning it on for jest here facebook/jest#3906.
Waiting for yarn 0.27 to become stable

jquense · 2017-07-05T18:37:01Z

yarn 27.0 is out yeah? can we merge this? I'd like to give it a shot

bestander · 2017-07-05T20:57:20Z

Yes, 0.27 is out and marked as stable.
Waiting for a review and merge please

evocateur · 2017-07-05T22:34:34Z

looking now, thanks for understanding

evocateur

Apologies for the delay!

evocateur · 2017-07-05T23:21:27Z

README.md

@@ -840,6 +840,23 @@ May also be configured in `lerna.json`:
 }
 ```

+#### --yarnWorkspaces


I think this would be clearer as --use-workspaces, with the long-term goal of making it client-independent.

evocateur · 2017-07-05T23:21:27Z

README.md

+{
+  ...
+  "npmClient": "yarn",
+  "yarnWorkspaces": ["bootstrap"]


I don't think this is an appropriate place to whitelist commands; that's an implementation detail if workspaces are enabled at all. In this PR alone there's already an implicit "ls" member in this array, and that's not very clear. The useWorkspaces (the camelization of --use-workspaces) value should be a boolean.

evocateur · 2017-07-05T23:21:27Z

src/Repository.js

@@ -55,6 +55,9 @@ export default class Repository {
  }

  get packageConfigs() {
+    if (this.lernaJson.yarnWorkspaces) {


This isn't sufficient because it doesn't pick up any potential --use-workspaces flag in the CLI args, only the "durable" config from a lerna.json (mostly equivalent to the relationship of yarn to a .yarnrc file, CLI flags always supersede the lerna.json values).

evocateur · 2017-07-05T23:21:27Z

src/commands/BootstrapCommand.js

+    const { yarnWorkspaces } = this.options;
+
+    if (yarnWorkspaces && (yarnWorkspaces.indexOf("bootstrap") >= 0)) {
+      this.installRootPackageOnly(callback);


No symlinking happens in this case? Or is that handled by Yarn now? (I dug through yarnpkg/yarn#3294 but my eyes glazed over before I could figure it out, sorry)

Yep, workspaces feature in Yarn creates symlinks in node_modules.
It also treats all workspaces as one installation, so it would hoist symlinks to the root node_modules unless there is a conflict, e.g. jest@next might be using jest-cli@stable in the root.

Awesome, thanks for the explanation!

bestander · 2017-07-05T23:25:59Z

Thanks @evocateur, no need to apologize.

bestander · 2017-07-05T23:40:09Z

@evocateur, I've switched to --use-workspaces and tested that it works as a CLI config as well.

bestander · 2017-07-06T02:48:20Z

I don't think my PR is responsible for appveyor fail

SimenB · 2017-07-06T04:53:54Z

Oh, npm 5 strikes again.

    "lerna info version __TEST_VERSION__
    +package-3: ┌───────────────────────────────────────────────────────────────┐
    +package-3: │                    npm update check failed                    │
    +package-3: │              Try running with sudo or get access              │
    +package-3: │             to the local update config store via              │
    +package-3: │ sudo chown -R $USER:$(id -gn $USER) C:\\Users\\appveyor\\.config │
    +package-3: └───────────────────────────────────────────────────────────────┘

Export NO_UPDATE_NOTIFIER in the tests?

evocateur · 2017-07-06T04:59:11Z

That's a great idea! Is that documented anywhere? I'm not against including that fix in this PR.

…

On Wed, Jul 5, 2017 at 9:53 PM, Simen Bekkhus ***@***.***> wrote: Oh, npm 5 strikes again. "lerna info version __TEST_VERSION__ +package-3: ┌───────────────────────────────────────────────────────────────┐ +package-3: │ npm update check failed │ +package-3: │ Try running with sudo or get access │ +package-3: │ to the local update config store via │ +package-3: │ sudo chown -R $USER:$(id -gn $USER) C:\\Users\\appveyor\\.config │ +package-3: └───────────────────────────────────────────────────────────────┘ Export NO_UPDATE_NOTIFIER in the tests? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#899 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAV5SyAgrZCxbecRqfYQsfiidSqZLXDks5sLGhigaJpZM4OGFNC> .

SimenB · 2017-07-06T05:06:59Z

https://github.com/yeoman/update-notifier#user-settings

I also sent a PR to skip the check on CI automatically. Not sure if it will be accepted, though. yeoman/update-notifier#116

evocateur

Thanks @bestander!

SimenB · 2017-07-06T15:39:36Z

README.md

+{
+  ...
+  "npmClient": "yarn",
+  "use-workspaces": true


Weird that this is not camel case

Whoops, I can fix momentarily.

bestander · 2017-07-06T18:53:22Z

Thanks a lot, guys!

hzoo · 2017-07-06T20:10:11Z

README.md

@@ -840,6 +840,22 @@ May also be configured in `lerna.json`:
 }
 ```

+#### --use-workspaces
+
+Enables integration with [Yarn Workspaces](https://github.com/yarnpkg/rfcs/blob/master/implemented/0000-workspaces-install-phase-1.md) (available since yarn@0.27+).  


Is there a a docs page on yarnpkg.com for workspaces?

Not yet, I'll work on it in the next 3-4 weeks.

hzoo · 2017-07-06T20:10:20Z

Nice work everyone!

jquense · 2017-07-06T21:55:43Z

@bestander sorry to ping you here, but it doesn't seem like bins from workspaces are linked or generated. I've got a workspace packages/foo with a bin, but after an install there isn't anything in node_modules/.bin in the root or any workspaces that depend on foo.

wonderful feature tho, i'm really excited about it !

bestander · 2017-07-06T21:56:57Z

@jquense, might be a bug in Yarn.
File an issues there, we'll resolve it.

Bnaya · 2017-07-06T22:14:05Z

I have it up and running in my project, with my .bin files in the appropriate places

SimenB · 2017-07-07T08:40:23Z

src/Repository.js

@@ -55,6 +55,9 @@ export default class Repository {
  }

  get packageConfigs() {
+    if (this.lernaJson.useWorkspaces) {
+      return this.packageJson.workspaces;


|| [DEFAULT_PACKAGE_GLOB];? This is not mentioned in the readme, so everything went boom when I tried to use this

Yarn does not have a default (PR welcome ;), so it would boom there if this setting is missing.
There is a comment about package.json/workspaces being used instead of packages https://github.com/lerna/lerna#--use-workspaces

You're correct, I'm just blind! 😀

PR welcome ;)

No doubt

thomasfr · 2017-07-10T12:27:22Z

Is it possible to exclude some node modules from beeing hoisted to the root folder? Like the lerna no-hoist flag did before?

But besides that. Awesome work. Cuts the installation time by half with yarn workspaces for us!! From ~2minutes down to 58secs

jquense · 2017-07-10T12:50:22Z

I don't think so regarding no-hoist since that's how yarn workspaces work and lerna is defering to it.

jquense · 2017-07-11T18:29:24Z

So i've done some more testing. It appears like yarns install with workspaces doesn't different things from lerna bootstrap. The big things i've notice are:

yarn doesn't link local bins
yarn doesn't run local prepublish/prepare scripts in linked packages, which is required for build steps

I've tried a bunch of different configs and setups with the same results. As a replacement for bootstrap it doesn't do all the things :)

bestander · 2017-07-11T19:27:04Z

Thanks, @jquense.
Feel free to raise issues in https://github.com/yarnpkg/yarn/issues with repro steps and ping me.

thomasfr · 2017-07-12T09:09:08Z

Just fyi i created an issue for --no-hoist at yarn: yarnpkg/yarn#3882

Hypnosphi · 2017-08-07T20:13:52Z

So is lerna bootstrap --use-workspaces a "shortcut" to yarn command?

lock · 2018-12-27T13:09:58Z

This thread has been automatically locked because there has not been any recent activity after it was closed. Please open a new issue for related bugs.

FezVrasta mentioned this pull request Jun 28, 2017

[refactor] Build improvements popperjs/popper-core#326

Merged

bestander mentioned this pull request Jun 29, 2017

Integrated with Yarn workspaces facebook/jest#3906

Merged

bestander changed the title ~~WIP: Support Yarn workspaces to replace bootstrap command~~ Support Yarn workspaces to replace bootstrap command Jun 29, 2017

evocateur requested changes Jul 5, 2017

View changes

bestander added 2 commits Jun 27, 2017

WIP: yarn workspaces

a6e8b9a

changed to --use-workspaces flag after feedback

a003740

SimenB mentioned this pull request Jul 6, 2017

Skip update notifiers on CI #907

Merged

4 of 9 tasks

fix typo [skip ci]

b801d57

evocateur approved these changes Jul 6, 2017

View changes

evocateur merged commit 616e44c into lerna:master Jul 6, 2017

evocateur added the Tag: Enhancement label Jul 6, 2017

SimenB reviewed Jul 6, 2017

View changes

hzoo reviewed Jul 6, 2017

View changes

SimenB reviewed Jul 7, 2017

View changes

Bnaya mentioned this pull request Jul 7, 2017

When using with workspaces, yarn ignores nohoist settings #909

Closed

marionebl mentioned this pull request Nov 18, 2017

[question] Using yarn commands with non-published lerna dependencies #501

Closed

kiriappeee mentioned this pull request Feb 7, 2018

Upgrading Node to 8.9.4 and/or upgrading the yarn version in the image bufferapp/buffer-publish#262

Closed

evocateur mentioned this pull request Mar 13, 2018

lerna bootstrap failing as a postinstall step with yarn workspaces #1308

Closed

tech4him1 mentioned this pull request Sep 1, 2018

Package has both yarn.lock and package-lock.json netlify/netlify-cms#1516

Closed

bd82 mentioned this pull request Nov 9, 2018

Convert from yarn to npm jhipster/prettier-java#93

Closed

lock bot locked as resolved and limited conversation to collaborators Dec 27, 2018

lerna / lerna Public

Support Yarn workspaces to replace bootstrap command #899

Support Yarn workspaces to replace bootstrap command #899

Conversation

bestander commented Jun 27, 2017 • edited

Description + Motivation and Context

How Has This Been Tested?

Types of changes

Checklist:

hzoo commented Jun 27, 2017

evocateur commented Jun 27, 2017

bestander commented Jun 29, 2017

Bnaya commented Jun 30, 2017

bestander commented Jun 30, 2017

jquense commented Jul 5, 2017

bestander commented Jul 5, 2017

evocateur commented Jul 5, 2017

evocateur left a comment

This comment has been minimized.

evocateur Jul 5, 2017 Member

This comment has been minimized.

bestander Jul 5, 2017 Author Contributor

This comment has been minimized.

evocateur Jul 5, 2017 Member

This comment has been minimized.

bestander Jul 5, 2017 Author Contributor

This comment has been minimized.

evocateur Jul 5, 2017 Member

This comment has been minimized.

bestander Jul 5, 2017 Author Contributor

This comment has been minimized.

evocateur Jul 5, 2017 Member

This comment has been minimized.

bestander Jul 5, 2017 Author Contributor

This comment has been minimized.

evocateur Jul 5, 2017 Member

bestander commented Jul 5, 2017

bestander commented Jul 5, 2017

bestander commented Jul 6, 2017

SimenB commented Jul 6, 2017

evocateur commented Jul 6, 2017

SimenB commented Jul 6, 2017

evocateur left a comment

This comment has been minimized.

SimenB Jul 6, 2017 Contributor

This comment has been minimized.

evocateur Jul 6, 2017 Member

bestander commented Jul 6, 2017

This comment has been minimized.

hzoo Jul 6, 2017 Member

This comment has been minimized.

bestander Jul 6, 2017 Author Contributor

hzoo commented Jul 6, 2017

jquense commented Jul 6, 2017 • edited

bestander commented Jul 6, 2017

Bnaya commented Jul 6, 2017

This comment has been minimized.

SimenB Jul 7, 2017 • edited Contributor

This comment has been minimized.

bestander Jul 7, 2017 Author Contributor

This comment has been minimized.

SimenB Jul 8, 2017 Contributor

thomasfr commented Jul 10, 2017 • edited

jquense commented Jul 10, 2017

jquense commented Jul 11, 2017

bestander commented Jul 11, 2017

thomasfr commented Jul 12, 2017

Hypnosphi commented Aug 7, 2017

lock bot commented Dec 27, 2018

bestander commented Jun 27, 2017 •

edited

evocateur Jul 5, 2017
Member

bestander Jul 5, 2017
Author Contributor

evocateur Jul 5, 2017
Member

bestander Jul 5, 2017
Author Contributor

evocateur Jul 5, 2017
Member

bestander Jul 5, 2017
Author Contributor

evocateur Jul 5, 2017
Member

bestander Jul 5, 2017
Author Contributor

evocateur Jul 5, 2017
Member

SimenB Jul 6, 2017
Contributor

evocateur Jul 6, 2017
Member

hzoo Jul 6, 2017
Member

bestander Jul 6, 2017
Author Contributor

jquense commented Jul 6, 2017 •

edited

SimenB Jul 7, 2017 •

edited

Contributor

bestander Jul 7, 2017
Author Contributor

SimenB Jul 8, 2017
Contributor

thomasfr commented Jul 10, 2017 •

edited