Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Yarn workspaces to replace bootstrap command #899

Merged
merged 3 commits into from Jul 6, 2017
Merged

Support Yarn workspaces to replace bootstrap command #899

merged 3 commits into from Jul 6, 2017

Conversation

@bestander
Copy link
Contributor

@bestander bestander commented Jun 27, 2017

Description + Motivation and Context

Yarn has a Workspaces feature that implements Lerna's bootstrap logic in a nice "more atomic" and more native from a package manager point of view way.
The progress and all the links to the feature is here yarnpkg/yarn#3294.

Workspaces are still considered "experimental" but we are moving forward and are ready to turn it on for a couple of projects, e.g. facebook/jest#3906.

This PR integrates Yarn workspaces with Lerna by replacing all the bootstrap/hoisting logic with a simple call to yarn install.
Also it makes Lerna use package.json/workspaces field as Yarn does.

How Has This Been Tested?

  1. Manual:
  • update to Yarn 0.27.2
  • checkout jest with this PR facebook/jest#3906
  • run ../lerna/bin/lerna.js bootstrap
  • verify that workspace symlinks were installed in jest/node_modules
  • verify that unhoistable dependencies got installed in workspaces (jest/packages/jest-runtime/node_modules/strip-bom)
  • run yarn test-ci
  • added integration test

Types of changes

  • New feature (non-breaking change which adds functionality)

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.
@hzoo
Copy link
Member

@hzoo hzoo commented Jun 27, 2017

Cool thanks @bestander!

@evocateur
Copy link
Member

@evocateur evocateur commented Jun 27, 2017

Thanks @bestander!

@bestander bestander changed the title WIP: Support Yarn workspaces to replace bootstrap command Support Yarn workspaces to replace bootstrap command Jun 29, 2017
@bestander
Copy link
Contributor Author

@bestander bestander commented Jun 29, 2017

@hzoo @evocateur this is ready for review.

@Bnaya
Copy link

@Bnaya Bnaya commented Jun 30, 2017

@bestander Thanks for the great work!
Where i can see example for this setup?

@bestander
Copy link
Contributor Author

@bestander bestander commented Jun 30, 2017

@Bnaya we are turning it on for jest here facebook/jest#3906.
Waiting for yarn 0.27 to become stable

@jquense
Copy link

@jquense jquense commented Jul 5, 2017

yarn 27.0 is out yeah? can we merge this? I'd like to give it a shot

@bestander
Copy link
Contributor Author

@bestander bestander commented Jul 5, 2017

Yes, 0.27 is out and marked as stable.
Waiting for a review and merge please

@evocateur
Copy link
Member

@evocateur evocateur commented Jul 5, 2017

looking now, thanks for understanding

Copy link
Member

@evocateur evocateur left a comment

Apologies for the delay!

README.md Outdated
@@ -840,6 +840,23 @@ May also be configured in `lerna.json`:
}
```

#### --yarnWorkspaces

This comment has been minimized.

@evocateur

evocateur Jul 5, 2017
Member

I think this would be clearer as --use-workspaces, with the long-term goal of making it client-independent.

This comment has been minimized.

@bestander

bestander Jul 5, 2017
Author Contributor

ok

README.md Outdated
{
...
"npmClient": "yarn",
"yarnWorkspaces": ["bootstrap"]

This comment has been minimized.

@evocateur

evocateur Jul 5, 2017
Member

I don't think this is an appropriate place to whitelist commands; that's an implementation detail if workspaces are enabled at all. In this PR alone there's already an implicit "ls" member in this array, and that's not very clear. The useWorkspaces (the camelization of --use-workspaces) value should be a boolean.

This comment has been minimized.

@bestander

bestander Jul 5, 2017
Author Contributor

ok

@@ -55,6 +55,9 @@ export default class Repository {
}

get packageConfigs() {
if (this.lernaJson.yarnWorkspaces) {

This comment has been minimized.

@evocateur

evocateur Jul 5, 2017
Member

This isn't sufficient because it doesn't pick up any potential --use-workspaces flag in the CLI args, only the "durable" config from a lerna.json (mostly equivalent to the relationship of yarn to a .yarnrc file, CLI flags always supersede the lerna.json values).

This comment has been minimized.

@bestander

bestander Jul 5, 2017
Author Contributor

ok

const { yarnWorkspaces } = this.options;

if (yarnWorkspaces && (yarnWorkspaces.indexOf("bootstrap") >= 0)) {
this.installRootPackageOnly(callback);

This comment has been minimized.

@evocateur

evocateur Jul 5, 2017
Member

No symlinking happens in this case? Or is that handled by Yarn now? (I dug through yarnpkg/yarn#3294 but my eyes glazed over before I could figure it out, sorry)

This comment has been minimized.

@bestander

bestander Jul 5, 2017
Author Contributor

Yep, workspaces feature in Yarn creates symlinks in node_modules.
It also treats all workspaces as one installation, so it would hoist symlinks to the root node_modules unless there is a conflict, e.g. jest@next might be using jest-cli@stable in the root.

This comment has been minimized.

@evocateur

evocateur Jul 5, 2017
Member

Awesome, thanks for the explanation!

@bestander
Copy link
Contributor Author

@bestander bestander commented Jul 5, 2017

Thanks @evocateur, no need to apologize.

@bestander
Copy link
Contributor Author

@bestander bestander commented Jul 5, 2017

@evocateur, I've switched to --use-workspaces and tested that it works as a CLI config as well.

@bestander
Copy link
Contributor Author

@bestander bestander commented Jul 6, 2017

I don't think my PR is responsible for appveyor fail

@SimenB
Copy link
Contributor

@SimenB SimenB commented Jul 6, 2017

Oh, npm 5 strikes again.

    "lerna info version __TEST_VERSION__
    +package-3: ┌───────────────────────────────────────────────────────────────┐
    +package-3: │                    npm update check failed                    │
    +package-3: │              Try running with sudo or get access              │
    +package-3: │             to the local update config store via              │
    +package-3: │ sudo chown -R $USER:$(id -gn $USER) C:\\Users\\appveyor\\.config │
    +package-3: └───────────────────────────────────────────────────────────────┘

Export NO_UPDATE_NOTIFIER in the tests?

@evocateur
Copy link
Member

@evocateur evocateur commented Jul 6, 2017

@SimenB
Copy link
Contributor

@SimenB SimenB commented Jul 6, 2017

https://github.com/yeoman/update-notifier#user-settings

I also sent a PR to skip the check on CI automatically. Not sure if it will be accepted, though. yeoman/update-notifier#116

@SimenB SimenB mentioned this pull request Jul 6, 2017
4 of 9 tasks
Copy link
Member

@evocateur evocateur left a comment

Thanks @bestander!

@evocateur evocateur merged commit 616e44c into lerna:master Jul 6, 2017
{
...
"npmClient": "yarn",
"use-workspaces": true

This comment has been minimized.

@SimenB

SimenB Jul 6, 2017
Contributor

Weird that this is not camel case

This comment has been minimized.

@evocateur

evocateur Jul 6, 2017
Member

Whoops, I can fix momentarily.

@bestander
Copy link
Contributor Author

@bestander bestander commented Jul 6, 2017

Thanks a lot, guys!

@@ -840,6 +840,22 @@ May also be configured in `lerna.json`:
}
```

#### --use-workspaces

Enables integration with [Yarn Workspaces](https://github.com/yarnpkg/rfcs/blob/master/implemented/0000-workspaces-install-phase-1.md) (available since yarn@0.27+).

This comment has been minimized.

@hzoo

hzoo Jul 6, 2017
Member

Is there a a docs page on yarnpkg.com for workspaces?

This comment has been minimized.

@bestander

bestander Jul 6, 2017
Author Contributor

Not yet, I'll work on it in the next 3-4 weeks.

@hzoo
Copy link
Member

@hzoo hzoo commented Jul 6, 2017

Nice work everyone!

@jquense
Copy link

@jquense jquense commented Jul 6, 2017

@bestander sorry to ping you here, but it doesn't seem like bins from workspaces are linked or generated. I've got a workspace packages/foo with a bin, but after an install there isn't anything in node_modules/.bin in the root or any workspaces that depend on foo.

wonderful feature tho, i'm really excited about it !

@bestander
Copy link
Contributor Author

@bestander bestander commented Jul 6, 2017

@jquense, might be a bug in Yarn.
File an issues there, we'll resolve it.

@Bnaya
Copy link

@Bnaya Bnaya commented Jul 6, 2017

I have it up and running in my project, with my .bin files in the appropriate places

@@ -55,6 +55,9 @@ export default class Repository {
}

get packageConfigs() {
if (this.lernaJson.useWorkspaces) {
return this.packageJson.workspaces;

This comment has been minimized.

@SimenB

SimenB Jul 7, 2017
Contributor

|| [DEFAULT_PACKAGE_GLOB];? This is not mentioned in the readme, so everything went boom when I tried to use this

This comment has been minimized.

@bestander

bestander Jul 7, 2017
Author Contributor

Yarn does not have a default (PR welcome ;), so it would boom there if this setting is missing.
There is a comment about package.json/workspaces being used instead of packages https://github.com/lerna/lerna#--use-workspaces

This comment has been minimized.

@SimenB

SimenB Jul 8, 2017
Contributor

You're correct, I'm just blind! 😀

PR welcome ;)

No doubt

@thomasfr
Copy link

@thomasfr thomasfr commented Jul 10, 2017

Is it possible to exclude some node modules from beeing hoisted to the root folder? Like the lerna no-hoist flag did before?

But besides that. Awesome work. Cuts the installation time by half with yarn workspaces for us!! From ~2minutes down to 58secs

@jquense
Copy link

@jquense jquense commented Jul 10, 2017

I don't think so regarding no-hoist since that's how yarn workspaces work and lerna is defering to it.

@jquense
Copy link

@jquense jquense commented Jul 11, 2017

So i've done some more testing. It appears like yarns install with workspaces doesn't different things from lerna bootstrap. The big things i've notice are:

  • yarn doesn't link local bins
  • yarn doesn't run local prepublish/prepare scripts in linked packages, which is required for build steps

I've tried a bunch of different configs and setups with the same results. As a replacement for bootstrap it doesn't do all the things :)

@bestander
Copy link
Contributor Author

@bestander bestander commented Jul 11, 2017

Thanks, @jquense.
Feel free to raise issues in https://github.com/yarnpkg/yarn/issues with repro steps and ping me.

@thomasfr
Copy link

@thomasfr thomasfr commented Jul 12, 2017

Just fyi i created an issue for --no-hoist at yarn: yarnpkg/yarn#3882

@Hypnosphi
Copy link
Contributor

@Hypnosphi Hypnosphi commented Aug 7, 2017

So is lerna bootstrap --use-workspaces a "shortcut" to yarn command?

@lock
Copy link

@lock lock bot commented Dec 27, 2018

This thread has been automatically locked because there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock bot locked as resolved and limited conversation to collaborators Dec 27, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

8 participants