workspaces: first phase: install node_modules aggregated from all workspaces

[bestander]

Workspaces phase 1: aggregated node_modules according to RFC yarnpkg/rfcs#60.

This change aggregates dependencies from sub projects

For example, if the structure of the source code is following

| jest/
| ---- package.json
| ---- packages/
| -------- babel-jest/
| ------------ packjage.json
| -------- babel-preset-jest/
| ------------ packjage.json
...

Top level package.json is like

{
  "private": true,
  "name": "jest",
  "devDependencies": {
    "ansi-regex": "^2.0.0"
  },
  "workspaces": [
    "packages/*"
  ]
}

babel-jest

{
  "name": "babel-jest",
  "description": "Jest plugin to use babel for transformation.",
  "version": "19.0.0",
  "repository": {
    "type": "git",
    "url": "https://github.com/facebook/jest.git"
  },
  "license": "BSD-3-Clause",
  "main": "build/index.js",
  "dependencies": {
    "babel-core": "^6.0.0",
    "babel-plugin-istanbul": "^4.0.0",
    "babel-preset-jest": "^19.0.0"
  }
}

babel-preset-jest

{
  "name": "babel-preset-jest",
  "version": "19.0.0",
  "repository": {
    "type": "git",
    "url": "https://github.com/facebook/jest.git"
  },
  "license": "BSD-3-Clause",
  "main": "index.js",
  "dependencies": {
    "babel-plugin-jest-hoist": "^19.0.0"
  }
}

If workspaces is enabled and yarn install is run at the root level of jest Yarn would install dependencies as if the package.json contained all the dependencies of all the package.json files combined, i.e.

{
  "devDependencies": {
    "ansi-regex": "^2.0.0",
  },
  "dependencies": {
    "babel-core": "^6.0.0",
    "babel-plugin-istanbul": "^4.0.0",
    "babel-preset-jest": "^19.0.0",
    "babel-plugin-jest-hoist": "^19.0.0"
  }
}

A single yarn.lock will be created at the root level and no node_modules and yarn.lock files will be generated for any of the workspaces.

Note: If there are different versions of the same package between workspaces or a workspace and the root then install will fail with error Dependency $0 has different versions in $1 and $2.
In a next phase conflicts will be resolved similar to package-hoister that works when node_modules folder is being built.

To set up.

Set project's private filed to true (so that people would not accidentally publish projects with non standard workspaces field)

{
  "private": true
}

Run

yarn config set workspaces-experimental true

[arcanis]

Maybe two extra modes could be interesting:

• Merge dependency requirements (so that the range would become ~1.0.0 ~1.0.3, for example)
• When conflict arise, prevent hoisting the conflicting modules (so that they end up being installed into their respective node_modules folders)

[arcanis]

When conflict arise, prevent hoisting the conflicting modules (so that they end up being installed into their respective node_modules folders)

Problem being that in such a case, we would have to run yarn install recursively into the workspace directories

[bestander]

Yeah, we'll do it in a later phase.

[bestander]

ping @arcanis for tests

[bestander]

This PR will be merged at the same time with #3365 and then we'll ship the feature to for testing behind the flag.

Next step would be package hoisting logic at the workspaces level similar to node_modules hoisting Yarn does.
That would require a new structure in project manifest to be passed through to the linking phase.

[arcanis]

Shouldn't this code also check if the cwd is inside a project and, if it is, run the install from the project root instead of the workspace (or at least prevent the install)? Otherwise, if people run yarn install from one of their workspace, they will create a node_modules that will shortcut the one created in the project root.

[arcanis]

I've added a few utility methods to the Config class to support workspaces (for example https://github.com/yarnpkg/yarn/pull/3365/files#diff-9d9a6cd82f41984872a66a3ab0d440c4R555), it would make sense to update the code to use them once we've merged this PR, what do you think?

[bestander]

does it support glob?
I think we can discuss that in the following PR

[bestander]

Shouldn't this code also check if the cwd is inside a project and, if it is, run the install from the project root instead of the workspace (or at least prevent the install)?

Yeah, it makes sense.
Let's follow up with a check for this, I think it is tightly related to your PR anyway

[arcanis]

👍 Good to merge, then?

[bestander]

Thanks