feat: Support the `workspace:` protocol #2450

nicolo-ribaudo · 2020-02-14T01:46:31Z

Description

Yarn 2 and pnpm (I know that it isn't supported, but saw that you are working on it) support the workspace: protocol for monorepo package versions.

When used, it explicitly marks a dependency to assert that it is never downloaded from the registry. Both the package managers allow making the workspace: protocol required: if it isn't specified, then the dependency is downloaded from the registry even if a local package would match the requested version.
yarn docs, pnpm docs.

This PR contains two different commits: the first one makes lerna not fail when someone uses the workspace protocol (which is what made me open this PR).

The second one implements a --local-dependencies option which makes lerna actually use the workspace protocol. I'm not sure that this is needed since people would probably directly enable the useWorkspaces option and delegate this task to yarn, but I added the commit just in case you could be interested in it. I would have no problems with dropping it.

Please don't review them as a single unit 🙏

Motivation and Context

I didn't open an issue about it, but this is currently blocking Babel from upgrading to yarn 2 (babel/babel#11142).

How Has This Been Tested?

I added tests to the affected packages.

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

My code follows the code style of this project.
My change requires a change to the documentation.
I have updated the documentation accordingly. (I will do it depending on what gets accepted)
I have read the CONTRIBUTING document.
I have added tests to cover my changes.
All new and existing tests passed.

evocateur

I am super-pumped for this functionality, I'm so sorry I ignored it for so long!

My chief concern is that we're not covering the version/publish scenario, which is pretty important for this protocol to be "fully" supported by Lerna.

evocateur · 2020-05-23T21:37:55Z

core/package/__tests__/core-package.test.js

@@ -273,6 +274,37 @@ describe("Package", () => {
      );
    });
  });
+
+  describe(".updateLocalDependency()", () => {
+    it("works with workspace: protocols", () => {


I didn't have any direct tests of this method here because it is repeatedly tested in the publish and version tests. This is good to have, though.

evocateur · 2020-05-23T21:40:39Z

core/package/__tests__/core-package.test.js

+      expect(pkg.toJSON()).toMatchInlineSnapshot(`
+        Object {
+          "dependencies": Object {
+            "a": "workspace:^2.0.0",


One wrinkle, however, is that sometimes pkg.updateLocalDependency() is intended to overwrite any protocol (such as file:) with the resolved version. Existing code has cheesy conditionals that exclude the call when the resolved.type === "directory", so we're gonna have to make this method smarter somehow (or split the intent into two methods...)

evocateur · 2020-05-23T21:42:13Z

core/package/index.js

@@ -209,6 +209,9 @@ class Package {
    if (resolved.registry || resolved.type === "directory") {
      // a version (1.2.3) OR range (^1.2.3) OR directory (file:../foo-pkg)
      depCollection[depName] = `${savePrefix}${depVersion}`;
+      if (resolved.explicitWorkspace) {
+        depCollection[depName] = `workspace:${depCollection[depName]}`;


As I mentioned in the newly-added test for this method, this isn't always the case (thinking specifically of prepublish resolution of ${protocol}: versions)...

evocateur · 2020-05-23T21:45:49Z

utils/query-graph/query-graph.js

   * @param {Boolean} [opts.rejectCycles] Whether or not to reject cycles
   * @constructor
   */
  constructor(packages, opts) {
    const options = QueryGraphConfig(opts);

    // Create dependency graph
-    this.graph = new PackageGraph(packages, options.graphType);
+    this.graph = new PackageGraph(packages, options.graphType, options.localDependencies);


run-topologically also needs this option passed through, as unfortunately most of its callsites are passing custom config, not the whole command-resolved options object...

evocateur · 2020-05-23T21:57:27Z

commands/link/__tests__/__fixtures__/explicit-local/lerna.json

+{
+  "commands": {
+    "link": {
+      "localDependencies": "explicit"


We're missing CLI option config for --local-dependencies, as well as README documentation. It's especially burdensome for an option like this because it is such a broad thing. Perhaps instead of a flag, we need to centralize the "am i using a custom protocol" logic better?

lerna/commands/publish/index.js

Lines 519 to 522 in 448f2ae

// resolve relative file: links to their actual version range

const updatesWithLocalLinks = this.updates.filter(node =>

Array.from(node.localDependencies.values()).some(resolved => resolved.type === "directory")

);

lerna/commands/version/index.js

Lines 522 to 524 in 448f2ae

if (depVersion && resolved.type !== "directory") {

// don't overwrite local file: specifiers, they only change during publish

pkg.updateLocalDependency(resolved, depVersion, this.savePrefix);

lerna/commands/bootstrap/index.js

Lines 240 to 248 in 448f2ae

rootHasLocalFileDependencies() {

const rootDependencies = Object.assign({}, this.project.manifest.dependencies);

return Object.keys(rootDependencies).some(

name =>

this.targetGraph.has(name) &&

npa.resolve(name, rootDependencies[name], this.project.rootPath).type === "directory"

);

}

After all, in an ideal world, the workspace: protocol would "just work", similar to how file:/link: is handled today.

ChiefORZ · 2020-08-18T07:16:55Z

why isn't this feature getting any more attention? we are waiting hard for this one...

nicolo-ribaudo · 2020-08-18T07:57:49Z

@ChiefORZ I won't have time to work on this PR for 1-2 months. If you want to take my PR and address the review comments it would be appreciated 🙏

arcanis · 2020-09-25T22:33:00Z

@evocateur if by any chance you saw a way to land this PR in any way I'd appreciate - it's currently difficult for third-party projects to use the workspace: protocol without having to drop Lerna, which makes the adoption ever so slightly harder for us 🙁

Carl-Foster · 2020-10-13T23:18:15Z

I'm happy to take over this PR and address the comments.

Carl-Foster · 2020-10-28T01:17:14Z

I've put some time into this PR and trying to add tests to address the comments however I'm finding it hard to navigate. I feel like lerna needs to detect if yarn berry is being used and transparently pass through most of the logic down to the @yarnpkg/core package. This would require a fairly significant rework of the implementation and I don't know enough about the architecture to make that decision.

okikio · 2020-12-20T22:14:31Z

Can someone please merge this pull request, I have been waiting for this feature for literally 9 months.... It looks like all the details concerning this have been ironed out, so, please merge it.

nicolo-ribaudo · 2020-12-20T22:31:40Z

@okikio I don't have time to work on this PR anymore, but @evocateur already reviewed it (#2450 (review)).
If you want to cherry-pick my commits, handle @evocateur's review and open a new PR, that would probably help a lot moving this forward.

okikio · 2020-12-20T22:55:48Z

@okikio I don't have time to work on this PR anymore, but @evocateur already reviewed it (#2450 (review)).
If you want to cherry-pick my commits, handle @evocateur's review and open a new PR, that would probably help a lot moving this forward.

I thought the PR was finalized and just not yet merged. Ok, I'll open a new PR but how do I get @evocateur to review.

okikio · 2020-12-26T22:18:32Z

@nicolo-ribaudo I have gone through your commits, I don't think they would adequately accomplish the task required. When discussing the workspace protocol, most devs using lerna would most likely want it to act like if the workspaces protocol wasn't used. By default, lerna can't set packages to use the workspace protocol, so why bother building it into lerna, why not just let users continue with the tools they were using prior, so, for example, yarn and pnpm, if devs are already using these tools it would be less work to implement, all that would be required would be to just add an option to ignore the workspace protocol and let either yarn or pnpm handle the publishing, so, lerna would handle versioning and changelog generation. What do you think @evocateur does this sound good?

matzeeable · 2021-05-06T13:00:43Z

core/package-graph/index.js

+        const explicitWorkspace = /^workspace:/.test(spec);
+        if (explicitWorkspace) {
+          spec = spec.replace(/^workspace:/, "");
+        }


Just a sidenote: The same should be done for the :patch protocol.

That one can't really be replaced

evocateur · 2021-06-03T16:20:58Z

I've rebased this onto the latest in main: https://github.com/lerna/lerna/compare/test/workspace-protocol

I'm going to do some local validation, maybe write some more tests. We're still lacking option config for --local-dependencies (and documentation), so this still isn't ready to go as-is.

I can't promise any timeline, but this is increasing in priority for me.

brianespinosa · 2021-08-12T20:52:35Z

For anyone else still needing this, it seems like the right approach might just be implementing publishing features yourself on top of yarn as they are doing in the above referenced PR from Storybook.

ghiscoding · 2022-05-11T01:15:14Z

hey everyone happy to share that workspace: protocol is now supported in Lerna-Lite with this new release >=1.3.0.
Give it a try and happy coding 🎉

fahslaj · 2022-12-29T23:55:08Z

To all it may concern, this PR is being closed for clarity since Lerna already supports the workspace: protocol. It was added a while ago as part of the pnpm support included in Lerna 6.

Lerna recognizes the workspace: prefix in Yarn 1, Yarn 2+ (with node_modules linker) and pnpm workspaces and correctly converts them to the appropriate dependency spec when publishing.

Thank you to everyone for your participation in the Lerna community!

binaryunary · 2023-01-02T10:18:45Z

Does this mean #2564 can be closed?

JamesHenry · 2023-01-05T19:02:08Z

Thanks for the reminder @binaryunary, yes!

cumt-robin · 2023-09-11T03:42:59Z

Couldn't find any versions for "@jview/utils" that matches "workspace:*"

I dont know why...

it's lerna v7.

JamesHenry · 2023-09-11T10:52:38Z

@cumt-robin You are using yarn v1/classic which never supported the workspace protocol. If you want to use it you migrate to latest yarn v3+

cumt-robin · 2023-09-11T10:53:10Z

您的邮件我已收到！我会在第一时间内回复您！非常感谢！

nicolo-ribaudo added 2 commits February 14, 2020 02:36

feat: Trasparently support the workspace: protocol

12b50a3

feat: Respect the workspace: protocol

451a199

bajtos mentioned this pull request Mar 10, 2020

feat(docs): add 20202Q2 roadmap loopbackio/loopback-next#4635

Merged

7 tasks

7rulnik mentioned this pull request Apr 18, 2020

build(*): change project setup alfa-laboratory/core-components#87

Merged

evocateur reviewed May 23, 2020

View reviewed changes

7rulnik mentioned this pull request May 26, 2020

Lerna does not support yarn 2 workspace:* url type #2564

Closed

chentsulin mentioned this pull request Jun 28, 2020

yarn v3 bottenderjs/messaging-apis#601

Closed

lboecker mentioned this pull request Sep 6, 2020

Use Yarn's workspace protocol lboecker/eslint-configs#3

Open

morlay mentioned this pull request Oct 12, 2020

switch to use workspace:* querycap/webappkit#28

Closed

evocateur changed the base branch from master to main November 8, 2020 00:40

matzeeable reviewed May 6, 2021

View reviewed changes

hydrosquall mentioned this pull request Jul 16, 2021

Yarn2: Use workspace:* versions storybookjs/storybook#15413

Closed

merceyz mentioned this pull request Aug 22, 2021

Migrate yarn to berry v3 ianstormtaylor/slate#4417

Merged

5 tasks

ja0nz mentioned this pull request Nov 14, 2021

Feature/yarn3 - migrate to the new version of yarn thi-ng/umbrella#326

Merged

ghiscoding mentioned this pull request Apr 8, 2022

Support pnpm workspaces lerna-lite/lerna-lite#100

Closed

This was referenced May 2, 2022

[Feature] Changelog support in version (?) plugin yarnpkg/berry#1510

Open

feat(core): support version/publish workspace: protocol lerna-lite/lerna-lite#148

Merged

ghiscoding mentioned this pull request Jul 21, 2022

Not picking up workspaces from lerna.json vercel/turbo#688

Closed

charlie-yao referenced this pull request in charlie-yao/react-aria-widgets Oct 5, 2022

remove lerna

9558983

charlie-yao mentioned this pull request Oct 5, 2022

Improve versioning/release/etc. process charlie-yao/react-aria-widgets#87

Open

fahslaj closed this Dec 29, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Support the `workspace:` protocol #2450

feat: Support the `workspace:` protocol #2450

nicolo-ribaudo commented Feb 14, 2020

evocateur left a comment

evocateur May 23, 2020

evocateur May 23, 2020

evocateur May 23, 2020

evocateur May 23, 2020

evocateur May 23, 2020

ChiefORZ commented Aug 18, 2020

nicolo-ribaudo commented Aug 18, 2020

arcanis commented Sep 25, 2020

Carl-Foster commented Oct 13, 2020

Carl-Foster commented Oct 28, 2020

okikio commented Dec 20, 2020 •

edited

Loading

nicolo-ribaudo commented Dec 20, 2020

okikio commented Dec 20, 2020

okikio commented Dec 26, 2020

matzeeable May 6, 2021

merceyz May 6, 2021

evocateur commented Jun 3, 2021

brianespinosa commented Aug 12, 2021

ghiscoding commented May 11, 2022 •

edited

Loading

fahslaj commented Dec 29, 2022

binaryunary commented Jan 2, 2023

JamesHenry commented Jan 5, 2023

cumt-robin commented Sep 11, 2023

JamesHenry commented Sep 11, 2023

cumt-robin commented Sep 11, 2023 via email

	// resolve relative file: links to their actual version range
	const updatesWithLocalLinks = this.updates.filter(node =>
	Array.from(node.localDependencies.values()).some(resolved => resolved.type === "directory")
	);

	if (depVersion && resolved.type !== "directory") {
	// don't overwrite local file: specifiers, they only change during publish
	pkg.updateLocalDependency(resolved, depVersion, this.savePrefix);

	rootHasLocalFileDependencies() {
	const rootDependencies = Object.assign({}, this.project.manifest.dependencies);

	return Object.keys(rootDependencies).some(
	name =>
	this.targetGraph.has(name) &&
	npa.resolve(name, rootDependencies[name], this.project.rootPath).type === "directory"
	);
	}

feat: Support the workspace: protocol #2450

feat: Support the workspace: protocol #2450

Conversation

nicolo-ribaudo commented Feb 14, 2020

Description

Motivation and Context

How Has This Been Tested?

Types of changes

Checklist:

evocateur left a comment

Choose a reason for hiding this comment

evocateur May 23, 2020

Choose a reason for hiding this comment

evocateur May 23, 2020

Choose a reason for hiding this comment

evocateur May 23, 2020

Choose a reason for hiding this comment

evocateur May 23, 2020

Choose a reason for hiding this comment

evocateur May 23, 2020

Choose a reason for hiding this comment

ChiefORZ commented Aug 18, 2020

nicolo-ribaudo commented Aug 18, 2020

arcanis commented Sep 25, 2020

Carl-Foster commented Oct 13, 2020

Carl-Foster commented Oct 28, 2020

okikio commented Dec 20, 2020 • edited Loading

nicolo-ribaudo commented Dec 20, 2020

okikio commented Dec 20, 2020

okikio commented Dec 26, 2020

matzeeable May 6, 2021

Choose a reason for hiding this comment

merceyz May 6, 2021

Choose a reason for hiding this comment

evocateur commented Jun 3, 2021

brianespinosa commented Aug 12, 2021

ghiscoding commented May 11, 2022 • edited Loading

fahslaj commented Dec 29, 2022

binaryunary commented Jan 2, 2023

JamesHenry commented Jan 5, 2023

cumt-robin commented Sep 11, 2023

JamesHenry commented Sep 11, 2023

cumt-robin commented Sep 11, 2023 via email

feat: Support the `workspace:` protocol #2450

feat: Support the `workspace:` protocol #2450

okikio commented Dec 20, 2020 •

edited

Loading

ghiscoding commented May 11, 2022 •

edited

Loading