[question] Using yarn commands with non-published lerna dependencies

[timoxley]

Is it possible to use npm/yarn with lerna if you've got packages which are not published to any registry?

e.g.

• ./packages/A depends on B, found in ./packages/B.
• B exists solely for code sharing, is just an internal package.
• edit: No intention to publish either A nor B to any registry.

Currently, if you run any yarn command in ./packages/A e.g. yarn remove express the command will fail because yarn looks for B and B isn't published in any registry, thus error Couldn't find package "B" on the "npm" registry.

Is there a way around this? A way to specify lerna dependencies outside of "dependencies" perhaps? Maybe I missed something in the instructions. Perhaps this use-case just isn't supported.

edit: this seems like a yarn shortcoming really.

At the moment my workaround is terrible; I'm manually removing all lerna managed dependencies from package.json just prior to running commands like yarn add, then adding them back afterwards. This is truly not great.

[timoxley]

Ideally lerna bootstrap would know how to symlink file dependencies:

{
  "name": "A",
  "dependencies": {
    "B": "file:../B"
  }
}

Update: The above file: method doesn't seem to break npm, yarn or lerna, though none of these tools will symlink the local package for you. My current workaround is to use linklocal to symlink file: dependencies:

lerna exec linklocal

[kschzt]

FYI this seems to work correctly with npm.

[evocateur]

If B is never published (a result of adding "private": true to its package.json), it certainly should never be specified in A's dependencies, as any consumer of A will never install correctly.

Adding B to A's devDependencies should be fine, and it is not necessary to specify the file: url to get the symlinking for free (the version just has to satisfy version in the root lerna.json).

[timoxley]

FYI this seems to work correctly with npm.

@kschzt sorry can you elaborate?

edit: ahh I think I know what you mean… npm currently doesn't process dependencies that are already satisfied, so it doesn't bother checking the registry for new versions of the unpublished package, thus doesn't error.

any consumer of A will never install correctly.

@evocateur Note there are no consumers of any of these packages other than themselves, they're not shared libraries, they're all components of a single app and internal to the monorepo.

[timoxley]

Also relevant to this issue is this npm/yarn RFC for changing the file: semantics and adding link: dependency specifiers:

• RFC: file: specifier changes npm/npm#15900
• :link dependencies yarnpkg/rfcs#34

[kylecordes]

I've found the answer to the initial question to be 100% yes - I'm using it that way in the following.

https://github.com/OasisDigital/many-to-many-angular

I use neither file: nor linklocal. I use yarn, although I get the impression that yarn support is still somewhat early.

[ravicious]

@kylecordes: I don't quite understand how you managed to make yarn with lerna work with unpublished packages. 🤔 If I clone your repo, run lerna bootstrap, go to applications/admin and run yarn add react, I get the following error:

error An unexpected error occurred: "https://registry.yarnpkg.com/@oasisdigital%2femployee-search: Not found".

This error, I believe, is the subject of the problem @timoxley and me have: that it's not possible to manipulate package.json/yarn.lock through yarn if you have a lerna-managed, unpublished packages in your package.json.

[timoxley]

@ravicious tried latest version of lerna?

[ravicious]

Yeah, I'm on 2.0.0-rc.4. How should the current workflow look like? Is linklocal still required?

I updated both yarn and then learn in @kylecordes's project to the newest version and I still get this error.

[jvivs]

If it helps at all, I'm seeing something very very similar with a private registry (artifactory) lerna@2.0.0-rc.4, and either yarn@0.24.3 or npm@3.10.8.

Turns out this bug is preventing us from using --hoist too (and other yarn scripty things).

[timoxley]

@ravicious

I think yarn has recently stopped hitting the registry for symlinked packages, so as long as you run linklocal first to install the symlinks for file:../ dependencies then lerna bootstrap & other lerna/yarn commands appear to work just fine.

My bootstrap script looks something like this:

{
  "scripts": {
    "bootstrap" "lerna init && lerna exec -- linklocal && lerna bootstrap --mutex=network"
    …
  }
}

[timoxley]

@jvivs I think that's more related to yarnpkg/yarn#2541

[ravicious]

@timoxley: Thanks, I that the combination of yarn, linklocal & lerna indeed works. Though passing --mutex=network doesn't work for me – the bootstrap command fails with Package "@foo/bar@" doesn't have a "version". (~~~maybe it's related to #789~~~ it's most probably related to #683). I have to set concurrency to 1 to make bootstrap work.

Initially I was confused by the mutex error and I wasn't sure if this combination of packages still works.

[jvivs]

@timoxley linklocal does look like a solid fix for file:...-driven dependencies.

The case I'm running into is slightly different, and quite possibly more of a problem for greenfield projects than a project w/ existing published dependencies.

I would love to be wrong here, but linklocal doesn't look like it solves the problem of yarn always attempting to fetch versioned dependencies from a registry that are local to the repo, but haven't yet been published.

Edit: yarnpkg/yarn#2541 seems similar, but I don't think it's the exact issue we're running into yet... the issue we're having occurs before a package is published in the first place (i.e.: creating a new package in a lerna-managed repo + doing anything w/ --hoist). Also sounds like I need to create an issue for it 🙊

[kylecordes]

@ravicious and others - yes, it is currently irritating to use in this way. The linklocal and mutex things help, though sometimes I just edit a package.json then re-bootstrap.

Here is the heart of the matter: regardless of whether working with non-published packages, typically when using Lerna one does not want yarn/npm to ignore the local deps and go look at the web instead - because when developing, the local packages might be arbitrarily different from the published ones.

Ultimately it may require a deeper integration between Lerna and Yarn or NPM to make all of this "just work". As things proceed, I expect that improvements for working with published packages will end up also improving things for unpublished packages.

[MiguelMadero]

I think #425 may help with this since yarn can work with relative paths and lerna would still symlink them.

[marionebl]

This is solved with the yarn workspaces integration and addition of lerna add:

• yarn workspaces: Support Yarn workspaces to replace bootstrap command #899
• lerna add feat: implement lerna add <pkg>[@version] [--dev] #1069

[lock]

This thread has been automatically locked because there has not been any recent activity after it was closed. Please open a new issue for related bugs.