lerna bootstrap tries to build dependency in the wrong order, so dependent build fails

[trusktr]

It appears that lerna links a local project A into a local project B, but does not run the build step of project A before it tries to build project B, so project B's build fails because it can not resolve the modules from project A because it does not have its dist folder.

Expected Behavior

Lerna should build A before B.

Current Behavior

Lerna doesn't, and so B's build fails.

Possible Solution

Steps to Reproduce (for bugs)

git clone --recursive git@github.com:infamous/umbrella.git
cd umbrella
git checkout lerna-issue-2453
lerna bootstrap

lerna.json

{
        "packages": ["apps/*", "packages/*"],
        "version": "independent"
}

lerna-debug.log

None.

Context

I was trying out lerna for the first time in an existing repo that has multiple projects. Each project is a git submodule.

Could the fact that the projects I have in apps/* and packages/* are git submodules be throwing of Lerna's build algo?

Your Environment

Executable	Version
lerna --version	3.20.2
npm --version	6.13.6
yarn --version	N/A
node --version	13.8.0

OS	Version
Manjaro Linux	rolling release

[trusktr]

I updated the issue with a reproduction.

[trusktr]

The cause of this was that I was using prepare scripts in some projects. After switching those scripts to prepack the issues went away. Then I can run lerna run build to perform the builds instead of them happening during install, and prepack will still be useful when installing any packages from git in other projects, which is what I was using prepare for in the first place.

EDIT: Warning: Switching from prepare to prepack is a breaking change. Installing a package from git will no longer work without prepare, and prepack scripts do not have access to dev dependencies, for example.

[trusktr]

Closing as invalid.

Tip: make sure to put build scripts in prepack instead of prepare, so that the bootstrap process doesn't also build your projects at the same time.

EDIT: This issue is still valid.

[trusktr]

This is still a problem, because with NPM, prepack scripts do not have access to dev dependencies, while prepare scripts do. Therefore, for things to work we need to put build steps in prepare scripts so that, for example, installing a package from git will build correctly with dev dependencies present.

But this conflicts with the requirement of needing to use prepack as above.

[FallenWarrior2k]

To add to this, due to yarnpkg/yarn#5047, prepack doesn't run at all under yarn when installing from Git. Similarly to the npm issue described in the previous comment, this necessitates putting build scripts in prepare instead of prepack.

[GiancarlosIO]

So, is there a workaround for this issue?
I'm manually installing and building the "primitive" packages first and then running yarn lerna bootstrap to build all packages without problems. 😢

[robbie-unlikelyai]

I'm also doing the above - is there a better way to handle this?

[benbrott]

Also running into this - if using the dependency tree to determine the package build order would be too much work, respecting the order of the packages array in lerna.json would be a nice improvement

[ShishKabab]

This is what I've been using successfully for the last years to run prepare scripts in the right order:
https://gist.github.com/ShishKabab/51ae7b6aab9292f865de0ee238c26052

[Noyabronok]

I solved the issue by adding a prepare script to the package that was not being prioritized. So the lesson is, make sure that if one package has a prepare script, packages that it depends on have a prepare script as well.
p.s. using lerna 6, haven't tried it with earlier versions

[JamesHenry]

Hi Folks 👋

Please take a look at our published roadmap for Lerna v7 here: #3410

One of the key items covered at length on there (please do read it for full context) is that now that we find ourselves in late 2022, it no longer makes sense for lerna to supplement package management concerns (such as installation, boostrapping, linking etc) which are covered reliably for monorepo workspaces by the three main package managers: npm, yarn and pnpm. lerna bootstrap et al were developed in a completely different era of the JavaScript ecosystem.

If you have any specific concerns please do join in on that discussion, and provide as much context as possible.

Many thanks 🙏