lerna-lite publish & pnpm: very slow ?

[wassim-ben-amor]

Hello,
After migrating a monorepo from yarn to pnpm, I have noticed that using @lerna-lite/cli lerna publish command is taking too much time compared to yarn before. It's taking more then 1 hour now compared to a maximum of 2 minutes before the migration.

After debugging, I have found that the pack-directory step is the problematic one and exactly on const tree = await arborist.loadActual();.
With more debugging, it's taking too much time when searching for missing edged on @npmcli/arborist package.

I think it's because of pnpm logic doing symlinks between node_modules. My guess is that having ^workspace protocol will result a full search in the repo when a lot of packages are linked.

My question is: do I really need this step: const tree = await arborist.loadActual(); ? I use @lerna-lite/cli lerna publish only to pack a ready to publish folders having already their dist folder (already built).
Thank you,

[ghiscoding]

I don't have that problem on my side and I'm using pnpm everywhere with Lerna-Lite and my project has 18 packages.

My question is: do I really need this step: const tree = await arborist.loadActual(); ? I use @lerna-lite/cli lerna publish only to pack a ready to publish folders having already their dist folder (already built).
Thank you,

I have no idea, I didn't create Lerna, I simply extended it into Lerna-Lite and keep updating its dependencies. I did add the missing workspace: protocol but I don't remember touching any of the arborist stuff.

[ghiscoding]

I think the only reason I've done that was to upgrade to newest versions (which I always try to do in Lerna-Lite), I'd be more than happy to receive a PR to fix it if you know how. Note that I'm pretty sure that even though Arborist wasn't directly linked as a dependency, it was an indirect dependency and is the consequence of upgrading to newer releases of Pacote and others as explained in the PR that you found. Rolling back or downgrading the dependencies doesn't seem like a good solution either

[wassim-ben-amor]

Thank you for clarifying.
Indeed, after checking more the different changelogs, I have found that npm-packlist introduces many breaking changes between v5.1.1 - version used by lerna and v7.0.4 or latest v8 used by lerna-lite.
After just doing an override of npm-packlist to v5.1.1 and removing the 2 lines of Arborist, I don't have the issue of slowness anymore.
I understand that downgrading npm-packlist doesn't seem like a good solution ..
I didn't find an explanation why it's not bumped yet on lerna side.
The last bump was the v5.1.1.

In my opinion, the fix is to add the possibility to not run the findMissingEdges part, which is the part that is causing problems. It's by simply doing:
const tree = await arborist.loadActual({ignoreMissing: true});
I am not 100% sure about the impact of it. That's why I opened a thread on npm-packlist to understand the impact.
Do you think that this can be possible ? (Maybe just having a flag to enable it on publish command and keep the default usage like it is).

[ghiscoding]

Thanks for the great troubleshooting and findings, at this point you seem to have a much greater understanding of the issue and I'm in acceptance with your findings especially if do see the problem going away. Before going with a downgrade though, I wanted to at least open an issue on the external lib and it looks like you did... so I'd be ok to receive a PR with this fix, that is even if it includes a downgrade (as long as it's documented somewhere, I'm good with it). However, to avoid upgrading back in the future and reintroducing the issue, please make sure to add an ignore rule in the Renovate config to ignore the dependencies causing the problem. Again thanks a lot for all of this troubleshooting.

[wassim-ben-amor]

Hello,
of corse ! I have created the PR
Update: The fix is released on v3.3.3