[Case Study] Perfomance of workspaces foreach

[7rulnik]

this issue not related to other projects but yarn itself

Describe the goal of the investigation

I noticed that when I'm running even very simple scripts like pwd via yarn workspaces foreach it takes a very long time. Is it expected behavior? Maybe there are some opportunities to optimize it?

Investigation report

For example, in a workspace of 100 packages, it takes almost 2min. Run via lerna takes a bit less, maybe about -15s. But I think it's still not great numbers.

[yarnbot]

Hi! 👋

It seems like this issue as been marked as probably resolved, or missing important information blocking its progression. As a result, it'll be closed in a few days unless a maintainer explicitly vouches for it.

[7rulnik]

Seems it's bothering only me :)

BTW in my case, it was a problem with TS and some other build tools so I just replaced tasks in packages with one in the root of the workspace.

[christophehurpeau]

I had the same issue (but with just a few packages), I reverted to using lerna run

[arcanis]

Seems it's bothering only me :)

It's more that there isn't an identified action item, so there isn't much to say 🙂

[7rulnik]

@christophehurpeau I got kinda same performance with lerna

[yarnbot]

Hi! 👋

It seems like this issue as been marked as probably resolved, or missing important information blocking its progression. As a result, it'll be closed in a few days unless a maintainer explicitly vouches for it.

[merceyz]

I have some performance optimizations in mind that might help, i'll get a PR up eventually

[devshorts]

Yeah, I'm also in the same boat. It seems that executing even a basic echo on a single workspace item is very slow

$ yarn workspaces foreach -v -p -j 4 --include '@app/service-types' run echo-test
➤ YN0000: [@app/service-types]: echo
➤ YN0000: Done in 2s 130ms

Where all this does for the script section is "echo echo"

Running this on an entire monorepo with 20 packages takes 20-30 seconds

[devshorts]

Did some more research, seems this is not really related to foreach and more just the invocation of yarn for each workspace. I attached a debugger to the workspace tool and it itself doesn't do much. So maybe with the improvements to yarn run startup time this will get better?

[merceyz]

Performance of running scripts was improved in #2564, #2576, #2577, and #2582 so it should be a lot better now, not perfect, but better. Related issues #2560 and #2575.

I'll close this as it's more related to running scripts in general which can be tracked in #2575

[christophehurpeau]

I tested with latest version of yarn from sources and I can confirm it's faster ! Thanks !