Strip in-workspace devDependencies when pruning

[marvinroger]

Describe the feature you'd like to request

Hi,

First of all, thanks for this awesome project, it's really powerful!

We're currently using the great turbo prune feature, and we noticed a potential optimization.
Imagine a project app having two "in-workspace" dependencies:

• lib-config as a dependency
• tooling as a devDependency

Currently, running turbo prune --scope=app will generate a directory out/ with app, lib-config and tooling.
In our case, we run turbo prune in our CI after all packages have been built. Therefore, we don't need the devDependencies in our out/ directory.

Ideally, our pruned out/ directory would contain only app and lib-config, which are the only packages needed at runtime.

Describe the solution you'd like

Having an optional --production flag that would ignore the devDependencies would be great.

Describe alternatives you've considered

I don't think there are any alternatives, except keeping the devDependencies, which is frankly not a big deal. It's just less optimal... But, isn't Turborepo all about optimization? 😉

[Ghirigoro]

This not always just a question of optimizing file size. I have a situation with a backend project that is strictly node, but uses some browser-based utilities when testing. In order to build those browser dependencies that get pulled in I also have to manually pull in the resources required to build those dependencies (for example I have separate base tsconfig.json files for node and for browser at the root of the monorepo). It's not the end of the world but you start to quickly lose the point of pruning if you're pulling in half your monorepo and littering your docker images with resources that are only used to workaround this issue.

P.S. - I'm realizing my tone probably comes off as more stern than I mean to. I agree it's not a huge deal but it would make Turborepo even more awesome if it wasn't the case :)

[azertyalex]

@joemckenney care to share your rm-dev-deps.mjs script referenced in issue #1488 ?
I'm having the same problem as you, 1GB docker image

[joemckenney]

@joemckenney care to share your rm-dev-deps.mjs script referenced in issue #1488 ? I'm having the same problem as you, 1GB docker image

@azertyalex Here's a gist https://gist.github.com/joemckenney/fd55f02ab0bf905283423aadeeca80ca

The script uses a package we built called @dopt/wutils - which assumes you use pnpm. It's open source but not published to any registry - you can find it here https://github.com/dopt/odopt/tree/main/packages/%40dopt/wutils and take inspiration from it or copy pieces to recreate the functionality.

Some additional context - this problem was solved by our shift from yarn to pnpm and pnpm's deploy command.

[azertyalex]

okay, thanks! I stumbled upon pnpm myself in the last hour, that seems the way to go

[gajus]

Tagging core team member to make sure this gets seen since it is a pretty major issue @gsoltis

Annoyingly, pnpm v8 introduced a regression where it broke pnpm deploy so there isn't even a fallback option.

Related pnpm issues:

• pnpm deploy --prod is copying extraneous dependencies pnpm/pnpm#6437 (comment)
• Deploy should output a lock file pnpm/pnpm#6540

[mehulkar]

A --production flag does seem like a good idea.

cc @chris-olszewski, how hard would this be to do? I got a little lost reading the prune code with the transitive_closure stuff, so I wasn't able to get an easy answer

[chris-olszewski]

This is something we do want to add, but a few things need to happen first:

• We need to change our package graph to contain what dependency types in the edges. This is a fairly straight forward change, but it will be pretty invasive throughout the codebase.
• Come to agreement on flag naming
  Once those are done, this feature will just be using the correct package graph in prune.

[mehulmpt]

Faced this today where one of our packages required some devDependencies for building it but turbo prune is pulling them inside production build the way our pipeline is built. We have to write a workaround for this but it would be great if it was natively available.

[wrporter]

Just came across this problem when dockerizing my app today. The only decent workaround I could think of is to extract packages that are devDependencies in my apps to a separate repo, publish those to a registry, and install them that way. Definitely not a great workaround in cases where you don't want to publish those.

[WINOFFRG]

Facing same issue my workspace has some tooling packages in devDeps and they are getting bundled in image. Some packages like prettier typescript are quite heavy like 22MB

[abcdefghiraj]

I spent some time struggling with the same issue. I am using pnpm and I tried pnpm deploy, but that messes with the symlinks. If I add the option to inject, then turbo complains.

Till there is a fix, here is a possible workaround -
I'm compiling the build output to a single file (using vercel/ncc - can also be tsup or something similar) and only copying that output to the final layer of my docker image, and I am at a third of the previous size. Be mindful of the pitfalls of this approach and make sure you test all code paths by bundling and running this locally before going to production