Monorepo support

[wbcs]

What is the problem this feature will solve?

bun not support monorepo(root package.json workspaces field, workspace:* schema, etc.)

What is the feature you are proposing to solve the problem?

The download speed of monorepo will be faster

What alternatives have you considered?

No response

[Jarred-Sumner]

yes, but probably not for a little while

[FinnRG]

Related: The workspace: field progress is tracked in #83.

[davidmeirlevy]

can the implementation of workspaces include:

1. the ability to have separated node_modulues on each workspace.
2. the equivalent of lerna exec to execute any command inside the workspaces, even if it's not inside the package.json.
   for example, I want to run bun dev on each workspace, but the dev itself probably won't be a script inside the package.json of each workspace. the same for tests...

[factoidforrest]

I just want to say that I haven't missed lerna exec at all after ditching lerna for pure yarn workspaces. Things like that are certainly nice-to-haves but I don't think going any further than the support provided by yarn or NPM is necessary as a first-pass.

[davidmeirlevy]

@factoidforrest yarn has yarn exec that can work on workspaces, so it's equivalent.

[unional]

Maybe consider monorepo support similar to turbo and rush. They are both very efficient.

[wbcs]

nsider monorepo support similar to turbo and rush. They are both very efficie

I prefer pnpm

[unional]

I prefer pnpm

npm/yarn/pnpm are orthogonal to lerna/turbo/rush. One are package managers, the others are monorepo tools.

[davidmeirlevy]

I prefer pnpm

npm/yarn/pnpm are orthogonal to lerna/turbo/rush. One are package managers, the others are monorepo tools.

He probably meant to pnpm workspaces.
I also prefer this way, because of the full scope isolation for each package.

[ozyman42]

I prefer pnpm

npm/yarn/pnpm are orthogonal to lerna/turbo/rush. One are package managers, the others are monorepo tools.

He probably meant to pnpm workspaces. I also prefer this way, because of the full scope isolation for each package.

Yarn 2+ Plug n Play also has full scope isolation for each package. A main reason the pnpm resolution strategy might be preferable to Plug n Play is because notable projects such as React Native are currently incompatible with Plug n Play. On the flip side, Plug n Play might be considered preferable because it may solve the doppleganger problem differently than pnpm (someone ought to investigate this comparison in detail). Personally I find the "non-single singletons" doppleganger consequence the most important one because it's the root cause behind monorepo bugs involving popular libraries such as graphql-js.

Overall though I think Bun should make an authoritative choice for workspaces/monorepo support rather than letting this be highly configurable. Too much configurability leads to too much project complexity both in implementation and from a client's point of view w/ too many options to choose from & too much boilerplate. Bun needs to have a clear story for workspace/monorepos before it can be seriously considered for adoption by medium to large projects.

[unional]

Agree. IMO pnpm doppleganger solution is also the way to go.

The point I want to make by bring yo turbo and rush is the ability to efficiently running multiple scripts with dependencies.

For example:
https://turborepo.org/docs/core-concepts/pipelines
https://rushjs.io/pages/advanced/incremental_builds/
https://rushjs.io/pages/developer/selecting_subsets

[wbcs]

Agree. IMO pnpm doppleganger solution is also the way to go.

The point I want to make by bring yo turbo and rush is the ability to efficiently running multiple scripts with dependencies.

For example: https://turborepo.org/docs/core-concepts/pipelines https://rushjs.io/pages/advanced/incremental_builds/ https://rushjs.io/pages/developer/selecting_subsets

turborepo is greate.
but how rush run custom npm scripts in subsets? It run reserved keyword instruction only.

[unional]

but how rush run custom npm scripts in subsets? It run reserved keyword instruction only.

It does not. rush has it's own way to add commands to the system.
(to run custom npm scripts in one package, use rushx run ...)
I personally like turbo better too. The rush system sounds too heavy and rigid to me (I tried both).
Also mentioning rush here for completeness and also may able to draw an idea or two from it's selective script execution (those from/to/since things).

[melMass]

Hi,

IMHO the only responsibility from Bun regarding workspaces is dependency resolution, the rest should be handled by a monorepo tool like Turbo or NX.

[o-alexandrov]

Is npm run SCRIPT_NAME --workspaces part of this issue, or does it need a separate one?

---

Related to previous comments: #533 (comment), #533 (comment)

[evelant]

Per #83 it looks like some support for workspace: protocol was added but it doesn't appear to work with pnpm workspaces. Did I miss something or are there still additional changes needed to allow bun to work with a pnpm monorepo?

[Bessonov]

Probably, related: #2517

[jtlapp]

Speed and disk space are the reasons pnpm is best for monorepos. Whenever multiple workspaces within a monorepo depend on the same npm package, npm and yarn download and install a separate copy of the npm package for each workspace, whereas pnpm symbolically links them all to the same download of the package. Yarn's PnP does muddy the waters a bit, but its problems keep me from using it. See the benchmarks.

[scinos]

Whenever multiple workspaces within a monorepo depend on the same npm package, npm and yarn download and install a separate copy of the npm package for each workspace

For yarn, it's configurable: https://yarnpkg.com/configuration/yarnrc#nmMode

[jakeleventhal]

Will migrate to bun when this works

[Bessonov]

I am not sure now, it isn't supported and described here? https://bun.sh/docs/install/workspaces Do I miss something or can this issue be closed as completed?

EDIT: I see it miss support for script execution inside monorepo, ex. pnpm run --filter mypackage start. Probably, should be tracked as a separate issue.

[Hebilicious]

@Bessonov I'm not sure it should be considered completed so soon, there's several workspace related bugs and missing features (like filtering that you mentioned)

I went through the open issues, it looks like there's a bunch of duplicate, but we could use this issue as a tracking one.

• bun i in workspaces doesn't symlink local dependencies #4830
• bun install in pnpm/yarn workspaces #2517
• bun install doesn't work with workspaces #4628
• monorepo package dependencies are usable across packages, also allows use of phantom dependencies #4274
• Support bun run --workspace  #2232
• SvelteKit not working with workspaces #4174
• Bun install does not install dependencies between packages in the same monorepo #4650
• Support workspaces without workspace: protocol #4676
• bun install workspace package parse error #3582
• Workspace-References are not used in bun install #4366
• in workspaces you can not use a ! to exclude and errors out on install #4621
• bun add fails in a sub-package of a monorepo workspace #3502
• bun install not working for workspace projects from root #3806
• workspace packages list gets stale if new workspace packages are depended on #3686
• Implement workspace:^version package.json dependency version specifier #3677
• workspace member package can't depend on workspace root #2773
• Wrong package.json workspace: prefix behavior #2938
• postinstall scripts in monorepos aren't run if there's errors with the installation #4447

[Bessonov]

@Hebilicious Thank you very much for the excellent overview! I am still of the opinion that documented basic use cases should work. So, the binary answer to this issue is "yes, a monorepo is supported". I am not opposed to leaving this issue open, but, in my opinion, it distracts from addressing actual problems. Alternatively, @wbcs could add your list to the top of this issue, making it a meta-issue for outstanding work.

[james-elicx]

I feel it's relevant to share this tweet that I saw from Jarred a couple of months ago. He mentioned wanting to have another go at implementing workspaces as he didn't think the support was that good, although I'm not sure if they ever got around to redoing it.

https://twitter.com/jarredsumner/status/1682860099528978432

[jakeleventhal]

Additionally plugging this issue: #4844

[itpropro]

@Hebilicious Thank you very much for the excellent overview! I am still of the opinion that documented basic use cases should work. So, the binary answer to this issue is "yes, a monorepo is supported". I am not opposed to leaving this issue open, but, in my opinion, it distracts from addressing actual problems. Alternatively, @wbcs could add your list to the top of this issue, making it a meta-issue for outstanding work.

It's hard to argue that bun has monorepo support in terms of a drop in replacement for pnpm. Replacing pnpm works for the most part, but as soon as you come to monorepos (which is very common in OSS and other projects these days), you have to redesign your whole CI/CD pipelines or have challenges that can be very hard to solve.
You can't just run bun run --filter "@scope/package1" build instead of pnpm run --filter "@scope/package1" build etc.. There is still a long way to go and I am willing to go it with bun, but it should be communicated transparently that there is no real monorepo support as people expect when coming from pnpm or yarn.

[jakeboone02]

@itpropro I think (1) "doesn't fail when used in a monorepo", (2) "supports monorepos", and (3) "includes tools that make working with monorepos easy" are all valid but very different definitions of "monorepo support". Bun currently meets the criteria for (1), sort of meets the criteria for (2), but definitely does not meet the criteria for (3).

As evidenced by @Hebilicious's list (#533 (comment)) and other comments here, there's a lot of work to do before that last criteria is met. Documentation should probably reflect that.

[danieldunderfelt]

@itpropro I think (1) "doesn't fail when used in a monorepo", (2) "supports monorepos", and (3) "includes tools that make working with monorepos easy" are all valid but very different definitions of "monorepo support". Bun currently meets the criteria for (1), sort of meets the criteria for (2), but definitely does not meet the criteria for (3).

As evidenced by @Hebilicious's list (#533 (comment)) and other comments here, there's a lot of work to do before that last criteria is met. Documentation should probably reflect that.

Criteria 1 is not met yet, Bun 1.0 still fails for me when adding packages as described in #3502.

[unional]

Bun, please do NOT consider to emulate pnpm. npm is the most popular has been safer for me to use in tens of different projects in various requirements.

I would disagree. Not saying bun SHOULD emulate pnpm, but it should learn and adopt the implementation that makes the most sense.

The doppelganers issue from npm and yarn is a deal breaker for many packages, especially those using TypeScript.

[o-alexandrov]

It’s much easier to develop an identical to npm DX.
No matter how you two call the non-existent in npm features (emulation, learning, adopting):

• in the end, you suggest to complicate the solution with your premature optimization

Any complication brings more bugs to the already unstable bun.
That’s why I strongly suggest to focus on replicating npm exactly. Once bun is stable, only then optimization and edge cases should be revisited.

[websorokinweb]

That's all fun and cool but I and other people with similar projects structure just will not use bun. No one is going to rewrite project to adapt package manager

[johncarmack1984]

That's all fun and cool but I and other people with similar projects structure just will not use bun. No one is going to rewrite project to adapt package manager

hehe, yeah someone would have to be out of their mind to do that... haha... hehe... quietly makes repository private

[jakeleventhal]

pnpm will soon have a feature to keep dependencies in sync across multiple packages in a monorepo (and more) pnpm/rfcs#1. A lot of thought and discussion has gone into the design and I think it's really solid so maybe something for bun to emulate.

As far as package management in bun goes, I think emulating pnpm in general would be a good choice. The patterns used by pnpm (content addressable store, "safe" no hoisting node_modules, etc) is overall safer, more correct, and eliminates a lot of problems common to yarn and npm. IMO keeping the old hoisted/flattened node_modules pattern around would be a step back when pnpm offers thoroughly proven solutions that could be emulated.

#4844

[ysfaran]

@Bessonov

@ysfaran @websorokinweb

What about dependency hoisting configuration?
I am thinking about migrating from yarn to bun as my package manager.
To make each workspaces really independent I use following confugration in yarn:

nmHoistingLimits: workspaces
nmMode: hardlinks-local

This basically means "Don't hoist dependencies above the workspace level, in case two workspaces use the same dependency@version use hardlinks". With this approach I am able to package every workspace independently in my pipeline and truly separate the packages file-wise from each other.
That's a must for any bigger monorepo IMO. Otherwise dependency management gets really messy and cumbersome.

Can you please explain the use case? I have been using monorepos for years for library, frontend, and backend development, and I have never experienced this need.

And I am not sure what you mean by "making each workspace really independent". Typically, monorepos are used for the opposite purpose, to create dependencies between projects.

What I mean with independent is if you for example have two packages package-a and package-b and both use axios as a dependency. What you can do now is to only install axios for package-a (meaning it will only be part of package-a's package.jons) and still do import axios from "axios" in package-b without any error. This happens because by default every package manager is hoisting dependencies to the root of a project and the node resolution algorithm will also look in every parent node_modules folder until it reaches the root.

IMO this is pure madness and should not be possible at all. Only if the root package.json specifies those dependencies it should be possible (but even that is a bad practice IMO and should only be valid for binaries or when you write a library and want to be sure to use just on version of a package for a smaller bundle size). By setting a hoisting limit you are able to specifiy how far up dependencies are allowed to be hoisted.

But hoisting limits are not enough because it is very often the case that multiple packages use the same dependencies with the exact same version. To prevent data duplication and multiple downloads you can use links (soft- or hardlinks) to either the root node_modules of your project (local) or to a node_modules folder shared accross projects (global).

The main difference between softlink (symbolic links) and hardlinks resides in the fact that hardlinks point to an actual address on your hardrive while softlinks point to a file path. In other words: if you zip a hardlinked file it will actually zip the file contents, but if you zip a softlinked file it will just zip the path to a file, which is (if you didn't also zip it) not part of the resulting zip file. Now if you unzip that file on another machine (or other location) your softlink is most likely invalid.

This is why I previously said:

That's a must for any bigger monorepo

But you may ask why?

1. Some libraries/frameworks which have commands like nuxt build (SSR for Vue.js) require that dependencies are directly listed in the node_modules of your package. If you write a frontend with nuxt and want to integrate into a monorepo workspace, you are required to prevent hoisting. If you then want to package your build artifact in a zip or docker image your also required to use hardlinks, unless you want to unnecessarily also zip the root node_modules which contain more than you need.

2. If your monorepo consists of a lot microservices, frontends and shared packages, you need to care about pipeline performance and separation. So what we do in our pipeline is to separate our microservices into their own pipeline job and run them in parallel. For each job we then only install dependencies required for the specific microservice. In the end each microservice job create a build artifact (often a docker image) which can then be deployed and used for end-2-end testing. If you have some kind of change detecion in place you can even skip some jobs. So if just one microservice changed, only run the pipeline job for that microservice and for the rest, reuse the artifacts of the previous pipeline runs.

3. You are able to let different teams work on the same monorepo. If you have a team that is responsible for microservice-a and another team responsible for microservice-b, they should not have any dependency in the code to each other (they might still use a shared package, e.g. for tracing, which is why they reside in the same workspace). With hoisting you implicitly have this dependency, where a package installed microservice-a can be used in microservice-b. This can lead to very unexpected errors and complications accross teams. With hoisting limits and hardlinks you can use GitHub's Codeowner Feature very effectively too.

I hope this answers your question.

[Bessonov]

@ysfaran

his happens because by default every package manager is hoisting dependencies to the root of a project and the node resolution algorithm will also look in every parent node_modules folder until it reaches the root.

This isn't the case with pnpm:

$ cat packages/package-*/package.json
{
	"name": "package-a",
	"dependencies": {
		"to-array": "0.1.4"
	}
}
{
	"name": "package-b",
	"dependencies": {
		"to-array": "0.1.4"
	}
}

$ tree -a
.
├── node_modules
│   ├── .modules.yaml
│   └── .pnpm
│       ├── lock.yaml
│       ├── node_modules
│       │   └── to-array -> ../to-array@0.1.4/node_modules/to-array
│       └── to-array@0.1.4
│           └── node_modules
│               └── to-array
│                   ├── index.js
│                   ├── LICENCE
│                   ├── .npmignore
│                   ├── package.json
│                   └── README.md
├── package.json
├── packages
│   ├── package-a
│   │   ├── node_modules
│   │   │   └── to-array -> ../../../node_modules/.pnpm/to-array@0.1.4/node_modules/to-array
│   │   └── package.json
│   └── package-b
│       ├── node_modules
│       │   └── to-array -> ../../../node_modules/.pnpm/to-array@0.1.4/node_modules/to-array
│       └── package.json
├── pnpm-lock.yaml
└── pnpm-workspace.yaml

IMO this is pure madness and should not be possible at all.

Well, we are in the same boot 👍 It wasn't clear to my what do you exactly mean by hoisting, because i have different understanding of that.

To prevent data duplication and multiple downloads you can use links (soft- or hardlinks)

pnpm uses symlinks, like shown above. Bun uses hardlinks.

The main difference between softlink (symbolic links) and hardlinks

hardlinks don't work across mounts.

[vlechemin]

As far as package management in bun goes, I think emulating pnpm in general would be a good choice. The patterns used by pnpm (content addressable store, "safe" no hoisting node_modules, etc) is overall safer, more correct, and eliminates a lot of problems common to yarn and npm. IMO keeping the old hoisted/flattened node_modules pattern around would be a step back when pnpm offers thoroughly proven solutions that could be emulated.

A concrete example is that eslint + typescript will parse every file in the root node_modules, even those which are not needed in the current package (because of hoisting), by doing so it will negatively impact bun --bun run lint which is much slower than pnpm run lint.

And on the same subject, auto install does not work with mono repo, because bun install is needed to link the workspace packages.

[hansoksendahl]

I wanted to switch to bun. I was really excited about it. I was able to remove a ton of dependencies from the packages in my monorepo. It really cleaned things up... UNTIL I learned that every *.d.ts file in the hoisted node_modules was available regardless of what package I was in.

Hoisting is a mistake in monorepo management, it sounds good in principle but node's module resolution algorithm makes it unsafe. I am sad to see that most of the commentary from people trying to use Bun in a monorepo configuration is from September 2023. That means it's been almost six months with no roadmap for developing a more robust solution in bun.

I'll subscribe to this thread and others but ATM I'm going to have to switch back to PNPM.

[Hebilicious]

I wanted to switch to bun. I was really excited about it. I was able to remove a ton of dependencies from the packages in my monorepo. It really cleaned things up... UNTIL I learned that every *.d.ts file in the hoisted node_modules was available regardless of what package I was in.

Hoisting is a mistake in monorepo management, it sounds good in principle but node's module resolution algorithm makes it unsafe. I am sad to see that most of the commentary from people trying to use Bun in a monorepo configuration is from September 2023. That means it's been almost six months with no roadmap for developing a more robust solution in bun.

I'll subscribe to this thread and others but ATM I'm going to have to switch back to PNPM.

Some progress has been made since the last time I updated the list here : #533 (comment)

I'm personally using a mix of pnpm and bun : pnpm to install deps and bun to basically replace tsx, zx, and node. I'm also using bun itself quite a lot, things like Bun.serve, bun --hot, bun test etc ... It's great!

However as a package manager it doesn't compete with pnpm now, proper mono-repo support will definitely increase usage, but some important features like patch are still needed.

[hansoksendahl]

@Hebilicious thanks for the follow up.

A hybrid approach is interesting and could work. It just seems that it might be hard to evangelize across a development team. An errant bun add [package] on the part of a developer could lead to a lot of head scratching.

[ozyman42]

Currently I have to use pnpm alongside all my bun projects since bun doesn't actually support monorepos yet.
Would be great to at some point only need bun.

1. Hoisting must be disabled. Not allowed. Hoisting breaks monorepos
2. sym / hard linking should be used similar to pnpm. That's the industry standard now in mid 2024

would be nice to see someone given a mandate to go implement this.

[81reap]

I'm a bit new to the front end world but is this what y'all are talking about?

• https://bun.sh/guides/install/workspaces
• https://bun.sh/docs/install/workspaces

[birkskyum]

Found I can use bun workspaces, but when i build a package i have to execute the build with node, because bun is hoisting all dependencies to root level node_modules, and only node will be able to find those from a /packages/package1/build/app.js by properly checking parent directories in module resolution. Bun seem to only check inside the package which is not where bun install puts the node_module.

[ozyman42]

@81reap as I mentioned in my comment and as is mentioned in those workspaces docs, Bun does hoisting but that is an inadequate way to support monorepos because it permits/encourages the phantom dependencies bug class.

[birkskyum]

With pnpm I sometimes have to set i.e. shamefully-hoist=true in the .npmrc, but that taps into another highly upvoted issue

• Support reading from .npmrc #643

[metaspartan]

yes, but probably not for a little while

Any updates on this? Confirmed monorepos not working still as of bun v1.1.4 on any OS

[wesleycoder]

@metaspartan This whole issue is huge and there's definitely work being done.
I've just seen some fixes being merged related to --filter which is a step closer to having a monorepo worflow.
To help with anxiety I keep an eye on the issue list starting with these that @Hebilicious was kind to filter.

No need to put more pressure on this great group of people doing the hard work.
When it's done it will definitely be broadly announced and comemorated.
For now we can keep an eye on the issueas and changelogs and experiment on the canary builds daily, giving feedback on the proper issues.

[hansoksendahl]

It looks like the features to support monorepos are getting close to being ready. Any ETA or an RC build that includes these features?

[wesleycoder]

Everything merged to the main branch is be available on the canary builds.

[c0ncentus]

When i try switch npm to bun in windows I hoping to see if monorepo will be there .
With npm there are lerna, nx, mooncli and now crap stuff works (manual script on all 20 packages T^T).

Hope that bunjs make it simple and easy like bun -w build && bun -w publish && del **/**/dist

[wesleycoder]

Nothing stops anyone from using third party monorepo tools with bun.
I've been using turborepo with bun for a while, and while we have to make some concessions and adjust some things we can make monorepos work if we are willing to make those.

The contributors at oven/bun are doing a great work, let's not turn this issue into a long discussion with lots of people just asking for the same thing they are already working on.

Progress will be reported when it is relevant, as every open source project do, and bun hasn't let us down.
Let them work in peace and keep the anxiety in control.

[hansoksendahl]

@wesleycoder I think a lot of us are just excited. We would like to simplify our install, unit-testing, browser testing, build, and runner down to a single tool. So no anxiety needed.

I did some refactoring of my codebase and the amount of packages I was able to rip out of my projects when using bun was just... chef's kiss.

That is until I realized the monorepo support and specificity that I got with tools like pnpm was just not there due to hoisting (the node_modules strategy that bun currently uses).

Another part of what has some of us excited is that installing, building, testing, running with bun would cut down on compute time and result in real cost savings for production apps.

[birkskyum]

With latest bun I was able to finally migrate from pnpm to bun workspaces (have 4 packages that rely on one another), without any of the issues with hoisting i mentioned prior - thanks!