Support different versions of the same dependency within a workspace

[alshdavid]

Problem

My Cargo workspace has multiple entry points that compile to executable/bin or dylib targets. Currently, when placed within a workspace, Cargo will attempt to combine their dependencies where they share the same "compatible" versions of a dependency.

The problem is that these project consume third party dependencies that have conflicting versions - either because library maintainers don't adhere to semver correctly or because library maintainers choose to specify exact versions of a dependency (e.g. =0.0.40).

My projects can be compiled on their own without issue however, if they share a workspace, cargo build will fail because Cargo tries/fails to resolve these conflicting dependencies.

Example Case

I have a workspace that has three packages

/crates-main
  /project-bin
    Cargo.toml
  /project-types
    Cargo.toml
/crates-plugins
  /plugin-dynamic-lib
    Cargo.toml
Cargo.toml

crates-main/project-bin
This is the main entrypoint for my application which compiles to an executable. This program is able to consume dynamic libraries using the libloading crate.

crates-main/project-types
This is a shared library compiled as a lib that has no external dependencies, only exporting types to be used by both the "plugins" and the main executable (a.k.a. the "contract").

crates-plugins/plugin-dynamic-lib
This is a library that is compiled to a dylib. It depends on project-typesfor the types required to initialize a plugin and will be consumed by the executable produced byproject-bin`

Note that this package could also be an executable, I am using a dylib in my example because that's my current use case

In summary we have 2 packages that compile to binaries (./project-bin and ./plugin-dynamic-lib.so) and one shared library that is statically linked within those two crates.

Problem

As an example, assume project-bin consumes log = "=0.4.20" and plugin-dynamic-lib consumes log = 0.4.21 (indirectly via a third party dependency external to the workspace).

In a combined workspace, cargo build will error saying that it cannot resolve a compatible version between the two specified.

In reality, these packages will compile to separate binaries so version conflicts of dependencies would not result in a material conflict at runtime.

Current Solution

To get around this today, I simply avoid using a Cargo workspace and compile the projects independently of each other from a build script where shared dependencies are referenced via my_pkg = { path = "../path/to/pkg" }.

The issue with this approach is that rust-analyzer is unable to provide suggestions for the packages when the top level folder is open in the editor - resulting in a less than ideal development experience

Proposed Solution

A few possible solutions to this:

Option 1

Allow for multiple incompatible dependencies to coexist within a workspace if their versions cannot be combined and their consumers are of crate-type bin or dylib

e.g. a package in crates.io with the latest version of 0.4.21

Consumer A

[bin]

[dependencies]
dependency = `^0.4.0`

Consumer B

[bin]

[dependencies]
dependency = `=0.4.20`

Consumer A would get 0.4.21
Consumer B would get 0.4.20

Option 2
Devise a way for rust-analyzer to work with multiple nested projects within a parent directory

Option 3
Perhaps some kind of support for workspaces, isolating dependencies between the workspaces

Notes

No response

[epage]

The problem is that these project consume third party dependencies that have conflicting versions - either because library maintainers don't adhere to semver correctly or because library maintainers choose to specify exact versions of a dependency (e.g. =0.0.40

One of the reasons Cargo is opinionated is to put pressure on libraries to "play nice" by

• Following Cargo community expectations around semver (which is different than a single person's interpretation)
• Discouraging the use of = operators (which we explicitly call out)

[epage]

As for the request, I feel like we should have an existing issue but my search skills are failing me.

• Specify in root that certain edges in the dependency graph should be "private" #7276 proposed the idea of private dependency graphs.
• In theory, RFC: Superseding public/private dependencies rfcs#3516 could help towards something like this.

I do wonder if we'll come to the point where dependency sub-graphs are treated as locked together, of the scale of gitoxide or bevy, making them act more like a cohesive single library. iiuc build-std has a specialized version of this.

[alshdavid]

put pressure on libraries to "play nice" by

I seriously wish this worked out in practice, it's been such a nightmare to work around. (you can skip the next part, I'm just ranting)

I'm writing a JavaScript bundler and want to embed Deno into it to function as a JavaScript runtime for dynamic plugins.

Deno depends on = versions of SWC because SWC doesn't play nice with semver.

I also depend on SWC directly however the conflict between Deno and my internal version of SWC prevents Cargo from resolving dependencies. Further, Deno has a fixed version of Tokio, log and other libraries which makes it difficult to embed.

To try to get around this, I created a new dylib crate in my workspace that bottles up my Deno integration and exposes behaviours via an FFI that I consume from my main application using libloading.

/deno_integration
/my_project

/target
  /debug
    my_project
    deno_integration.so

This is okay enough however my deno_integration dylib crate cannot be part of my workspace Cargo.toml because of those conflicting dependencies (even though it will compile to a discrete target).

So I add deno_integration to my workspace [workspace.exclude] and build it separately to the packages in my workspace.

cargo build # for the workspace
cd deno_integration && cargo build

If I have my workspace open in my editor, I get no rust-analyzer coverage on the deno_integration folder unless I open that folder separately in a new editor window.

It's such an annoying workflow - but it works and unblocks me for now.

[weihanglo]

I believe you've tried this. Just asking in case we forgot: Have you used the [patch] table to patch deno's Cargo.toml?

The other way can help upstream is helping them integrate cargo-semver-checks. The tool is quite handy and easy to use. Cargo the project has used it in every pull request

[epage]

My other question is if swc and deno are meant to be pulled in as libraries like this. These look to be fairly large, complex applications and sometimes the "libraries" for them are more meant for internal purposes. I know I maintain several bins whose lib portion is an implementation detail and I very intentionally do not offer semver compatibility guarantees for those APIs.

[alshdavid]

In the case of swc, it's definitely intended to be used as a library as that's its primary use case (it's an AST parser, etc) - and they have a reputation for poorly adhering to semver. I tried to politely bring it up with the maintainers but they were not receptive.

As for Deno, their crates and docs seem to indicate that this is a use case they support. I don't think this use case is a huge focus for them though as the dependency composition of their crates seem to indicate that.

There are other projects (like Supabase - an OSS "aws lambda"-like engine) who integrate Deno. Supabase's solution was to use some of the deno crates from crates.io and vendor the parts that conflicted