Conan 2 workspaces

[memsharded]

This ticket purpose is to gather and centralize discussion of all the other opened tickets, now that we are starting to resume work on workspaces for Conan 2.

• The idea of using CMakeDeps in [question] Can an editable package be build using only cmake commands #15880 (comment) by @mattangus needs to be explored
• Adding the capability of having local configuration (custom commands) that compose with the global cache custom commands [feature] Folder specific custom commands #16023
• Adding the capability of having local config (global.conf) in a project, and compose it with the global one: [feature] On-the-fly configuration for 2.0 #6761
• Being able to have "light" workspaces which are mostly just a list of editables: Lightweight workspaces aka Temporary editables #5566
• CMake necessary changes to make the old 1.X workspace work: [workspaces] update documentation #7206
• [question] Shared code and workspaces #5939 seems it would have been solved with new layout() definition
• The issue [workspaces] Workspaces should be Super Builds #5762 contains a large discussion, the globalness of editables need to be addressed (probably the first issue a workspace 2.X will solve)
• Issue [workspaces] Decoupled Workspaces #6844 suggests more decoupled workspaces
• Ticket [feature] Extend .conanrc for profiles information #15209 suggest to be able to have profiles locally defined by .conanrc. If considered, this would be a workspace feature.

[aander80]

I am very keen on seeing workspaces being introduced!

Currently, I can think of two features that I would like to have.

Single build command for building multiple packages in a workspace

I have created a fork of your own fork and implemented a very hacky solution of this: https://github.com/aander80/memsharded-conan/tree/feature/workspace. I believe you should be able to clone my fork, checkout feature/workspace, cd into the workspace folder and run ./demo.sh (I ran it in WSL, have not tested it in Windows).

The idea is to have a command like conan workspace build <targets> (conan ws:build <targets> in my fork) and have conan build all the targets and editable dependencies in the workspace given the build order graph (note: I was not sure how to get this correctly using the API so my solution is probably not fully correct, but it seems to work for my demo). If <targets> is empty, the entire workspace is built. Of course, this would require efficient caching in each build.

On-the-fly editables and overriding dependencies

I have been looking into Cargo workspaces for Rust. There you have the wonderful feature of setting dependencies for the workspace, and in each package/crate re-use the workspace dependencies. Example here:

• Workspace: https://github.com/astral-sh/uv/blob/main/Cargo.toml#L19
• Package/crate: https://github.com/astral-sh/uv/blob/main/crates/uv-cli/Cargo.toml#L16

I imagine something similar would be very powerful for conan, where you can select to override dependencies in a workspace. When you are working with large projects with many commits being merged every day and with dynamic versioning where each commit gets a unique version, managing static editables will be tough to manage (especially since just creating a new commit during local development would force you to change your editables).

To solve this, it would be very useful if you have multiple repos, add them to a workspace, and the dependencies would be automatically injected in the dependency graph as editables, without having to configure them statically as editables in the conan cache.

In my example, let's assume I have two repos: say and hello. Here, I likely always want to make sure I use my local copy of say, and not the latest version in the cache (which might override my local copy/repo). I have currently thought of these different behaviors:

• You always inject/override dependencies in the graph. If hello points to say/[~1.0] but I have say/1.1.0 in my local repo, Conan would still select my local say/1.1.0 version.
• Alternatively, this could be an option/additional parameter that you set, similar to include_prerelease. I am not creative when it comes to names, but it could be something along the lines of allow_workspace_override. This could also be a core configuration that you can set to apply to all dependencies, similar to core.version_ranges:resolve_prereleases.
• It could also be that you must conform to the version ranges in the version ranges. However, Conan could prioritize the workspace versions ahead of the cache, so that if I have say/1.1.0 in my local repo but there is also say/1.1.1 in my cache, the dependency say/[~1.1] would still resolve to my workspace say/1.1.0. (I guess this is identical to the previous item in this list, with the allow_workspace_override feature turned off)

I understand that this could be very tricky to solve since you might have multiple versions and contexts of packages; some might require say/1.0, another say/2.0 etc., some might want to use say as a require, other as a tool-require and so on. I imagine this could be managed by a combination of the second and third items in the behavior list above, but I am probably missing loads here.

For me I think a key feature for simplicity is to have on-the-fly editables that are only injected in the workspace/for the current command.

I hope I managed to make myself understood in my ramblings here! 😄

[memsharded]

Hi @aander80

Thanks for your feedback.

Regarding your first point:

The idea is to have a command like conan workspace build (conan ws:build in my fork) and have conan build all the targets and editable dependencies in the workspace given the build order graph (note: I was not sure how to get this correctly using the API so my solution is probably not fully correct, but it seems to work for my demo). If is empty, the entire workspace is built. Of course, this would require efficient caching in each build.

This can be done today with the conan install <path-to-root> --build=editable. This will call the build() of every package in editable mode in the current dependency graph of <path-to-root> in the right order.

This will be most likely improved with a conan workspace build or similar, but that command will very likely just be a --build=editable under the hood, leveraging some definition in the workspace of the "path-to-root".

About your second point, I think it is very aligned with our current vision, and I have already started to work on it, I have some working code in a branch. Keep tuned, I will try to share when possible to start gathering feedback, I will share it in this thread.

[aander80]

Thanks for your reply! I am happy to hear about your vision, looking forward to seeing this implemented some day! 😄 It is also great to hear building multiple editables is possible already today, that should make multirepo development easier already now, although it might require a few extra commands!