[feature] Multiple parallel profiles support in cmake_layout()

[mpusz]

Until now I used a simple workflow for each of the compilers using Ninja Multi-Config as the default CMake generator:

mkdir -p build/GCC-11 && cd build/GCC-11
conan install ../.. -pr gcc11
conan install ../.. -pr gcc11 -s build_type=Debug
cmake ../.. --toolchain conan_toolchain.cmake
cmake --build . --config Release
cmake --build . --config Debug

I also set the following in VSCode which makes it work like a charm (with just 3 first steps from the above):

• cmake.buildDirectory to ${workspaceFolder}/build/${buildKitVendor}-${buildKitVersionMajor}
• cmake.configureArgs to --toolchain conan_toolchain.cmake

However, the introduction of cmake_layout() as a recommended practice in modern Conan recipes will make the above a lot harder if not impossible so I am looking for the best alternative.

Right now while using cmake_layout(), no matter what the profile file or a current directory is, all the artifacts land out in:

• <project_root>/build/generators
• either <project_root>/build (muti-config) or <project_root>/cmake-build-<build_type> (single-config)

The above scenario accounts only for the differences in the build_type between several Conan runs. However, there are more variables there: compiler, compiler version, C++ version used, the standard library used, ... If we run Conan for such configurations the artifacts will collide as they will land out in the same subdirectories. We can use -of <dir> (i.e. -of build/GCC-11) to handle the differences manually but:

• CMakeUserPresets.json will not land in the correct place to be found by CMake invocations so one must copy it (and override) to the project source directory before every CMake build.
• There will be no aggregated CMakeUserPresets.json that will allow opening such a tree of configurations in the IDE. Even if there was one, all the preset names would collide as they do not use any configuration-specific prefix today.
• We might end up with some strange paths to work with like build/GCC-11/build/Release and build/GCC-11/build/generators

I strongly believe that is a good practice to check the source code on several different compilers before committing changes to the repo. Also such configurations are useful to easily reproduce issues raised by the customers or CI. It would be good if Conan could still facilitate such scenarios with the new cmake_layout() in the recipes.

Unfortunately, I do not have a production-ready solution at hand right now but I hope that we might find one together. The simplest one would be to use a profile name as a prefix/discriminator if it is different from the default. That, however, does not account for settings changes provided from the command line (like conan install . -pr gcc11 -s compiler .version=10) but I do not believe this is a common practice. Probably the most common is conan install . -pr gcc11 -s build_type=Debug which would be fine as those already land-out under the same file tree.

• I've read the CONTRIBUTING guide.

[memsharded]

This works (assuming CMake 3.23, and I am using Windows VS, but the case is identical for other compilers):

conan new hello/0.1 -m=cmake_lib
conan install . -s compiler.version=15 -of=build/vs15 -if=build/vs15
# this generates a CMakeUserPresets.json besides the CMakeLists.txt
conan install . -s compiler.version=15 -of=build/vs15 -if=build/vs15 -s build_type=Debug
conan install . -s compiler.version=17 -of=build/vs17 -if=build/vs17
conan install . -s compiler.version=17 -of=build/vs17 -if=build/vs17 -s build_type=Debug

cmake --preset=default  # configures vs15
cmake --build --preset=Release # builds vs15, Release in build/vs15/build/Release
cmake --build --preset=Debug # builds vs15, Debug in build/vs15/build/Debug

# Replace "vs15" => "vs17" in the ``CMakeUserPresets.json`` ``include()``
cmake --preset=default  # configures vs17
cmake --build --preset=Release # builds vs17, Release in build/vs17/build/Release
cmake --build --preset=Debug # builds vs17, Debug in build/vs17/build/Debug

I would say the user experience is fantastic so far, you can easily test for multiple compilers and configurations.
Recall that the extra -if=xxx will not be necessary anymore in 2.0.

Regarding your comments:

• CMakeUserPresets.json will not land in the correct place to be found by CMake invocations so one must copy it (and override) to the project source directory before every CMake build. I guess you mean CMakePresets.json and not CMakeUserPresets.json. This is definitely a feature. Conan will not save its CMakePresets.json in the root folder, because the risk of destroying the user CMakePresets.json is too high. It will only generate CMakeUserPresets.json if it detects the file doesn't exist, but the CMakePresets.json will be generated in the Conan specific folder to make sure it doesn't destroy any user file.
• The build/vsXX folder is not a "strange" path. It is the place where you told Conan to put the files. You can tell it any other locations, like -of=project_vs15. Inside that there will be cmake-build-debug/cmake-build-release in single-config environments and build/Release, build/Debug for multi-config environments, but that shouldn't be an issue at all, instead I think it is a great behavior, to be always consistent, so if you don't specify the -of=vs15 you will still have a clean and nice layout. Why is it an issue? Note that in your example, it looks less cumbersome because you are using Ninja Multi-Config, but for users using regular Ninja, they will be forced to go inside a further debug or release folder anyway. With cmake_layout() this is not necessary, will be automatically managed for all different single and multi config CMake generators.
• Conan is still not compounding CMakeUserPresets.json, mostly because this is a user file, and Conan doesn't want to mess with it, there is risk to corrupt user data. But you can see that a simple include() to the right folder does work greatly. The basic generation works out of the box for 1-compiler setup, and using multi-compiler setup is just one sed away.

So in summary, I think the workflow and layout that you want to achieve is not only possible, but also natural and smooth.

There could be some further improvements, like trying to do some composition over CMakeUserPresets.json for the different includes, but probably that very much depends on every user needs and projects, I don't even know if possible. If you have some idea about how such a CMakeUserPresets.json composition looks like, please let us know and we will consider it.

It might also be possible to add a conf for cmake_layout that factors the compiler/version in the paths, that could be relatively straightforward. The only thing is that I cannot find anything that resembles a convention for such cases. Ok, we can use build/<compiler><version> for multi-config environments, but what would be the path for single-config environments? cmake-build-<compiler>-<version>-release? ````build//cmake-build-release? And then, what about the architecture arch``? Shouldn't we factor the architecture too? Maybe the ``cppstd``, ``libcx``, ``runtime`` too? I can guarantee there are users that are building for different flavors of those things too. Do you want a ``build/gcc_11_armv8_libcxx11_gnu14_debug`` folder? I think the variability of use cases here is too much , and with the ``-of`` above the flow is fully customizable at the user desired level, trying to do something built-in beyond the build systems standard most basic layouts (mostly Debug/Release), it is too much complexity for the value.

[mpusz]

Recap: My goal is to generate all the files for all the configurations once with Conan and after that open the project in the IDE and use this IDE to switch between different configurations and build all of the targets.

I guess you mean CMakePresets.json and not CMakeUserPresets.json.

Yes, my fault.

and using multi-compiler setup is just one sed away

Well, this is not that easy. It is not only a matter of replacing or stacking all the includes in the CMakeUserPresets.json but also changing the names of presets in every single CMakePresets.json as those will collide with each other.

The only thing is that I cannot find anything that resembles a convention for such cases.

Yes, I admit this is probably the main problem here. I suggested using a profile file name above but it is far from being a perfect solution. Maybe there should be a dedicated command line parameter (i.e. --preset <name>) that could allow the user to name each specific configuration?

# this generates the files in `./out/<preset_name>`
conan install . -pr gcc10 -of out --preset gcc10
conan install . -pr gcc10 -s build_type=Debug -of out --preset gcc10
conan install . -pr gcc11 -of out --preset gcc11
conan install . -pr gcc11 -s build_type=Debug -of out --preset gcc11

cmake --preset gcc10-default
cmake --build --preset=gcc10-Release
cmake --build --preset=gcc10-Debug
cmake --preset gcc11-default
cmake --build --preset=gcc11-Release
cmake --build --preset=gcc11-Debug

[lasote]

Recap: My goal is to generate all the files for all the configurations once with Conan and after that open the project in the IDE and use this IDE to switch between different configurations and build all of the targets.

👍🏼

Maybe there should be a dedicated command line parameter (i.e. --preset ) that could allow the user to name each specific configuration?

Well, we cannot couple a --preset (CMake stuff) to a generic conan install but I get the idea.
We would need to change:

1. The 'build folder' must be different
2. The 'Preset names' must be "unique" and point to the right 'build folder'
3. Ideally, the CMakeUserPresets.json should aggregate all the different files that come from conan install commands.

We will do a POC with a conf entry, kind of tools.cmake.CMakeToolchain.layout_settings (name to define) with a list of settings, e.g ["compiler", "compiler.version"] to generate both different 'build folders' and 'preset names` and see how it looks.

I admit that managing all these different configurations with your IDE just selecting the Preset would be pretty cool. Let's see if we can do it in an easy and "sane" way for the conan Client.

Thanks Mateusz

[mpusz]

Thanks @lasote!v The parameter I proposed does not have to be named preset. I actually thought that maybe having a user-provided name for a specific configuration could be useful in other places as well?

[mpusz]

I just tried the feature in 1.49. It seems there is some issue for a multi-config generator. If I set tools.cmake.cmake_layout:build_folder_vars=["settings.compiler", "settings.compiler.version"] in global.cof and I call conan install for Release and Debug only Release presets are in CMakePresets.json and Debug ones are not appended to this file.

Also, there is one thing that could be improved in my opinion. Is it possible to put "Release/Debug" part at the end of the list rather in the beginning of the preset and directories names?

Right now I get the following file tree:

• build-clang-14/generators
• build-gcc-11/generators
• cmake-build-debug-clang-14
• cmake-build-debug-gcc-11
• cmake-build-release-clang-14
• cmake-build-release-gcc-11

It is quite hard to reason about and maintain.

I think that the following file tree has more sense:

• build-clang-14/generators
• build-clang-14/debug
• build-clang-14/release
• build-gcc-11/generators
• build-gcc-11/debug
• build-gcc-11/release

The above does not use cmake- prefix for build directories and puts debug and release in a subdirectory parallel to generators. Alternatively, those could land in build-clang-14-debug and build-clang-14-release which still seems to be improved with the default sorting of folder names. For example, it is easier to get rid of artifacts for clang-14 with just rm -rf build-clang-14* which will remove all artifacts for this compiler. With the current approach, it is much harder to maintain such a file tree.

[mpusz]

Also, conaninfo.txt and its "friends" are being generated for me in the main directory again even though I use cmake_layout.

[lasote]

Hi!

I just tried the feature in 1.49. It seems there is some issue for a multi-config generator. If I set tools.cmake.cmake_layout:build_folder_vars=["settings.compiler", "settings.compiler.version"] in global.cof and I call conan install for Release and Debug only Release presets are in CMakePresets.json and Debug ones are not appended to this file.

It should be working, I'll take a look.

About the folders, as the generators (CMakeDeps, CMakeToolchain) are "multi-config", we needed to extract the folder from the build because it is shared between debug and release. So we also decided to have them in the same place for single and multi configuration cmake generators.
The composition of cmake-build-release (and now with more suffixes if you use this feature) is there mostly because of Clion layout. I agree that it might be a bit confusing. Your proposal meet all the requirements I did, so I will discuss with the team.

Thanks! I will let you know about the bug your reported as soon as I reproduce.

[lasote]

Hi again Mateusz!

I'm not able to reproduce the issue. Both Release and Debug configurations are generated correctly. Would it be possible to get an example with the steps to reproduce?

Also, conaninfo.txt and its "friends" are being generated for me in the main directory again even though I use cmake_layout.

About that, yes, the good news is that that file and friends are gone in Conan 2.0. In Conan 1.X they don't respond to the layout.

[mpusz]

Hi @lasote! I am so sorry for the confusion. I really do not know what happened. Today the same command lines for the same projects in the same IDE (I just put my laptop to sleep overnight) do not reproduce the problem. Strange... and sorry.

Still, the directory layout could be optimized a bit ;-)
My preferred layout could be even like this:

• build/clang-14/generators
• build/clang-14/debug
• build/clang-14/release
• build/gcc-11/generators
• build/gcc-11/debug
• build/gcc-11/release

[lasote]

I am so sorry for the confusion. I really do not know what happened. Today the same command lines for the same projects in the same IDE (I just put my laptop to sleep overnight) do not reproduce the problem. Strange... and sorry.

No problem! if you reproduce again let me know.

Still, the directory layout could be optimized a bit ;-)

yes! I tried Clion and the "bin dir" is set to the one you put in the Preset so I would say, we don't have further reasons to keep that layout so I will open a PR to change it and discuss it with the team. I also like more this one:

• build/clang-14/generators
• build/clang-14/debug
• build/clang-14/release
• build/gcc-11/generators
• build/gcc-11/debug
• build/gcc-11/release

[mpusz]

Is it possible to prevent repeating the same preset in CMakePresets.json? If I run conan install several times I end up with:

   ...,
    "buildPresets": [
        {
            "name": "Release-gcc-11",
            "configurePreset": "default-gcc-11",
            "configuration": "Release"
        },
        {
            "name": "Release-gcc-11",
            "configurePreset": "default-gcc-11",
            "configuration": "Release"
        }
    ],
   ...