How to set executable path of compiler provided by [build_requires] in profile

[klausholstjacobsen]

Hello
I am trying to cross compile my project for arm.
I have packaged the linaro gcc toolchain in a package of its own.

I have put the new linaro package in the build_requires section of my arm host profile.
But how can the linaro package "communicate" to the profile where the compiler is located so that the profile can set the "CC" variable.

I have tried setting everything in the package_info() method of the linaro package, but the information does not propagate to the profile and my project, resulting in conan/cmake trying to autodetect the compiler and thus finding my installed gcc.

In the ideal world I would like all info about where the compiler is and where the sysroot is to be located in the linaro package itself as this is the one with the knowledge. And then only let my profile or project conanfile set the compiler flags.

I fear that there are concepts within conan that I have completely misunderstood.
Help is appreciated!

Here is relevant files:
linaro conanfile.py:

class LinaroGccConan(ConanFile):
    name = "linaro-gcc"
    version = "8.3.0"
    settings = "os", "arch"
    description = "<Description of LinaroGcc here>"
    url = "None"
    license = "None"
    author = "None"
    topics = None

    def build(self):
        url = ("https://armkeil.blob.core.windows.net/developer/Files/downloads/gnu-a/8.3-2019.03/binrel/gcc-arm-8.3-2019.03-x86_64-arm-linux-gnueabihf.tar.xz")
        tools.get(url)
        
    def package(self):
        self.copy("*")

    def package_info(self):
        path = os.path.join(self.package_folder, "gcc-arm-8.3-2019.03-x86_64-arm-linux-gnueabihf")

        self.env_info.CC = os.path.join(path, "bin/arm-linux-gnueabihf-gcc")
        self.env_info.CXX = os.path.join(path, "bin/arm-linux-gnueabihf-g++")
        self.env_info.SYSROOT = os.path.join(path, "arm-linux-gnueabihf/libc")

Please note that the above CC and CXX variables seem to have no effect.

host profile:

[settings]
arch=armv7hf
os=Linux
build_type=Debug

compiler=gcc
compiler.version=8.3
compiler.libcxx=libstdc++11

[build_requires]
linaro-gcc/8.3.0

Rgds
Klaus

[memsharded]

Hi @klausholstjacobsen

It would be good to know if you are already using the "build" and "host" profile approach (very recommended for cross-building), and it would also be good to know the consumer part, because depending on the generators and other tools that you might be using, it could work or not. In the general case, this should work, so it might be something in the consumer side.

In general, specially if you are onboarding now, I'd probably recommend trying to use the new VirtualBuildEnv generator, because that is the one that will survive in Conan 2.0. It declares self.buildenv_info.define_path("CC", ....), and in the consumer side it always generate an activation script with the variables that will run (it is easier to use as a consumer).

If you want to setup a small reproducible toy project in Github, with a "linaro-gcc" fake recipe, and a consumer fake recipe, that could also really help to check what could be happening.

[klausholstjacobsen]

Yes, I am indeed using the "build" and "host" profile approach.
My build profile is:

[settings]
arch=x86_64
os=Linux
build_type=Debug

compiler=gcc
compiler.version=7.5
compiler.libcxx=libstdc++11

This system does recognise that I am crossbuilding.
I have tried with both a project conanfile.txt and also with a conanfile.py. Both with no success.

conanfile.txt looks like this:

[requires]
libbacktrace/cci.20210118
boost/1.73.0
openssl/1.1.1k
log4cplus/2.0.5
libssh2/1.9.0
xerces-c/3.2.3
libcurl/7.70.0
protobuf/3.12.4
libmicrohttpd/0.9.59
bcm2835/1.68
pca9685/1.0
libarchive/3.5.1

[generators]
CMakeDeps
CMakeToolchain

[options]
boost:without_stacktrace=True
log4cplus:unicode=False
log4cplus:fPIC=True
log4cplus:shared=True

I integrate with my cmake project this way:

conan_cmake_install(
        PATH_OR_REFERENCE ${CMAKE_CURRENT_LIST_DIR}/conanfile.txt
        PROFILE_HOST ${CMAKE_CURRENT_LIST_DIR}/profile.armhf-gcc7
        PROFILE_BUILD ${CMAKE_CURRENT_LIST_DIR}/profile.amd64-gcc7
        BUILD missing
)
include(${CMAKE_BINARY_DIR}/conan_toolchain.cmake)

I have been a bit reluctant to try the "virtualenv" route as I am unsure if it integrates well with clion. I do not want the developers working on the project to have to run special/magic scripts before they can build.

I will try to set-up a small "working" (or rather not working) example project on github.

Regards
Klaus

[memsharded]

I start to understand. The thing is that if you are not using any virtualenv capability, then most of the responsibility falls in the build system. In order for the environment to be activated when cmake is called, there must be something there that passes the information. If the build is done inside a build() method, then Conan will manage to inject the environment. But if it is a pure "consumer"; in this case, called directly from cmake (via the conan.cmake helper), then it is not possible for Conan to pass this environment, because CMake runs and detects the compiler even before Conan is called, right?

[klausholstjacobsen]

That does make sense. I was however under the impression that the cmake toolchain file genereated by the CMakeToolchain generator would do exactly that and bridge the gap between conan and my project.

[memsharded]

That does make sense. I was however under the impression that the cmake toolchain file genereated by the CMakeToolchain generator would do exactly that and bridge the gap between conan and my project.

The responsibility of the CMakeToolchain is mainly mapping from the Conan settings to the CMake arguments and variables (things like the architecture, for example). But full management of environment is not easy with CMake. For example, a very typical build_requires that puts itself in the PATH is the cmake/version Conan package (available in ConanCenter). By definition it is impossible to do anything in the environment that would help to locate the cmake executable inside the Conan package.

At some point, it is possible that the CMakeToolchain improves and is able to read from the environment variables like CC and CXX and apply them to the conan_toolchain.cmake, even if that is just a limited solution, it might help some users.

But in any case, the "chicken and egg" problem still remains, if you are using the conan.cmake integration, the moment you are calling cmake first, and cmake calls Conan, it doesn't matter what conan_toolchain.cmake the CMakeToolchain will generate for you, that it would be already too late to apply it. The conan_toolchain.cmake can only be used with the "canonical" flow conan install + cmake ... -DCMAKE_TOOLCHAIN_FILE=...

[puetzk]

calling cmake first, and cmake calls Conan, it doesn't matter what conan_toolchain.cmake the CMakeToolchain will generate for you, that it would be already too late to apply it.

No, that's not true. CMake does not load CMAKE_TOOLCHAIN_FILE until you enable languages (usually in project()). It should be entirely possible to conan_cmake_install() (and set CMAKE_TOOLCHAIN_FILE) before that happens. It would mean you can't use conan_cmake_autodetect (since nothing is set up yet), but that's fine if you know that conan_toolchain.cmake is going to make them match the other way around.

[memsharded]

Yes, but this is because conan_cmake_autodetect() is basically the core feature of the conan.cmake integration: to map the current CMake invocation to Conan settings, so you don't need to call conan install before calling cmake.

I am not sure if I understand, which flow are you proposing then for this conan.cmake integration?

[puetzk]

Sorry, this was really just a drive-by comment: I found this while searching for anything existing before I decided to write up #10553.

I was just clarifying that it is is possible to change toolchains at the top of CMakeLists.txt (before project()). In fact, you could modify the environment too, if some generator wrote a conanbuildenv.cmake that was full of set(ENV{name} value) lines.

But, I didn't have any specific use in mind. At least, not beyond the overall goal of opening a CMakeLists.txt in an IDE and getting the same outcome as a conan.tools.cmake.CMake(self).configure(), which is not very well-served by virtualenv shell scripts (launching your IDE from a terminal after sourcing a script is clumsy, and that also propagates the environment to all kinds of IDE stuff other than the project build). But a CMakePresets generator (as discussed elsewhere) is probably a better path forward on that goal.

[puetzk]

In terms of "why would one use conan_cmake_install() without conan_cmake_autodetect(), my usual case (which really has nothing to do with this issue) is to conan install a conanfile.txt which lists a lot more requirements (needed only for building) than the conanfile.py knows about (which has only the stuff that a consumer should transitively be aware of). I.e. the sort of stuff we talked about at conan-io/tribe#26 (comment) (and other places).