[question] How do you set the NDK path for Android builds in CMakeToolchain?

[ssrobins]

I'm trying to figure out how to build for Android using the Conan 2.0 CMakeToolchain. I get this error:

ConanException: CMakeToolchain needs tools.android:ndk_path configuration defined

The docs mention tools.android:ndk_path, but not how to set it. There are a few clues in the source code, but all the things I've tried in conanfile.py and conan.conf haven't worked. This was very easy with the cmake generator, I just did this:

cmake.definitions["CMAKE_TOOLCHAIN_FILE"] = os.getenv("ANDROID_NDK_ROOT") + "/build/cmake/android.toolchain.cmake"

...but it doesn't work anymore.

Currently, I share this Python script with my Conan 1.x packages to define build settings for desktop and mobile platform builds:
https://github.com/ssrobins/conan-cmake_utils/blob/217df94/cmake_utils.py.

In addition to being stuck on this, I'm a bit concerned that Conan 2.0 will make it not possible or at least harder to use custom toolchains, which is critical for close-sourced platforms that don't have any built-in CMake support.

[memsharded]

Hi @ssrobins

Configuration can be added to profiles, you can try:

[settings]

[conf]
tools.android:ndk_path=your/path/to/android

In addition to being stuck on this, I'm a bit concerned that Conan 2.0 will make it not possible or at least harder to use custom toolchains, which is critical for close-sourced platforms that don't have any built-in CMake support.

Conan 1.37, to be released today, contains CMakeToolchain changes to allow both providing your toolchain file (replacing conan_toolchain.cmake one), and also to provide an extra toolchain file to be included from the Conan. Please have a look at that, the intention is that doing this is more explicit, clear and convenient than in previous versions! https://github.com/conan-io/docs/pull/2105/files

[ssrobins]

Thanks @memsharded, putting that code in ~/.conan/profiles/default allowed NDK to be found. Unfortunately, it failed right after with:

Invalid Android STL: None.
CMake Error: CMake was unable to find a build program corresponding to "Ninja Multi-Config".  CMAKE_MAKE_PROGRAM is not set.  You probably need to select a different build tool.

I tried adding ANDROID_STL and CMAKE_MAKE_PROGRAM, even though ninja is in my PATH and is working in non-Conan 2.0 packages, but it didn't help. Here's my full generate() method:

def generate(self):
    tc = CMakeToolchain(self)
    if self.settings.os == "Android":
        tc.generator = "Ninja Multi-Config"
        tc.variables["CMAKE_MAKE_PROGRAM"] = "/usr/local/bin/ninja"
        tc.variables["ANDROID_STL"] = "c++_static"
    elif self.settings.os == "iOS":
        tc.generator = "Xcode"
        tc.variables["CMAKE_OSX_DEPLOYMENT_TARGET"] = self.settings.os.version
    elif self.settings.os == "Linux":
            tc.generator = "Ninja Multi-Config"
    elif self.settings.os == "Macos":
        tc.generator = "Xcode"
        tc.variables["CMAKE_OSX_DEPLOYMENT_TARGET"] = self.settings.os.version
    tc.generate()
    deps = CMakeDeps(self)
    deps.generate()

The CMAKE_OSX_DEPLOYMENT_TARGET for macOS and iOS are working so this seems to only affect the Android builds.

For Conan 1.x, this is all I had to add to my conanfile.py to build Android:

if settings.os == "Android":
    cmake.generator = "Ninja Multi-Config"
    if settings.arch == "armv7":
        cmake.definitions["ANDROID_ABI"] = "armeabi-v7a"
    elif settings.arch == "armv8":
        cmake.definitions["ANDROID_ABI"] = "arm64-v8a"
    cmake.definitions["ANDROID_TOOLCHAIN"] = "clang"
    cmake.definitions["ANDROID_STL"] = "c++_static"
    cmake.definitions["CMAKE_TOOLCHAIN_FILE"] = os.getenv("ANDROID_NDK_ROOT") + "/build/cmake/android.toolchain.cmake"

I was just able to pass CMake params exactly how it would work without Conan. Yes it's more code than if it was taken care of for me, but I appreciated the transparency and simplicity. It let me use the ANDROID_NDK_ROOT environment variable as the single definition for NDK on my system that works for CMake and Conan 1.x. It's also starting to become the standard in cloud CI environments like GitHub Actions.

For the Android toolchain as well as any custom toolchain, I would love for this to be done with a setting in the conanfile.py, like it was before. The cmake interface is excellent in Conan 1.x, I don't understand why it's being made less discoverable and spread to multiple files for Conan 2.x.

[memsharded]

was just able to pass CMake params exactly how it would work without Conan. Yes it's more code than if it was taken care of for me, but I appreciated the transparency and simplicity. It let me use the ANDROID_NDK_ROOT environment variable as the single definition for NDK on my system that works for CMake and Conan 1.x. It's also starting to become the standard in cloud CI environments like GitHub Actions.

For the Android toolchain as well as any custom toolchain, I would love for this to be done with a setting in the conanfile.py, like it was before.

The main reason is the developer flows, which at the moment are broken. The developer experience that needs to develop without Conan, just with the IDE or command line. Everything that the CMake() helper was injecting in the build() method would need to be passed by the developer, to achieve the same build, and that is completely unreasonable, there are too many things. So the flow is being changed to:

• Instead of the CMake() helper in build() method to decide what arguments to pass to cmake, it is the generate() method and the new toolchains and generators.
• The generate() method will generate files, that both developers and the build() method can use to achieve the same build with simplicity (like providing a CMake toolchain).

The cmake interface is excellent in Conan 1.x, I don't understand why it's being made less discoverable and spread to multiple files for Conan 2.x.

I am glad that you find it useful, but that is not the majority of the users feedback, that push towards a more CMake idiomatic integration (find_package and toolchain files, basically).

Dont worry too much, I think that you are experiencing the first iterations of the new integrations so some releases might be necessary to stabilize and provide the same (or hopefully better) integration and transparency. In your case, I am moving this to a feature request, because from what I see, there are some items which are good candidates to be implemented in the CMakeToolchain:

• tc.variables["ANDROID_STL"] = "c++_static"
• tc.variables["CMAKE_OSX_DEPLOYMENT_TARGET"] = self.settings.os.version

And CMAKE_MAKE_PROGRAM to be able to point to your own Ninja could probably be made a configuration, so it doesn't need to be passed in recipes.

Regarding the generator, if you are using profile files, you might also simplify your recipe by defining also the generator in configuration with tools.cmake.cmaketoolchain:generator=xxxx (from Conan 1.37 too).

[ssrobins]

@memsharded
Would you agree that the developer workflow problem is more of a CMake problem than a Conan problem? Whatever developers were doing before using Conan to avoid having to type out long cmake commands should surely work with Conan, right? Over the years, I personally had bash/batch/python scripts that would store cmake command line options for my chosen platform variants so I could share it between developer workflows and automation. It was not ideal, but it was effective.

Thankfully, Kitware helped solve this in CMake itself with the new CMake Presets feature. Here's an example file. Using presets, the CMake command line is cmake --preset=name-of-preset, which could also be used in Conan AND the CMake GUI puts the presets in the dropdown menu for easy use there too. There's also new VIsual Studio integrations for developers using that.

I'm not saying to force the use of presets instead of toolchains. Presets came into CMake in 3.19 and I'm sure some will desire older CMake compatibility than that. I'm just saying don't force us to use either one. Seems way more flexible and easier on you, while still allowing for reasonable developer workflows to just let me pass in anything to cmake and cmake --build. As a result of that clean, simple interface, I've been able to make Conan packages for platforms it has probably never been officially tested with. With Conan 2.x, I'm having to learn new interfaces and deal with new problems to try to get that functionality back.

If there's anything I can do to help with documentation or convincing those who disagree, let me know. I have a lot of open source examples floating around. I'd also be happy to contribute code to other people's projects to help address workflow issues with Conan 1.x. But please, don't take away the flexibility!

[ssrobins]

As for CMAKE_MAKE_PROGRAM, I'm not setting that in my raw CMake commands, nor in a Conan 1.x recipe. It's just using what's in my environment PATH, as a result of installing Ninja. I'm not sure why it's not working in Conan 2.x. It's probably fixable, though...

I'd like to avoid profiles if I can, it's just one more thing I have to handle getting installed in the developer and automation environments. I want someone to be able to clone my package repo, install Conan, and run a single command to build/package (and do fancier stuff with the various conan commands if the developer chooses). For my apps that consume Conan, I want someone to be able to clone my repo, install Conan, and use CMake, with all the Conan commands being called in CMake. This has all been fantastic in Conan 1.x.

[memsharded]

Would you agree that the developer workflow problem is more of a CMake problem than a Conan problem?

Not really. CMake commands might not be the most ergonomic, but people know them. The problem that Conan introduced is that if you want to build your binary, locally, as a developer, and get the same binary as Conan would build in the cache, you would need to type something like:

cmake -G "Visual Studio 16 2019" -A "x64" -DCONAN_LINK_RUNTIME="/MD" -DCONAN_IN_LOCAL_CACHE="ON" 
-DCONAN_COMPILER="Visual Studio" -DCONAN_COMPILER_VERSION="16" -DCONAN_CXX_FLAGS="/MP8" 
-DCONAN_C_FLAGS="/MP8" -DBUILD_SHARED_LIBS="OFF" 
-DCMAKE_INSTALL_PREFIX="C:\Users\memsharded\.conan\data\hello\0.1\_\_\package\3fb49604f9c2f729b85ba3115852006824e72cab" 
-DCMAKE_INSTALL_BINDIR="bin" 
-DCMAKE_INSTALL_SBINDIR="bin" -DCMAKE_INSTALL_LIBEXECDIR="bin" 
-DCMAKE_INSTALL_LIBDIR="lib" -DCMAKE_INSTALL_INCLUDEDIR="include" -DCMAKE_INSTALL_OLDINCLUDEDIR="include" 
-DCMAKE_INSTALL_DATAROOTDIR="share" 
-DCMAKE_EXPORT_NO_PACKAGE_REGISTRY="ON" -DCONAN_EXPORTED="1" 
-Wno-dev C:\Users\memsharded\.conan\data\hello\0.1\_\_\build\3fb49604f9c2f729b85ba3115852006824e72cab\src
BUILD COMMAND  cmake --build C:\Users\memsharded\.conan\data\hello\0.1\_\_\build\3fb49604f9c2f729b85ba3115852006824e72cab 
--config Release -- /m:8 /verbosity:minimal

This is not acceptable, and it makes a lot of sense that users have been requesting a more idiomatic way to achieve this. This will be replaced with the new toolchain by:

cmake . -G "Visual Studio 16" -DCMAKE_TOOLCHAIN_FILE=conan_toolchain.cmake

At some points we will probably integrate better CMakePresets, but yes, at this moment we are aiming for CMake 3.15 as the supported base version.

Yes, the CMAKE_MAKE_PROGRAM might be a bug or something, we should check it and fix it.

[ssrobins]

Ah, I was not considering the custom paths and settings specific to Conan to get a CMake build result 100% like the Conan build. I agree, that's not acceptable.

Thanks for listening and thanks for considering those two CMakeToolchain additions. Feel free to close this, if it's not being used to track that other potential work. I'll keep working with Conan 2.0 as I see more improvements come out.

[ssrobins]

Finally got Android builds working with Conan 2.0. Got past this error:

Invalid Android STL: None.
CMake Error: CMake was unable to find a build program corresponding to "Ninja Multi-Config".  CMAKE_MAKE_PROGRAM is not set.  You probably need to select a different build tool.

I just had to set compiler.libcxx=c++_static. I didn't realize the Android STL setting came from there. Setting compiler.libcxx=c++_shared works too. I guess CMake was interpreting the early error as a problem finding the make program, hence the second error.

[memsharded]

Ok, great, happy to hear that you got it working!

Lets keep this open, to make sure the above cases are correctly covered, if everything is good, we will close it in 1.38.

[memsharded]

#9075 closed this issue, will be in 1.38

[ssrobins]

I still have to use CMAKE_OSX_DEPLOYMENT_TARGET to get -version-min set properly in macOS and iOS builds in Conan 1.38.

Here's my generate function:

def generate(self):
        tc = CMakeToolchain(self)
        tc.generator = "Ninja Multi-Config"
        tc.generate()
        deps = CMakeDeps(self)
        deps.generate()

macOS configuration:

[settings]
arch=x86_64
arch_build=x86_64
build_type=Debug
compiler=apple-clang
compiler.libcxx=libc++
compiler.version=12.0
os=Macos
os.version=10.9
os_build=Macos

iOS configuration:

[settings]
arch=armv7
arch_build=x86_64
build_type=Debug
compiler=apple-clang
compiler.libcxx=libc++
compiler.version=12.0
os=iOS
os.version=9.0
os_build=Macos

Adding this to the generate function still fixes it:

if self.settings.os == "iOS":
            tc.variables["CMAKE_OSX_DEPLOYMENT_TARGET"] = self.settings.os.version
        elif self.settings.os == "Macos":
            tc.variables["CMAKE_OSX_DEPLOYMENT_TARGET"] = self.settings.os.version

So I'm not blocked, but it sounded like you were interested in implementing this capability. Let me know if I should log a separate item.

[memsharded]

Hi @ssrobins

Yes, please, as the title of this issue is NDK for Android, I think it is better to file a new issue dedicated for it. From your code it seems that it would be:

if self.settings.os == "iOS" or self.settings.os == "Macos":
      # assign CMAKE_OSX_DEPLOYMENT_TARGET

Do you know if it also applies to watchOS or tvOS?

[memsharded]

Just created a new one from your comment in #9282, please track it and provide feedback there. Thanks!