CMake broken with r18rc1

[aberaud]

NDK r18 breaks CMake' Android module
Building native code for Android using CMake is broken, even with the latest CMake version (3.12).

https://gitlab.kitware.com/cmake/cmake/blob/master/Modules/Platform/Android/Determine-Compiler-Standalone.cmake#L7
explicitly makes use of the gcc binary, and fails if it's missing with:
Android: No '*-gcc' compiler found in CMAKE_ANDROID_STANDALONE_TOOLCHAIN

Even after bypassing this, I still have issues when trying to use compiled static libraries on x86_64:
x86_64-linux-android/bin/ld: error: xx: no archive symbol table (run ranlib)
and
error: xx.a(yy.o): requires dynamic R_X86_64_PC32 reloc against 'zz' which may overflow at runtime; recompile with -fPIC

This is probably more a CMake issue or issues with our build configuration, however it might be a good idea not to release r18 while CMake haven't been updated for this change.

[enh]

i'm wondering whether this is another implicit vote in favor of the idea of having symlinks from gcc and g++ to clang and clang++?

in your case, if you add those two symlinks, does your build "just work", or do you need to make extra changes?

[DanAlbert]

We recommend against using the built-in CMake NDK features for exactly this reason. The only supported (by Android) method of using CMake with the NDK is to use the NDK's CMake toolchain file (cmake -DCMAKE_TOOLCHAIN_FILE=$NDK/build/cmake/android.toolchain.cmake ...). There are plans to unify these two implementations (#463), but the engineer that was heading that up left the team. It's something I'll be getting to in r19+ (it gets much simpler after the toolchain improvements that will be coming in that release). Until then, best to stick to the NDK's toolchain file.

Even after bypassing this, I still have issues when trying to use compiled static libraries on x86_64:
x86_64-linux-android/bin/ld: error: xx: no archive symbol table (run ranlib)

I'm guessing CMake conflates "no GCC" and "no binutils", so it ended up using your system's (presumably a Mac?) ar, which does not perform ranlib automatically like GCC's does. Either that, or maybe it's a host arch vs target arch issue?

error: xx.a(yy.o): requires dynamic R_X86_64_PC32 reloc against 'zz' which may overflow at runtime; recompile with -fPIC

Did CMake switch from -fPIC to -fpic at some point? This looks like an issue with the flags passed by CMake.

it might be a good idea not to release r18 while CMake haven't been updated for this change.

We won't hold a release of the NDK while every third-party build system gets their house in order. Especially not when the cause is improper handling of the removal of GCC, which we announced three years ago.

[aberaud]

Trying this now
It mostly work, only the PIC issue remains (no problem building/running this with r17 and lower).

However IMHO linking gcc to clang is not a good idea because gcc is not clang, these are two different compilers with different options and behavior even if they are mostly compatible. Those who '"just want a compiler" use the cc and c++ executables.

[DanAlbert]

i'm wondering whether this is another implicit vote in favor of the idea of having symlinks from gcc and g++ to clang and clang++?

Not symlinks because Windows (and afaik the SDK manager never learned how to unzip a symlink), but we can use wrapper scripts: #758

[enh]

Those who '"just want a compiler" use the cc and c++ executables.

you say that, but apparently cmake hard-coded references to gcc...

[DanAlbert]

Those who '"just want a compiler" use the cc and c++ executables.

you say that, but apparently cmake hard-coded references to gcc...

Yeah, while I completely agree that that would be better, almost no one does this in practice :(

[DanAlbert]

Out of curiosity, why are you using the non-Android supported flavor of CMake support for the NDK rather than ours? Is it just because that's what you came across first, or is there some support we're missing?

[aberaud]

@DanAlbert Thanks for the insights

We mostly used the CMake version because we already generate a cross-compilation CMake toolchain file from our build system, which is then used on every platform (macOS, iOS, Windows etc. and Android up to now), by setting:

CMAKE_SYSTEM_NAME=(Android|Darwin|Windows)
CMAKE_C_COMPILER=...
CMAKE_CXX_COMPILER=...
CMAKE_FIND_ROOT_PATH=$(our build prefix)
...

However I'm now trying to use the NDK' toolchain file and it seems to work fine :-)

[DanAlbert]

However I'm now trying to use the NDK' toolchain file and it seems to work fine :-)

Glad to hear it!

[aberaud]

Fixed in CMake with https://gitlab.kitware.com/cmake/cmake/commit/ca97d4cb5f5872fb63b36a09fff6ee865bbcbb23

[crujose]

Out of curiosity, why are you using the non-Android supported flavor of CMake support for the NDK rather than ours? Is it just because that's what you came across first, or is there some support we're missing?

Are you talking about the cmake toolchain file in the NDK? The main reason I wouldn't use it is because the way Android native code is built has changed so frequently over the past ~five years that it feels like a bad idea to use any "supported" build tools.

Even the arguments to the current make-standalone-toolchain.sh (which is now deprecated if I understand correctly) have breaking changes from the r10 NDK which is only a little over three years old. For example, the --system flag was removed so older build scripts fail if using a modern version of the NDK. That flag didn't even need to be removed and would still be useful in producing packaged standalone toolchains for Windows, Linux, and Mac from a single host OS (assuming you've downloaded the NDK for each host and ran the script in their respective NDKs).

That's not even talking about the "dark ages" when the Eclipse ADT plugin was dropped in favor Android Studio and gradle, but without supporting C/C++ for the first few years (I'm still salty about that).

Android is just one of ~seven platforms that I need to maintain builds for so keeping up-to-date is difficult as it is, and is only made more difficult when "upgrading" breaks things (before I can even compile) which just means more work to figure out what needs to be changed and how.

[DanAlbert]

The main reason I wouldn't use it is because the way Android native code is built has changed so frequently over the past ~five years that it feels like a bad idea to use any "supported" build tools.

The supported methods are ndk-build, the cmake toolchain file, and standalone toolchains. These interfaces have not changed significantly over the past five years. Implementation details have changed, but we always make this as transparent to the user as possible. I'm not sure what about these systems has changed so drastically as to make you believe that it would be the less stable path. Quite the opposite: you'll have the most migration pain if you don't use any of these.

For example, the --system flag was removed so older build scripts fail if using a modern version of the NDK. That flag didn't even need to be removed and would still be useful in producing packaged standalone toolchains for Windows, Linux, and Mac from a single host OS (assuming you've downloaded the NDK for each host and ran the script in their respective NDKs).

Looking through the git history, it looks like this was not an intentional removal. The same pattern appeared in many of our internal scripts that live in the same directory and this was collateral damage as part of a clean up. If you'd filed the bug years ago we'd have fixed it then. r19 makes standalone toolchains unnecessary, so I don't think there's a point in filing it now. If I'm wrong though then please file it.

Seriously, if you're having issues just file bugs. That's what we're here for.