[Caffe2] Android build broken clog - undefined reference to clog_vlog_*

[AliShug]

🐛 Bug

The android build of either libclog.a or libcpuinfo.a seems to be broken somehow. Linking errors for the clog_* functions appear despite the generated libclog.a being linked in my Android project's CMakeLists.

To Reproduce

Steps to reproduce the behavior:

1. Build Caffe2 for Android ./scripts/build_android.sh -DANDROID_ABI=arm64-v8a -DANDROID_TOOLCHAIN=clang
2. Create an empty Android Studio project with C++ support. In this project:
   1. Add built libs under app/libs
   2. Add includes under app/libs/include with built *.pb.h and macros.h
   3. Change CMakeLists.txt and native-lib.cpp to match this Gist
   4. Build the Android project for an arm64-v8a target

Error:

Build command failed.
Error while executing process D:\Dev\Android\SDK\cmake\3.6.4111459\bin\cmake.exe with arguments {--build D:\Documents\AndroidStudioProjects\Caffe2Test\app\.externalNativeBuild\cmake\debug\arm64-v8a --target native-lib}
[1/4] Building C object CMakeFiles/cpufeatures.dir/D_/Dev/Android/Sdk/ndk-bundle/sources/android/cpufeatures/cpu-features.c.o
[2/4] Linking C static library libcpufeatures.a
[3/4] Building CXX object CMakeFiles/native-lib.dir/src/main/cpp/native-lib.cpp.o
[4/4] Linking CXX shared library ..\..\..\..\build\intermediates\cmake\debug\obj\arm64-v8a\libnative-lib.so
FAILED: cmd.exe /C "cd . && D:\Dev\Android\Sdk\ndk-bundle\toolchains\llvm\prebuilt\windows-x86_64\bin\clang++.exe  --target=aarch64-none-linux-android26 --gcc-toolchain=D:/Dev/Android/Sdk/ndk-bundle/toolchains/aarch64-linux-android-4.9/prebuilt/windows-x86_64 --sysroot=D:/Dev/Android/Sdk/ndk-bundle/sysroot -fPIC -isystem D:/Dev/Android/Sdk/ndk-bundle/sysroot/usr/include/aarch64-linux-android -g -DANDROID -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -Wa,--noexecstack -Wformat -Werror=format-security -std=c++11 -std=c++11 -fexceptions -O0 -fno-limit-debug-info  -Wl,--exclude-libs,libgcc.a -Wl,--exclude-libs,libatomic.a -nostdlib++ --sysroot D:/Dev/Android/Sdk/ndk-bundle/platforms/android-26/arch-arm64 -Wl,--build-id -Wl,--warn-shared-textrel -Wl,--fatal-warnings -LD:/Dev/Android/Sdk/ndk-bundle/sources/cxx-stl/llvm-libc++/libs/arm64-v8a -Wl,--no-undefined -Wl,-z,noexecstack -Qunused-arguments -Wl,-z,relro -Wl,-z,now -shared -Wl,-soname,libnative-lib.so -o ..\..\..\..\build\intermediates\cmake\debug\obj\arm64-v8a\libnative-lib.so CMakeFiles/native-lib.dir/src/main/cpp/native-lib.cpp.o  ../../../../libs/libclog.a ../../../../libs/libcaffe2.a ../../../../libs/libcaffe2_protos.a ../../../../libs/libc10.a ../../../../libs/libnnpack.a ../../../../libs/libpthreadpool.a ../../../../libs/libcpuinfo.a ../../../../libs/libprotobuf.a libcpufeatures.a -llog -landroid -ldl -latomic -lm "D:/Dev/Android/Sdk/ndk-bundle/sources/cxx-stl/llvm-libc++/libs/arm64-v8a/libc++_static.a" "D:/Dev/Android/Sdk/ndk-bundle/sources/cxx-stl/llvm-libc++/libs/arm64-v8a/libc++abi.a" && cd ."
../../../../libs/libcpuinfo.a(api.c.o): In function `cpuinfo_log_fatal':
/home/ali/Dev/pytorch/third_party/cpuinfo/src/log.h:17: undefined reference to `clog_vlog_fatal'
../../../../libs/libcpuinfo.a(init.c.o): In function `cpuinfo_log_error':
/home/ali/Dev/pytorch/third_party/cpuinfo/src/log.h:16: undefined reference to `clog_vlog_error'
../../../../libs/libcpuinfo.a(chipset.c.o): In function `cpuinfo_log_error':
/home/ali/Dev/pytorch/third_party/cpuinfo/src/log.h:16: undefined reference to `clog_vlog_error'
../../../../libs/libcpuinfo.a(current.c.o): In function `cpuinfo_log_fatal':
/home/ali/Dev/pytorch/third_party/cpuinfo/src/log.h:17: undefined reference to `clog_vlog_fatal'
../../../../libs/libcpuinfo.a(processors.c.o): In function `cpuinfo_log_error':
/home/ali/Dev/pytorch/third_party/cpuinfo/src/log.h:16: undefined reference to `clog_vlog_error'
../../../../libs/libcpuinfo.a(smallfile.c.o): In function `cpuinfo_log_error':
/home/ali/Dev/pytorch/third_party/cpuinfo/src/log.h:16: undefined reference to `clog_vlog_error'
clang++.exe: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.

You can see above that libclog.a is included in the command which fails.

Expected behavior

Successful linking

Environment

• PyTorch Version (e.g., 1.0): master @6093f294094ad68970ad88d7fbd673a3b7f1908d
• OS: Ubuntu 18.04.1 / Windows 10
• How you installed PyTorch (conda, pip, source): source
• Build command you used (if compiling from source): ./scripts/build_android.sh -DANDROID_ABI=arm64-v8a -DANDROID_TOOLCHAIN=clang on Ubuntu
• Python version: NA
• CUDA/cuDNN version: NA
• GPU models and configuration: NA
• Any other relevant information: Android Studio 3.2.1, NDK 18.1.5063045

Other

I've tried changing the visibility of the functions in clog, with no effect. The symbols are definitely in the lib, as nm build_android/lib/libclog.a shows them all (with T). Is there a way to get cpuinfo compiled with clog linked statically into it to begin with?

Complication - I built Caffe2 on Ubuntu 18.04.1, but I'm building my Android project on Windows 10. Other libraries in the build, including libcpuinfo.a seem to have linked OK; I've been fixing unresolved reference errors as they've come up by adding the corresponding libraries, as it's not otherwise clear which libraries to link. Edit: I've found the same failure occurs linking with Ubuntu/Android Studio.

[AliShug]

libcpuinfo.a obviously leaves these symbols undefined. Not sure if that's intentional.

pytorch$ nm build_android/lib/libcpuinfo.a | grep clog
                 U clog_vlog_fatal
                 U clog_vlog_error
                 U clog_vlog_error
                 U clog_vlog_error
                 U clog_vlog_fatal
                 U clog_vlog_error

I've also confirmed the same failure occurs when using Ubuntu to build the Android Studio project (again 18.04.1, studio 3.2.1 and same NDK version).

[AliShug]

I'm an idiot, clog needs to be linked after cpuinfo. Just had to reorder in target_link_libraries(...) in the android project's CMakeLists.txt.