[BUG]My compiled .so file depends on other shared object files, but has path prefixes included.

[cyxcw1]

Description

When I use readelf to check the dependencies of my .so file, I found that the libbz2.so file it depends on has a specific path prefix.

greadelf -a /Users/cyxcw1/work-space/csdk_core/csdklib/build/intermediates/library_jni/debug/jni/arm64-v8a/libmylib.so | grep NEED
  [ 5] .gnu.version_r    VERNEED          000000000002fbe0  0002fbe0
 0x0000000000000001 (NEEDED)             Shared library: [libprotobuf.so]
 0x0000000000000001 (NEEDED)             Shared library: [libfolly.so]
 0x0000000000000001 (NEEDED)             Shared library: [/Users/cyxcw1/work-space/csdk_core/csdklib/src/main/cpp/../../../libs/arm64-v8a/libbz2.so]
 0x0000000000000001 (NEEDED)             Shared library: [libfmt.so]
 0x0000000000000001 (NEEDED)             Shared library: [libdouble-conversion.so]
 0x0000000000000001 (NEEDED)             Shared library: [libgflags.so]
 0x0000000000000001 (NEEDED)             Shared library: [libglog.so]
 0x0000000000000001 (NEEDED)             Shared library: [libevent-2.1.so]
 0x0000000000000001 (NEEDED)             Shared library: [libevent_core-2.1.so]
 0x0000000000000001 (NEEDED)             Shared library: [libevent_pthreads-2.1.so]
 0x0000000000000001 (NEEDED)             Shared library: [libssl.so]
 0x0000000000000001 (NEEDED)             Shared library: [libcrypto.so]
 0x0000000000000001 (NEEDED)             Shared library: [libbz2.so]
 0x0000000000000001 (NEEDED)             Shared library: [liblz4.so]
 0x0000000000000001 (NEEDED)             Shared library: [libzstd.so]
 0x0000000000000001 (NEEDED)             Shared library: [libsnappy.so]
 0x0000000000000001 (NEEDED)             Shared library: [libsodium.so]
 0x0000000000000001 (NEEDED)             Shared library: [liblog.so]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so]
 0x0000000000000001 (NEEDED)             Shared library: [libc++_shared.so]
 0x0000000000000001 (NEEDED)             Shared library: [libdl.so]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so]
 0x000000006ffffffe (VERNEED)            0x2fbe0
 0x000000006fffffff (VERNEEDNUM)         2

(The other .so file is ok)
I’m using the latest NDK version: 25.2.9519653, and the problem still exists.

Reproduce: (taking arm64-v8a as an example):

1. Compile the third-party library bzip2 using the tool: https://github.com/leleliu008/ndk-pkg
   1.1 Run “ndk-pkg update” and “ndk-pkg install bzip2 –min-sdk-api-level=21” to compile
   1.2 You can find libbz2.so in .ndk-pkg/install.d/android/21/bzip2/arm64-v8a
2. Import bzip2 in Android Studio’s CMake file.

add_library(bzip2 SHARED IMPORTED)
set_target_properties(bzip2 PROPERTIES IMPORTED_LOCATION
        ${ARM_DIR}/${ANDROID_ABI}/libbz2.so)
...
target_link_libraries(
	mylib
	bzip2
	...
)

With the above steps, the problem can be reproduced.

Affected versions

r23

Canary version

No response

Host OS

Mac

Host OS version

macOS 13.2.1

Affected ABIs

arm64-v8a

Build system

CMake

Other build system

No response

minSdkVersion

21

Device API level

30

[enh-google]

i haven't looked, but i'd guess ndk-pkg is broken. did you use readelf to check the SONAME in the libbz2.so file? i also notice that you have two references to libbz2.so; there's this one too:

 0x0000000000000001 (NEEDED)             Shared library: [libbz2.so]

[cyxcw1]

i haven't looked, but i'd guess ndk-pkg is broken. did you use readelf to check the SONAME in the libbz2.so file? i also notice that you have two references to libbz2.so; there's this one too:

 0x0000000000000001 (NEEDED)             Shared library: [libbz2.so]

I check the SONAME of libbz2.so, have nothing to show:

greadelf -a /Users/cyxcw1/.ndk-pkg/install.d/android/21/bzip2/arm64-v8a/lib-no-versioning/libbz2.so | grep NAME

Is this the reason that cause this bug?

(Sorry, the second libbz2.so is not what I really want to describe. I accidentally manual added this line and it's not actually there.)

[enh-google]

heh, yeah, that's your problem then --- having no SONAME will have the same effect. the linker will use the full pathname to the .so file in that case. you (or rather ndk-pkg) need to use --soname=libbz2.so when building the library so that it has the correct DT_SONAME in its dynamic section. then, when linking your code, the linker will just refer to libbz2.so instead of the full path.

[cyxcw1]

heh, yeah, that's your problem then --- having no SONAME will have the same effect. the linker will use the full pathname to the .so file in that case. you (or rather ndk-pkg) need to use --soname=libbz2.so when building the library so that it has the correct DT_SONAME in its dynamic section. then, when linking your code, the linker will just refer to libbz2.so instead of the full path.

Great. Thanks very much for your help.

[leleliu008]

It seems that https://android.googlesource.com/platform/bionic/+/master/android-changes-for-ndk-developers.md need to be updated.

This document says:

Each ELF shared object (“native library”) must have a SONAME (Shared Object Name) attribute. 
The NDK toolchain adds this attribute by default, so its absence indicates either a misconfigured alternative toolchain or a misconfiguration in your build system.

The NDK toolchain adds this attribute by default, obviously, it's not correct.

Anyway, I fixed this problem at ndk-pkg side, I use patchelf to add DT_SONAME to the sofile after cross-compiling is finished.

https://github.com/leleliu008/ndk-pkg/blob/master/bin/ndk-pkg#L6691-L6705

[enh-google]

The NDK toolchain adds this attribute by default, obviously, it's not correct.

i doubt that, since no-one else is having trouble.

unless proven otherwise, i think ndk-pkg is either providing incorrect arguments to the compiler/linker, or changing the ELF file afterwards (since you mention post-processing with "patchelf").

[leleliu008]

It can be easily proved.

step1: create a C source file named test.c

int test() {
    return 0;
}

step2: create a shared library

~/Downloads/android-ndk-r25c/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android21-clang --shared -o libtest.so test.c

step3: display the dynamic section

~/Downloads/android-ndk-r25c/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-readelf -d libtest.so

There is no DT_SONAME

[enh-google]

yeah, i think the intention here was to say build system. it looks like ndk-build does add -soname:

~/Downloads/android-ndk-r23b$ grep -r -- "-soname"
./build/core/default-build-commands.mk:    -Wl,-soname,$(notdir $(LOCAL_BUILT_MODULE)) \
Binary file ./toolchains/llvm/prebuilt/linux-x86_64/bin/lld matches
Binary file ./toolchains/llvm/prebuilt/linux-x86_64/bin/ld matches
~/Downloads/android-ndk-r23b$ 

i'm guessing cmake does it by default, so we don't need to explicitly say it?

i've sent out https://android-review.googlesource.com/c/platform/bionic/+/2535080 for the bionic dynamic linker docs, and i'll let the NDK folks decide what to do with https://android.googlesource.com/platform/ndk/+/master/docs/BuildSystemMaintainers.md which doesn't currently mention sonames at all...

[leleliu008]

Yes, I guess The NDK toolchain here want to refer to ndk-build not refer to C/C++ Comiper driver. so It's better to explicitly say ndk-build and/or build/cmake/android.toolchain.cmake

[leleliu008]

cmake build system automatically add -Wl,-soname,libxx.so to C/C++ Compiler driver unless NO_SONAME property is set.

Reference: https://cmake.org/cmake/help/latest/prop_tgt/SOVERSION.html

[enh-google]

yeah, i've argued in the past that the linker default (of no soname, rather than soname == basename(output filename)) is an unfortunate historical accident that we should fix there instead, but i don't think anyone has the stomach to try to change that. not even @MaskRay!

[cyxcw1]

heh, yeah, that's your problem then --- having no SONAME will have the same effect. the linker will use the full pathname to the .so file in that case. you (or rather ndk-pkg) need to use --soname=libbz2.so when building the library so that it has the correct DT_SONAME in its dynamic section. then, when linking your code, the linker will just refer to libbz2.so instead of the full path.

I strongly recommend that gradle or ndk should provide us some warning or errors to indicate this case(like: there is no DT_SONAME in xxx.so, replace with the full path), in order to make us to troubleshoot ourselves.

I have checked the gradle log with --debug --info, and seen no useful information.

[enh-google]

well, i don't think gradle has any idea that this is even happening. cmake already does the right thing. i suspect the problem with ndk-build is that (a) it's hard to know whether you're specifying --soname yourself and (b) it's hard to know what the output file name will be?

folks who actually work on ndk-build will know better, but i fear this might be something we either need to change lld's default behavior for, or go back in time and make the ndk-build syntax and behavior more like cmake's. (because there's a risk of breaking existing builds if we change it now.)

[DanAlbert]

I strongly recommend that gradle or ndk should provide us some warning or errors to indicate this case(like: there is no DT_SONAME in xxx.so, replace with the full path), in order to make us to troubleshoot ourselves.

That's #220.

i suspect the problem with ndk-build is

What problem with ndk-build? All I see in the thread says that ndk-build's behavior matches CMake's. ndk-build definitely sets SONAME, otherwise nothing it built would load correctly.

[enh-google]

ndk-build definitely sets SONAME, otherwise nothing it built would load correctly.

yeah, sorry, we confirmed that last week. in which case, yeah, i don't think there's anything else to do here.

[DanAlbert]

https://android-review.git.corp.google.com/c/platform/ndk/+/2541751