Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Shared library symbols not visible on dlopen()? #201

Closed
fornwall opened this issue Sep 18, 2016 · 3 comments
Closed

Shared library symbols not visible on dlopen()? #201

fornwall opened this issue Sep 18, 2016 · 3 comments
Assignees
@dimitry-

Comments

@fornwall
Copy link

I have a shared library libshared.so containing a function libshared_get_value().

I also have a shared library libplugin.so, which does not link against libshared.so but references the extern function libshared_get_value()

Finally there is an executable linking against libshared.so. At runtime this executable tries to dlopen() the libplugin.so shared library.

This works on "normal" Linux (tested on Ubuntu 16.04).

This does not work on Android (tested on Android 5.0/arm and 6.0/aarch) but fails at runtime with the error message cannot locate symbol "libshared_get_value" referenced by "libplugin.so".

This difference in behaviour causes problems for things like perl and node, where extensions built as shared libraries expects symbols from already loaded libraries to be visible without needing to link against the libraries containing the symbols explicitly. See e.g. How To Link -lperl to Extensions During Build where the following is said:

Thanks to its broken linker, I need to link libperl.so with every extension that uses symbols from it.

Is the Android linker (or tools) really broken in this regard, or is the behaviour the expected/desired one?

See https://github.com/termux/shared-library-testcase for a small test case.
@fornwall fornwall mentioned this issue Sep 18, 2016
Closed
@dimitry-
Copy link
Contributor

tl;dr; This one of the not many differences between linux loader and android loader. On Android only the main executable and LD_PRELOADs are considered to be RTLD_GLOBAL, all the dependencies of the main executable remain RTLD_LOCAL.

Longer explanation:

ld-linux.so marks everything linked against main executable with RTLD_GLOBAL, this is why the example works on Linux. Android loader was not handling RTLD_GLOBAL correctly up until M release. It was fixed in M for things like dlopen/dlsym() but not for the dependencies of the main executable, because it lead to many compatibility problems.

Workaround:

Adding libshared.so dependency to libplugin.so should solve this problem and make the executable.

@dimitry- dimitry- self-assigned this Sep 19, 2016
@fornwall
Copy link
Author

@dimitry- Thanks a lot for the information!

I guess this issue can be closed here as it's not an NDK issue.

@fornwall fornwall mentioned this issue Oct 16, 2016
Closed
fornwall added a commit to termux/termux-packages that referenced this issue Mar 30, 2017
@fornwall
php: Link pgsql extension against libpcre
1a6cff7 
The pgsql extension uses pcre functions but does not link against
libpcre by default, as the php executable which dlopen():s the
extension already links against libpcre.

However, on Android this doesn't work, see
	android/ndk#201
so we need to link against libpcre explicitly.
@fornwall fornwall mentioned this issue Mar 30, 2017
Closed
its-pointless pushed a commit to its-pointless/termux-packages that referenced this issue May 7, 2017
@fornwall @its-pointless
php: Link pgsql extension against libpcre
af5cf37 
The pgsql extension uses pcre functions but does not link against
libpcre by default, as the php executable which dlopen():s the
extension already links against libpcre.

However, on Android this doesn't work, see
	android/ndk#201
so we need to link against libpcre explicitly.
@fornwall fornwall mentioned this issue Jun 22, 2017
Closed
@bluelightning32 bluelightning32 mentioned this issue Oct 28, 2017
Closed
@fornwall fornwall mentioned this issue Nov 29, 2017
Closed
@ghost ghost mentioned this issue Apr 3, 2018
Closed
@Grimler91 Grimler91 mentioned this issue Apr 30, 2018
Merged
@jaimecbernardo jaimecbernardo mentioned this issue May 16, 2018
Open
@tomty89 tomty89 mentioned this issue May 20, 2018
Closed
@tomty89 tomty89 mentioned this issue Jun 7, 2018
Closed
@Grimler91 Grimler91 mentioned this issue Jun 14, 2018
Closed
@fornwall fornwall mentioned this issue Aug 27, 2018
Open
@fornwall fornwall mentioned this issue Oct 2, 2018
Closed
@erlanger erlanger mentioned this issue Nov 14, 2018
Closed
This was referenced Feb 18, 2019
Closed
Closed
@ghost ghost mentioned this issue Apr 10, 2019
Open
@Grimler91 Grimler91 mentioned this issue Apr 13, 2019
Closed
@ghost ghost mentioned this issue Aug 23, 2019
Closed
Kudo added a commit to Kudo/v8-android-buildscripts that referenced this issue Nov 7, 2019
@Kudo
Rename libv8.so to libv8android.so
5ba6d53 
Summary:
    On some Android device, there is libv8.so in system ROM.
    E.g. /system/lib/libv8.so or /vendor/lib/libv8.so

    On old Android, dlopen() with RTLD_GLOBAL is not handled well and it's by default RTLD_LOCAL.
    Even we call System.loadLibrary("v8") before,
    during the time RNV8 call System.loadLibrary("v8executor") and search libv8.so in DT_NEEDED.
    It seems bionic linker will use system libv8.so instead of ours.
    Since our libv8.so is customized with libplatform.so, this leads to unresolved symbol as
    Kudo/react-native-v8#29.

    That's why I am proposing to rename libv8.so as libv8android.so and to prevent name conflict from system libv8.so.

    Reference for RTLD_GLOBAL: android/ndk#201
Kudo added a commit to Kudo/react-native-v8 that referenced this issue Nov 7, 2019
@Kudo
Rename libv8.so to libv8android.so
7ae3b42 
Summary:
    On some Android device, there is libv8.so in system ROM.
    E.g. /system/lib/libv8.so or /vendor/lib/libv8.so

    On old Android, dlopen() with RTLD_GLOBAL is not handled well and it's by default RTLD_LOCAL.
    Even we call System.loadLibrary("v8") before,
    during the time RNV8 call System.loadLibrary("v8executor") and search libv8.so in DT_NEEDED.
    It seems bionic linker will use system libv8.so instead of ours.
    Since our libv8.so is customized with libplatform.so, this leads to unresolved symbol as
    #29.

    That's why I am proposing to rename libv8.so as libv8android.so and to prevent name conflict from system libv8.so.

    Reference for RTLD_GLOBAL: android/ndk#201
Kudo added a commit to Kudo/react-native-v8 that referenced this issue Nov 7, 2019
@Kudo
Rename libv8.so to libv8android.so
5a78dd5 
Summary:
    On some Android device, there is libv8.so in system ROM.
    E.g. /system/lib/libv8.so or /vendor/lib/libv8.so

    On old Android, dlopen() with RTLD_GLOBAL is not handled well and it's by default RTLD_LOCAL.
    Even we call System.loadLibrary("v8") before,
    during the time RNV8 call System.loadLibrary("v8executor") and search libv8.so in DT_NEEDED.
    It seems bionic linker will use system libv8.so instead of ours.
    Since our libv8.so is customized with libplatform.so, this leads to unresolved symbol as
    #29.

    That's why I am proposing to rename libv8.so as libv8android.so and to prevent name conflict from system libv8.so.

    Reference for RTLD_GLOBAL: android/ndk#201
Kudo added a commit to Kudo/react-native-v8 that referenced this issue Nov 7, 2019
@Kudo
Rename libv8.so to libv8android.so
4f00791 
Summary:
    On some Android device, there is libv8.so in system ROM.
    E.g. /system/lib/libv8.so or /vendor/lib/libv8.so

    On old Android, dlopen() with RTLD_GLOBAL is not handled well and it's by default RTLD_LOCAL.
    Even we call System.loadLibrary("v8") before,
    during the time RNV8 call System.loadLibrary("v8executor") and search libv8.so in DT_NEEDED.
    It seems bionic linker will use system libv8.so instead of ours.
    Since our libv8.so is customized with libplatform.so, this leads to unresolved symbol as
    #29.

    That's why I am proposing to rename libv8.so as libv8android.so and to prevent name conflict from system libv8.so.

    Reference for RTLD_GLOBAL: android/ndk#201

(cherry picked from commit 5a78dd5)
Kudo added a commit to Kudo/react-native-v8 that referenced this issue Nov 7, 2019
@Kudo
Rename libv8.so to libv8android.so
f687ded 
Summary:
    On some Android device, there is libv8.so in system ROM.
    E.g. /system/lib/libv8.so or /vendor/lib/libv8.so

    On old Android, dlopen() with RTLD_GLOBAL is not handled well and it's by default RTLD_LOCAL.
    Even we call System.loadLibrary("v8") before,
    during the time RNV8 call System.loadLibrary("v8executor") and search libv8.so in DT_NEEDED.
    It seems bionic linker will use system libv8.so instead of ours.
    Since our libv8.so is customized with libplatform.so, this leads to unresolved symbol as
    #29.

    That's why I am proposing to rename libv8.so as libv8android.so and to prevent name conflict from system libv8.so.

    Reference for RTLD_GLOBAL: android/ndk#201

(cherry picked from commit 5a78dd5)
@ghost ghost mentioned this issue Jan 3, 2020
Merged
@fahadali32
Copy link

I get some error

@ghost ghost mentioned this issue Jun 22, 2020
Closed
@YggdrasiI YggdrasiI mentioned this issue Nov 30, 2020
Closed
@ghost ghost mentioned this issue Feb 11, 2021
Closed
This was referenced Jul 19, 2021
Closed
Closed
@xtkoba xtkoba mentioned this issue Nov 22, 2021
Closed
@xtkoba xtkoba mentioned this issue Dec 4, 2021
Closed
@xtkoba xtkoba mentioned this issue Dec 19, 2021
Merged
This was referenced Feb 4, 2022
Merged
Closed
xtkoba added a commit to termux/termux-packages that referenced this issue Feb 7, 2022
@xtkoba
libtranscript: Add patch for dynamic linking issue
114a715 
as described in android/ndk#201.
nirbheek added a commit to nirbheek/meson that referenced this issue Feb 14, 2022
@nirbheek
shared module: Allow linking on Android
3acf202 
Android requires shared modules that use symbols from other shared
modules to be linked before they can be dlopen()ed in the correct
order. Not doing so leads to a missing symbol error:
android/ndk#201

We need to always allow linking for this. Also add a soname, although
it's not confirmed that it's needed, and it doesn't really hurt if it
isn't needed.
@nirbheek nirbheek mentioned this issue Feb 14, 2022
Merged
eli-schwartz pushed a commit to mesonbuild/meson that referenced this issue Feb 15, 2022
@nirbheek @eli-schwartz
shared module: Allow linking on Android
2d56ff1 
Android requires shared modules that use symbols from other shared
modules to be linked before they can be dlopen()ed in the correct
order. Not doing so leads to a missing symbol error:
android/ndk#201

We need to always allow linking for this. Also add a soname, although
it's not confirmed that it's needed, and it doesn't really hurt if it
isn't needed.
nirbheek added a commit to mesonbuild/meson that referenced this issue Mar 11, 2022
@nirbheek
shared module: Allow linking on Android
2ec2a4e 
Android requires shared modules that use symbols from other shared
modules to be linked before they can be dlopen()ed in the correct
order. Not doing so leads to a missing symbol error:
android/ndk#201

We need to always allow linking for this. Also add a soname, although
it's not confirmed that it's needed, and it doesn't really hurt if it
isn't needed.
@xtkoba xtkoba mentioned this issue Mar 13, 2022
Closed
@xtkoba xtkoba mentioned this issue May 4, 2022
Closed
@Grimler91 Grimler91 mentioned this issue Jun 18, 2022
Closed
@xtkoba xtkoba mentioned this issue Oct 27, 2022
Closed
@xtkoba xtkoba mentioned this issue Nov 19, 2022
Closed
@xtkoba xtkoba mentioned this issue Nov 30, 2022
Closed
@Grimler91 Grimler91 mentioned this issue Jan 1, 2023
Closed
@agoose77 agoose77 mentioned this issue Aug 4, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dimitry- dimitry-

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@fornwall @dimitry- @fahadali32 and others