Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Work around Android 9+ bug with extractNativeLibs flag #4993

Merged
merged 1 commit into from
Aug 10, 2020

Conversation

grendello
Copy link
Contributor

@grendello grendello commented Aug 10, 2020

Fixes: #4983
Context: https://developer.android.com/reference/android/content/pm/ApplicationInfo.html#FLAG_EXTRACT_NATIVE_LIBS

Xamarin.Android has supported the extractNativeLibs
attribute (introduced by Android API 23) on the <application> element
in AndroidManifest.xml since 95ca102,
with a single significant modification in
feb9ea2 after we discovered that
Android build system can set the flag when constructing the APK after we
are done packaging.

feb9ea2 introduced a runtime check to see whether the
FLAG_EXTRACT_NATIVE_LIBS is not set, which meant that the native
libraries are to stay in the APK file and we need to set up our DSO
search paths to point to the inside of APK files instead of the
traditional filesystem location.

However, it appears that Android 10 (API 29, on both devices and in emulators)
and Android 9 (API 28, on just the devices) broke the
FLAG_EXTRACT_NATIVE_LIBS semantics in that it is possible for the flag
to be set (which means libraries are extracted) with the libraries
not extracted from APKs, thus breaking the logic implemented in
feb9ea2.

It is possible that other Android versions are affected as well, which
means we can no longer trust the flag value and need to implement
another way of checking whether the libraries are on the filesystem or
in the APK. The simplest approach is to check for existence of a known
library in the filesystem location, regardless of the API level, and
assume the flag is not set if the shared library is missing. This is
what this commit implements. The check is performed once on application
startup, thus minimizing the performance impact.

Co-Authored by: Jonathan Peppers (@jonathanpeppers)

Fixes: dotnet#4983
Context: https://developer.android.com/reference/android/content/pm/ApplicationInfo.html#FLAG_EXTRACT_NATIVE_LIBS

Xamarin.Android has supported the [extractNativeLibs][0]
attribute (introduced by Android API 23) on the `<application>` element
in AndroidManifest.xml since 95ca102,
with a single significant modification in
feb9ea2 after we discovered that
Android build system can set the flag when constructing the APK after we
are done packaging.

feb9ea2 introduced a runtime check to see whether the
`FLAG_EXTRACT_NATIVE_LIBS` is *not* set, which meant that the native
libraries are to stay in the APK file and we need to set up our DSO
search paths to point to the inside of APK files instead of the
traditional filesystem location.

However, it appears that Android 10 (API 29, on both devices and in emulators)
and Android 9 (API 28, on *just* the devices) broke the
`FLAG_EXTRACT_NATIVE_LIBS` semantics in that it is possible for the flag
to be *set* (which means libraries are *extracted*) with the libraries
not extracted from APKs, thus breaking the logic implemented in
feb9ea2.

It is possible that other Android versions are affected as well, which
means we can no longer trust the flag value and need to implement
another way of checking whether the libraries are on the filesystem or
in the APK.  The simplest approach is to check for existence of a known
library in the filesystem location, regardless of the API level, and
assume the flag is *not* set if the shared library is missing.  This is
what this commit implements.  The check is performed once on application
startup, thus minimizing the performance impact.

[0]: https://developer.android.com/guide/topics/manifest/application-element#extractNativeLibs

Co-Authored by: Jonathan Peppers (@jonathanpeppers)
Comment on lines 15 to +28
void
BasicAndroidSystem::setup_app_library_directories (jstring_array_wrapper& runtimeApks, jstring_array_wrapper& appDirs, int androidApiLevel)
BasicAndroidSystem::detect_embedded_dso_mode (jstring_array_wrapper& appDirs) noexcept
{
if (androidApiLevel < 23 || !is_embedded_dso_mode_enabled ()) {
// appDirs[2] points to the native library directory
simple_pointer_guard<char[]> libmonodroid_path = utils.path_combine (appDirs[2].get_cstr (), "libmonodroid.so");
log_debug (LOG_ASSEMBLY, "Checking if libmonodroid was unpacked to %s", libmonodroid_path.get ());
if (!utils.file_exists (libmonodroid_path)) {
log_debug (LOG_ASSEMBLY, "%s not found, assuming application/android:extractNativeLibs == false", libmonodroid_path.get ());
set_embedded_dso_mode_enabled (true);
} else {
log_debug (LOG_ASSEMBLY, "Native libs extracted to %s, assuming application/android:extractNativeLibs == true", appDirs[2].get_cstr ());
set_embedded_dso_mode_enabled (false);
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a user has AndroidFastDeploymentType=Assemblies::Dexes, to enable fast deployment of native libraries & dexes.

Would this file_exists check also fail in that case?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, because the path refers to the system-created library path, not the override directories. With fastdev we will check the override locations anyway, regardless of this setting.

@grendello
Copy link
Contributor Author

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@jonpryor
Copy link
Member

jonpryor commented Aug 10, 2020

Copy-edited squash-and-merge message:

Fixes: https://github.com/xamarin/xamarin-android/issues/4983

Context: https://developer.android.com/reference/android/content/pm/ApplicationInfo.html#FLAG_EXTRACT_NATIVE_LIBS

Android API-23 added support for
[`//application/@android:extractNativeLibs`][0], which allows an app
to state that native libs within the `.apk` should *not* be extracted
as part of app installation, to save on overall installation size.

Support for `extractNativeLibs` was added in 95ca1025 and feb9ea2a.

Unfortunately, we have determined that on some Android 9 devices and
Android 10 devices & emulators, the [`FLAG_EXTRACT_NATIVE_LIBS`][1]
check used in commit feb9ea2a doesn't work as intended or expected:

	boolean embeddedDSOsEnabled = (runtimePackage.flags & ApplicationInfo.FLAG_EXTRACT_NATIVE_LIBS) == 0;

What we observe on the offending devices is that `embeddedDSOsEnabled`
is False when `//application/@extractNativeLibs` is false, which is
the *opposite* of what we expect.  Consequently, when we *should* be
looking for native libs within the `.apk`, we don't!

The result is that we can't find `libmonodroid.so` ('cause it's in
the `.apk`, not on disk):

	DllImport error loading library '__Internal': 'Could not load library: Library '/system/lib/libmonodroid.so' not found.'.

Note the *path* that is being checked: `/system/lib/libmonodroid.so`,
because we aren't checking within the `.apk`, so we "fallback" to
various other paths, including `/system/lib`, none of which have
`libmonodroid.so`.

The result is that, eventually, the app crashes during startup:

	Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x6f63736d in tid 7607 (…), pid 7607 (…)

It is possible that other Android versions are affected as well, which
means we can no longer trust the flag value and need to implement
another way of checking whether the libraries are on the filesystem or
in the `.apk`.

Work around this Android issue by checking for the presence of a known
library in a filesystem location, regardless of the API level.
Specifically, check for the presence of `libmonodroid.so`.
If `libmonodroid.so` is not found, assume that all libraries should be
loaded from the `.apk`, as if `//application/@extractNativeLibs`=false.

The check is performed once on application startup, thus minimizing the
performance impact.

[0]: https://developer.android.com/guide/topics/manifest/application-element#extractNativeLibs
[1]: https://developer.android.com/reference/android/content/pm/ApplicationInfo#FLAG_EXTRACT_NATIVE_LIBS

Co-Authored by: Jonathan Peppers (@jonathanpeppers)

@jonpryor jonpryor merged commit bf657e8 into dotnet:master Aug 10, 2020
@grendello grendello deleted the embedded-dso-fix branch August 10, 2020 20:22
jonpryor pushed a commit that referenced this pull request Aug 10, 2020
…4993)

Fixes: #4983

Context: https://developer.android.com/reference/android/content/pm/ApplicationInfo.html#FLAG_EXTRACT_NATIVE_LIBS

Android API-23 added support for
[`//application/@android:extractNativeLibs`][0], which allows an app
to state that native libs within the `.apk` should *not* be extracted
as part of app installation, to save on overall installation size.

Support for `extractNativeLibs` was added in 95ca102 and feb9ea2.

Unfortunately, we have determined that on some Android 9 devices and
Android 10 devices & emulators, the [`FLAG_EXTRACT_NATIVE_LIBS`][1]
check used in commit feb9ea2 doesn't work as intended or expected:

	boolean embeddedDSOsEnabled = (runtimePackage.flags & ApplicationInfo.FLAG_EXTRACT_NATIVE_LIBS) == 0;

What we observe on the offending devices is that `embeddedDSOsEnabled`
is False when `//application/@extractNativeLibs` is false, which is
the *opposite* of what we expect.  Consequently, when we *should* be
looking for native libs within the `.apk`, we don't!

The result is that we can't find `libmonodroid.so` ('cause it's in
the `.apk`, not on disk):

	DllImport error loading library '__Internal': 'Could not load library: Library '/system/lib/libmonodroid.so' not found.'.

Note the *path* that is being checked: `/system/lib/libmonodroid.so`,
because we aren't checking within the `.apk`, so we "fallback" to
various other paths, including `/system/lib`, none of which have
`libmonodroid.so`.

The result is that, eventually, the app crashes during startup:

	Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x6f63736d in tid 7607 (…), pid 7607 (…)

It is possible that other Android versions are affected as well, which
means we can no longer trust the flag value and need to implement
another way of checking whether the libraries are on the filesystem or
in the `.apk`.

Work around this Android issue by checking for the presence of a known
library in a filesystem location, regardless of the API level.
Specifically, check for the presence of `libmonodroid.so`.
If `libmonodroid.so` is not found, assume that all libraries should be
loaded from the `.apk`, as if `//application/@extractNativeLibs`=false.

The check is performed once on application startup, thus minimizing the
performance impact.

[0]: https://developer.android.com/guide/topics/manifest/application-element#extractNativeLibs
[1]: https://developer.android.com/reference/android/content/pm/ApplicationInfo#FLAG_EXTRACT_NATIVE_LIBS

Co-Authored by: Jonathan Peppers (@jonathanpeppers)
jonpryor pushed a commit that referenced this pull request Aug 12, 2020
…4993)

Fixes: #4983

Context: https://developer.android.com/reference/android/content/pm/ApplicationInfo.html#FLAG_EXTRACT_NATIVE_LIBS

Android API-23 added support for
[`//application/@android:extractNativeLibs`][0], which allows an app
to state that native libs within the `.apk` should *not* be extracted
as part of app installation, to save on overall installation size.

Support for `extractNativeLibs` was added in 95ca102 and feb9ea2.

Unfortunately, we have determined that on some Android 9 devices and
Android 10 devices & emulators, the [`FLAG_EXTRACT_NATIVE_LIBS`][1]
check used in commit feb9ea2 doesn't work as intended or expected:

	boolean embeddedDSOsEnabled = (runtimePackage.flags & ApplicationInfo.FLAG_EXTRACT_NATIVE_LIBS) == 0;

What we observe on the offending devices is that `embeddedDSOsEnabled`
is False when `//application/@extractNativeLibs` is false, which is
the *opposite* of what we expect.  Consequently, when we *should* be
looking for native libs within the `.apk`, we don't!

The result is that we can't find `libmonodroid.so` ('cause it's in
the `.apk`, not on disk):

	DllImport error loading library '__Internal': 'Could not load library: Library '/system/lib/libmonodroid.so' not found.'.

Note the *path* that is being checked: `/system/lib/libmonodroid.so`,
because we aren't checking within the `.apk`, so we "fallback" to
various other paths, including `/system/lib`, none of which have
`libmonodroid.so`.

The result is that, eventually, the app crashes during startup:

	Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x6f63736d in tid 7607 (…), pid 7607 (…)

It is possible that other Android versions are affected as well, which
means we can no longer trust the flag value and need to implement
another way of checking whether the libraries are on the filesystem or
in the `.apk`.

Work around this Android issue by checking for the presence of a known
library in a filesystem location, regardless of the API level.
Specifically, check for the presence of `libmonodroid.so`.
If `libmonodroid.so` is not found, assume that all libraries should be
loaded from the `.apk`, as if `//application/@extractNativeLibs`=false.

The check is performed once on application startup, thus minimizing the
performance impact.

[0]: https://developer.android.com/guide/topics/manifest/application-element#extractNativeLibs
[1]: https://developer.android.com/reference/android/content/pm/ApplicationInfo#FLAG_EXTRACT_NATIVE_LIBS

Co-Authored by: Jonathan Peppers (@jonathanpeppers)
@github-actions github-actions bot locked and limited conversation to collaborators Jan 24, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

SIGSEGV crash when trying to Debug app with Xamarin.Android SDK 11.0.0.3
3 participants