Mechanism for using static library on Android

[buzmeg]

I can install my dynamic library libopus.so into my Flutter project by placing it in the android/src/main/jniLibs/arm64-v8a folder. That works fine and DynamicLibrary.open() finds it.

However, if I try to use a static library, "libopus.a" that fails.

Where do I put the ".a" file on Android? And how do I lookup the symbol?

This is the Dart/Flutter code I am using:

final DynamicLibrary lib_opus_native = DynamicLibrary.process(); // This fails on symbol lookup
//final DynamicLibrary lib_opus_native = DynamicLibrary.open("libopus.a");  // This fails immediately
//final DynamicLibrary lib_opus_native = DynamicLibrary.open("libopus.so");  // This succeeds but is pulling in dynamic not static

final int Function(int channels) opus_encoder_get_size = lib_opus_native
    .lookup<NativeFunction<Int32 Function(Int32)>>("opus_encoder_get_size")
    .asFunction();  // This worked for the dynamic library

For those tempted to close this immediately, please note that the case of "Static Library For Android" is missing from:

"Binding to native code using dart:ffi"
https://flutter.dev/docs/development/platform-integration/c-interop

And, having already looked rather extensively through both Google and StackOverflow, it seems like nobody who asked this question actually got any solution.

At minimum, this is a documentation fault. At maximum, this can't be done and is a genuine bug.

I know that I certainly don't have enough knowledge about Flutter to determine which category this falls into.

Refiling because @pedromassangocode closed #75492 immediately and then didn't reopen like he said he would.

Thanks.

[nt4f04uNd]

#33227 looks similar to your issue

[buzmeg]

Maybe. There is a possibility that the Android side is also shaking the tree and wiping out the symbols in the .a file like Xcode does. If so, we'll need a workaround for Android Studio.

However, it's also possible that this needs a bit of build.gradle and CMakeLists.txt magic to pull the .a file into scope.

I don't know. And it doesn't seem like the general programming public knows either. Thus why this was filed as a Flutter bug.

[TahaTesser]

Hi @buzmeg
Can you please provide your flutter doctor -v and flutter run --verbose logs, and a minimal complete reproducible code sample including an example static library in the repository
Thank you

[buzmeg]

Sure. It'll probably take me a day or so to get to it so that I can give you something minimal.

However, right now I'm not going to be able to give you something that needs to bang on cmake as I'm fighting with #75600.

So, the example will have to just be accessing the library from Flutter/Dart without external cpp files.

Thanks.

[buzmeg]

Prefix: Everything I'm doing is on Ubuntu 20.04.2 LTS (uname -a: Linux andrewl-x1 5.4.0-65-generic #73-Ubuntu SMP Mon Jan 18 17:25:17 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux)

Okay, I have attached a flutter project directory which sets up a baseline so that we can talk about what is going on. The project is pretty close to the default "new plugin" project as set up by flutter from the command line.

If you unpack and do "flutter run", the project should compile and work. This demonstrates that basic loading of a dynamic library works as advertised. If this project doesn't work out of the box, we need to figure out what is wrong before proceeding.

Background: At the top level of the project there is a libs_test directory. This directory contains the source code as well as the build commands to produce and install the dynamic libraries as well as the static libraries. That way you can check if I'm doing something wrong in actually producing the libraries themselves if you need to.

There are four libraries that get installed into /android/src/main/jniLibs/: libcomplicated_square_dynamic.so, libcomplicated_square_static.a, libsimple_square_dynamic.so, and libsimple_square_static.a. These are the libraries I'm trying to access.

The "simple" libraries only use clang for compilation. The "complicated" libraries use "clang++" and so invoke the C++ machinery which causes its own issues.

The project should "flutter run" without error as it stands. This loads "libsimple_square_dynamic.so" from the jniLibs area and calls "simple_square(4)" in order to make sure that everything worked right.

There are three flutter-only scenarios that cause bugs and need to be worked out before we can add a ".cpp" file compiled by Flutter/Dart/Gradle/CMake into the mix.

Scenario Number 001:
If you uncomment the code which attempts to invoke "complicated_square", the system crashes. It attempts to invoke C++ runtime behavior and it errors out. Something is going to need to be set somewhere that pulls the appropriate bits of the C++ runtime into scope for Flutter/Dart.

Scenario Number 002:
If you uncomment the code which attempts to access the "simple_square" code via the static library "libsimple_square_dynamic.a" which is also in jniLibs, neither DynamicLibrary.process() nor DynamicLibrary.executable() can pull the symbol into scope. Presumably, there needs to be something which tells the system to grab the *.a file somewhere. I simply don't know where.

Scenario Number 003:
Not listed in the code. But obviously the final test is to pull in the libcomplicated_square_static.a which invokes both the static behavior and the C++ runtime behavior. The will demonstrate that we've got the right changes in the right place and actually understand the underlying mechanisms.

Let me know if you have any further questions.

andr_lib_test_001.tar.gz

[buzmeg]

Sorry, you asked for flutter doctor as well:

$ flutter doctor -v
[✓] Flutter (Channel stable, 1.22.6, on Linux, locale en_US.UTF-8)
    • Flutter version 1.22.6 at /home/andrewl/local/src/flutter
    • Framework revision 9b2d32b605 (3 weeks ago), 2021-01-22 14:36:39 -0800
    • Engine revision 2f0af37152
    • Dart version 2.10.5

[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.3)
    • Android SDK at /home/andrewl/Android/Sdk
    • Platform android-30, build-tools 30.0.3
    • Java binary at: /home/andrewl/local/src/android-studio/jre/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6222593)
    • All Android licenses accepted.

[!] Android Studio
    • Android Studio at /home/andrewl/local/src/android-studio
    ✗ Flutter plugin not installed; this adds Flutter specific functionality.
    ✗ Dart plugin not installed; this adds Dart specific functionality.
    • android-studio-dir = /home/andrewl/local/src/android-studio
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6222593)

[✓] Connected device (1 available)
    • Pixel 4a (mobile) • 0A191JEC202802 • android-arm64 • Android 11 (API 30)

! Doctor found issues in 1 category.

[chinmaygarde]

It is not possible to lookup symbols from a freestanding static library. What you are attempting to do can't be supported. You have to use dynamic library.

The documentation does specify static linking as a necessity before DynamicLibrary.process() may be used. When you are using just the archive in isolation, there is no linking step. So I think the documentation is pretty clear with regards to this already. But this does not seem to be the case. Can you suggest an addition to the documentation to make this clear?

[buzmeg]

Hi, @chinmaygarde

The documentation specifies:

A native library can be linked into an app either dynamically or statically. A statically linked library is embedded into the app’s executable image, and is loaded when the app starts.

The problem is that it doesn't give a procedure to do so.

Do I need to put that .a file somewhere specific so Flutter picks it up and compiles it in? What changes do I need to make to build.gradle or CMakefile.txt to get Flutter to pick up and compile that static library?

That's why I went through all the grief of creating a full project with the cases clearly enumerated so that we can talk about the specifics of "which files to edit and how" instead of a vague "library is embedded".

Thanks.

[buzmeg]

Just a ping to see what the status of this is.

Thanks.

[buzmeg]

cc Flutter tools team: @jonahwilliams @jmagman

Sorry to annoy you folks, but I figure you folks can assign this to the appropriate people. This seems to have been dropped for almost 3 weeks, and I'd like to get it moving again.

Thanks.

[dcharkes]

@buzmeg, at this point we do not have that procedure. Feel free to explore and document it, and make a pull request to the documentation.

What is your use case for wanting to use a statically linked library on Android rather than dynamically linked?

(Related issue: #77252)

[buzmeg]

@dcharkes THANK YOU! Okay, no procedure for static, so I can stop trying to find one.

Primarily, the use case was that it might be easier to integrate a static library on the c++ side than a dynamic library as the linkage was already done and resolved. However, that seems to not be the case. At some point, I will pull the documentation and explicitly note "Here Be Dragons" on static library on Android.

That having been said: the final case is how to build/use the c++ dynamic library with the appropriate machinery and get it called. Currently, when I attempt to do that, the test project I included crashes.

What is the current procedure for using a dynamically linked c++ library?

Thanks.

[dcharkes]

the final case is how to build/use the c++ dynamic library with the appropriate machinery and get it called. Currently, when I attempt to do that, the test project I included crashes.

Please file a new issue for this.

(Note that dart:ffi is a Dart-C FFI, any c++ functions need to be annotated with extern "C" to have the C calling convention.)

[buzmeg]

Okay, I will redo that project to only include the c and c++ dynamic library cases and file a new bug.

[dcharkes]

Static linking on Android will not work because there is no linking step in Android.

One should use a dynamic library instead.

If someone were to have a static library and needs a workaround: You can link the static library into a dynamic library and export all of its symbols. Here's a CMakeLists.txt to accomplish that:

# mylib_staticlib_wrapper.c is empty.
add_library(mylib_staticlib_wrapper SHARED mylib_staticlib_wrapper.c)

set_target_properties(mylib_staticlib_wrapper PROPERTIES
    PUBLIC_HEADER mylib_staticlib.h
    OUTPUT_NAME "mylib_staticlib_wrapper"
)

# Link in the static library, not stripping any symbols.
target_link_libraries(mylib_staticlib_wrapper
    -Wl,--whole-archive
    libmylib_staticlib.a
    -Wl,--no-whole-archive
)

Please note that this will create a dynamic library that is larger than normal dynamic library because --whole-archive includes everything. So if at all possible, build a dynamic library to begin with instead.

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.