[BUG] Missing symbol for ANativeWindow_get/setBuffersDataSpace()

[MarijnS95]

Description

While trying out some NDK features, I ran into a two missing symbols that should already be available since API level 28. This is occurring on both an Xperia X IV (Android 13) and Samsung S22 Ultra (Android 14).

java.lang.UnsatisfiedLinkError: dlopen failed: cannot locate symbol "ANativeWindow_getBuffersDataSpace" referenced by "/data/app/~~9JIDvej3rl1VhE9aK8yqfA==/rust.androidnativesurface-HYHuNX7qwiCcvj83Xyt4gg==/base.apk!/lib/arm64-v8a/libandroid_native_surface.so"...

The same occurs when requesting ANativeWindow_setBuffersDataSpace().

Since a repro was requested:

https://github.com/MarijnS95/AndroidNativeSurface/tree/nw-buffers-data-space

This rerpo is written in Rust with mozilla.rust-android-gradle. I can repro the same issue in a native Rust app though.

Other functions on ANativeWindow operate fine:

https://cs.android.com/android/platform/superproject/+/android-13.0.0_r74:frameworks/native/libs/nativewindow/ANativeWindow.cpp

Function import looks as follows (sorry, cannot link a line here):

https://docs.rs/crate/ndk-sys/latest/source/src/ffi_aarch64.rs

Affected versions

r26

Canary version

No response

Host OS

Linux

Host OS version

Arch Linux

Affected ABIs

arm64-v8a

Build system

Other (specify below)

Other build system

Both Mozilla's rust-android-gradle as well as a native library/APK build from https://github.com/rust-mobile/xbuild

minSdkVersion

28, 33, 34

Device API level

33 and 34

[DanAlbert]

idk how this build system works, so I've got no idea: did you link libnativewindow? If you haven't, do that. If you have, it might be that those devices didn't include that API, but we have a CTS test that covers it so that's very unlikely.

In any case, unless that API 28 annotation is actually wrong (and the git history says it isn't: https://googleplex-android.googlesource.com/platform/frameworks/native/+/654f87b3c2b6777ecf53587dba10ecaaacbe3f4e%5E%21/), I don't think it's even possible for this to be an NDK bug. The stubs have it, the headers have it, and you're not using our build systems so that can't be it. It's potentially a build system issue, a device issue, or user error, none of which are in my power to fix. I'm not really sure how else to help. If you find otherwise, lmk and we'll reopen.

[MarijnS95]

Now that is awkward. I was about to ask if these these functions have moved to a separate library, as all other "standard" functions thus far were available from libandroid.so. Indeed, linking libnativewindow.so gives us the missing functions.

Any clue where I can find what goes where? Figured since the linked file contains functions that are available directly, is libandroid.so reexporting some but not all of them?

[MarijnS95]

https://developer.android.com/ndk/guides/stable_apis#android_native_application_apis says:

Library: libnativewindow for more recent Native Window functionality

[DanAlbert]

as all other "standard" functions thus far were available from libandroid.so

Just all the ones you've tried so far. There are quite a few libraries. We actually don't like that it's that way because it would be much more convenient for users to have everything in libandroid, but we can't fix the past, and we haven't come up with a sufficiently clever trick to make the past pretend it's not that way. Our best plan atm is to add stuff to androidx to hide the details from users (and as an added benefit, with a more ergonomic API), but that wouldn't help you since you're effectively trying to build the same "androidx" libraries for rust.

Any clue where I can find what goes where?

No good one, annoyingly. I find them with cs.android.com with the API name filtered by file:.*\.map\.txt. In this case https://cs.android.com/search?q=ANativeWindow_getBuffersDataSpace%20file:.*%5C.map%5C.txt&sq=&ss=android, which found libnativewindow.map.txt. tbqh our doc tooling needs to be replaced entirely but that's just not in the cards.

[MarijnS95]

Looks like every API from NativeWindow and HardwareBuffer is in there; I'll update our crate to forcibly link to libnativewindow if any of it is used 👍
(EDIT: Before first checking that it is available on older devices)

---

Note that we do link a few libraries for various APIs, I just didn't expect it around the "core" NativeWindow that thus far had everything available directly in libandroid.so.

---

Guess I should investigate androidx; that always stuck in my head as "some Java thing".

[DanAlbert]

Guess I should investigate androidx; that always stuck in my head as "some Java thing".

Right now it is (modulo one or two C++ things that doesn't solve this particular problem). It's a plan to fix this kind of thing in androidx. I wouldn't hold your breath. I've been saying that for a very long time and we've never had the bandwidth to do anything about it.

[MarijnS95]

Fine by me. I've never been into androidx / "compat" stuff, and rather just coded natively for Android. Didn't have to deal with API levels back then (and don't now, when writing against the pure C NDK in Rust).

[DanAlbert]

That's the idea. What you're doing yourself is what androidx would handle on your behalf. But androidx would be doing that behind the scenes in a C++ library, so the crates you're working on are probably the appropriate place to do it for rust.

[MarijnS95]

The thing it "would" handle is... linking? We don't have many behavioural API changes in the NDK otherwise, that I recall. Mostly, as brought up before, we need clear annotations for pointer nullability, lifetimes of references, thread-safety guarantees, threading (and much more) information for callbacks. Unless there's semantic differences across API levels there, that "androidx for C++" would hide away?

[DanAlbert]

Unless there's semantic differences across API levels there

That does happen. androidx is also the place where backports of newer APIs could happen in some cases (things like c++ wrapper request )

[What would] that "androidx for C++" would hide away?

The user-unfriendly-but-has-to-be-for-ABI-stability C API. Same reason you have ndk-sys but also ndk in rust. androidx would be able to use safer types (yes yes, I know, it's still not very safe, but it's a lot better) like std::vector and std::string, would include RAII types, that sort of thing.

[MarijnS95]

I'd be very interested to look at C++ wrappers for the various APIs that I had questions about. Perhaps it is able to answer some of my questions regarding e.g. lifetime of registered callbacks (and maybe [lack of] thread safety by usage of mutexes). As long as the types/functions in question have such a wrapper available, of course.

I'd assume "backports of newer APIs" happen by writing JNI against the Java API that typically already exists?

[DanAlbert]

I'd be very interested to look at C++ wrappers for the various APIs that I had questions about. Perhaps it is able to answer some of my questions regarding e.g. lifetime of registered callbacks (and maybe [lack of] thread safety by usage of mutexes). As long as the types/functions in question have such a wrapper available, of course.

Well, if they're ever written, androidx will be the place to look.

I'd assume "backports of newer APIs" happen by writing JNI against the Java API that typically already exists?

In some cases. In others it'd perhaps be adding an alternative implementation directly (AImageDecoder, for example, probably doesn't need the platform APIs at all).

[MarijnS95]

Right, it would have to be written in the first place. My crusade to request improved and standardified the documentation with regards to certain lifetime/threading/safety guarantees would benefit a theoretical C++ androidx too.

[DanAlbert]

FWIW you probably won't find anyone that disagrees with you, but everyone here is short handed so it rarely makes the cut.

[MarijnS95]

For now I've requested improvements, rather than submitting pull-requests. But good to know that submitting e.g. improvements WRT enum typing and taking enums in function parameters and struct fields (as discussed before: only when they have a defined ABI that matches the ABI that's already out in the wild) won't really be considered, unfortunately :(

As said in the other issue, the compilation fix for #1920 still hasn't gained any attention.

[MarijnS95]

And on that note, I just learned today that Google themselves is implementing the exact same bindings as the ndk crate:

https://android-review.googlesource.com/c/platform/frameworks/native/+/2906958

Is the intent to release that at some point as public API, or can we set up some collaboration so that all the effort isn't done twice?

---

And here we find a threading annotation for example:

https://cs.android.com/android/platform/superproject/main/+/main:frameworks/native/libs/nativewindow/rust/src/lib.rs;l=261-273;drc=a0b3a9d8dd9af818988b2aae64cf093f5ea278c5

[DanAlbert]

It can't possibly be a public API. Rust does not have a stable ABI.

Is the intent to release that at some point as public API, or can we set up some collaboration so that all the effort isn't done twice?

I'm not involved with the work. Leave a comment on that CL, or email the author?

[MarijnS95]

It can't possibly be a public API. Rust does not have a stable ABI.

Rust crates are shipped as source code, and (typically) don't need a public stable ABI because they will be compiled on the fly with the project that consumes it. Especially wrapper crates that have no details to hide.

If you want, e.g. to write a C-compatible library in Rust, extern "C" exists to expose symbols.

---

I'll ping the CL to make them aware.

[DanAlbert]

That wouldn't be an OS API then.

[MarijnS95]

No, the crate/CL I was linking to merely wrapped an OS API in a convenient/safe Rust shell. Identical to what we do in our ndk crate.

[DanAlbert]

What's your question then?

[MarijnS95]

You already answered it: ping the CL authors for collaboration. Seems a bit useless to both be doing the exact same thing.