Android maven resource won't build in Bazel

[CaydenPierce]

Problem

We are trying to use the android maven resource com.alphacephei:vosk-android:0.3.32 within an Android application built by Bazel.

The build is failing when com.alphacephei:vosk-android:0.3.32 is listed as a dependency.

Investigation

The error (below) has to do with vosk-android library not having access to the net.java.dev.jna:jna library during build.

Bazel uses rules_jvm_external to handle maven resources, and this is implemented in our application. I believe rules_jvm_external uses the pom.xml of the Maven resource to determine if a dependency for that library is needed during build. During system build, when com.alphacephei:vosk-android:0.3.32 is desugared in Bazel, net.java.dev.jna:jna library is not made available to it, because net.java.dev.jna:jna is not in com.alphacephei:vosk-android:0.3.32's declared dependencies.

In the Vosk android example jna is imported with Gradle as a dependency of the Android application. However, net.java.dev.jna is not a dependency of the example application (i.e. the users application that imports Vosk), it is a dependency of the vosk-android library, and so should be declared a dependency of that library (in the POM.xml of the Maven artifact).

Config

Bazel uses WORKSPACE file where we import the maven resource:

load("@rules_jvm_external//:defs.bzl", "maven_install")

# Important: there can only be one maven_install rule. Add new maven deps here.
maven_install(
    name = "maven",
    artifacts = [
        "androidx.concurrent:concurrent-futures:1.0.0-alpha03",
        "androidx.lifecycle:lifecycle-common:2.2.0",
        "androidx.annotation:annotation:aar:1.1.0",
        "androidx.appcompat:appcompat:aar:1.1.0-rc01",
        "androidx.camera:camera-core:1.0.0-beta10",
        "androidx.camera:camera-camera2:1.0.0-beta10",
        "androidx.camera:camera-lifecycle:1.0.0-beta10",
        "androidx.constraintlayout:constraintlayout:aar:1.1.3",
        "androidx.core:core:aar:1.1.0-rc03",
        "androidx.legacy:legacy-support-v4:aar:1.0.0",
        "androidx.recyclerview:recyclerview:aar:1.1.0-beta02",
        "androidx.test.espresso:espresso-core:3.1.1",
        "com.github.bumptech.glide:glide:4.11.0",
        "com.google.android.material:material:aar:1.0.0-rc01",
        "com.google.code.findbugs:jsr305:3.0.2",
        "com.google.flogger:flogger-system-backend:0.3.1",
        "com.google.flogger:flogger:0.3.1",
        "com.google.guava:guava:27.0.1-android",
        "com.google.guava:listenablefuture:1.0",
        "junit:junit:4.12",
        "org.hamcrest:hamcrest-library:1.3",
        "net.java.dev.jna:jna:aar:5.8.0",
        "com.alphacephei:vosk-android:0.3.32",
    ],
    repositories = [
        "https://jcenter.bintray.com",
        "https://maven.google.com",
        "https://dl.google.com/dl/android/maven2",
        "https://repo1.maven.org/maven2",
        "https://alphacephei.com/maven/",
    ],
    fetch_sources = True,
    version_conflict_policy = "pinned",
)

The android application Bazel BUILD file, where the application is importing those maven resources:

android_binary(
    name = "wearableai",
    srcs = glob(["*.java", "LandmarksTranslate/*.java"]),
    assets = [
        "//mediapipe/graphs/wearable_ai_gpu:wearable_ai_gpu.binarypb",
        "//mediapipe/modules/face_landmark:face_landmark.tflite",
        "//mediapipe/modules/iris_landmark:iris_landmark.tflite",
        "//mediapipe/modules/hand_landmark:hand_landmark.tflite",
        "//mediapipe/modules/facial_emotion_recognition:face_emotion.tflite",
        "//mediapipe/modules/facial_emotion_recognition:test_emotion.tflite",
        "//mediapipe/modules/facial_emotion_recognition:iris_landmark_two.tflite",
        "//mediapipe/modules/hand_landmark:handedness.txt",
        "//mediapipe/modules/holistic_landmark:hand_recrop.tflite",
        "//mediapipe/modules/pose_detection:pose_detection.tflite",
        "//mediapipe/modules/pose_landmark:pose_landmark_upper_body.tflite",
        "//mediapipe/modules/pose_landmark:pose_landmark_full_body.tflite",
        "//mediapipe/graphs/holistic_tracking:holistic_tracking_gpu.binarypb",
        "//mediapipe/modules/face_detection:face_detection_front.tflite",
    ],
    assets_dir = "",
    manifest = "AndroidManifest.xml",
    resource_files = glob(["res/**"]),
    manifest_values = {
        "applicationId": "com.google.mediapipe.apps.wearableai",
        "appName": "Wearable AI",
        "mainActivity": ".MainActivity",
        "cameraFacingFront": "True",
        "binaryGraphName": "wearable_ai_gpu.binarypb",
        "inputVideoStreamName": "input_video",
        "outputVideoStreamName": "output_video",
        "flipFramesVertically": "True",
        "converterNumBuffers": "2",
    },
    multidex = "native",
    deps = [
        ":mediapipe_jni_lib",
        "//mediapipe/framework/formats:landmark_java_proto_lite",
        "//mediapipe/java/com/google/mediapipe/components:android_camerax_helper",
        "//mediapipe/java/com/google/mediapipe/components:android_components",
        "//mediapipe/java/com/google/mediapipe/framework:android_framework",
        "//mediapipe/java/com/google/mediapipe/glutil",
        "//third_party:androidx_appcompat",
        "//third_party:androidx_core",
        "//third_party:androidx_constraint_layout",
        "//third_party:androidx_material",
        "//third_party:androidx_recyclerview",
        "//third_party:opencv",
        "//third_party:androidx_annotation",
        "@maven//:net_java_dev_jna_jna",
        "@maven//:com_alphacephei_vosk_android",
    ],
)

(The //third_party: dependencies are also Maven, I've just made our own calls directly pull from maven here for clarity.)

Error

Run the command:

bazel build -c opt --config=android_arm64 --verbose_failures mediapipe/examples/android/src/java/com/google/mediapipe/apps/wearableai:wearableai --linkopt=-s

And receive error:

Starting local Bazel server and connecting to it...
INFO: SHA256 (https://github.com/bazelbuild/rules_foreign_cc/archive/refs/tags/0.0.9.zip) = 9f96cae32dca0578960468a5a9f66e1ddb8cba8cc210e3e8198f70a20dba45d1
DEBUG: Rule 'rules_foreign_cc' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "9f96cae32dca0578960468a5a9f66e1ddb8cba8cc210e3e8198f70a20dba45d1"
DEBUG: Repository rules_foreign_cc instantiated at:
  /home/cayden/biggem/Tester/WearableIntelligenceSystem/android_smart_phone/main/WORKSPACE:45:13: in <toplevel>
Repository rule http_archive defined at:
  /home/cayden/.cache/bazel/_bazel_cayden/568d99cd4ee70f624f9d528fbac94170/external/bazel_tools/tools/build_defs/repo/http.bzl:336:31: in <toplevel>
INFO: SHA256 (https://github.com/bazelbuild/rules_cc/archive/main.zip) = 7e3caad112850996a9c6d78b9f7593d01c030ca50872884cad35e7caf754497c
DEBUG: Rule 'rules_cc' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "7e3caad112850996a9c6d78b9f7593d01c030ca50872884cad35e7caf754497c"
DEBUG: Repository rules_cc instantiated at:
  /home/cayden/biggem/Tester/WearableIntelligenceSystem/android_smart_phone/main/WORKSPACE:33:13: in <toplevel>
Repository rule http_archive defined at:
  /home/cayden/.cache/bazel/_bazel_cayden/568d99cd4ee70f624f9d528fbac94170/external/bazel_tools/tools/build_defs/repo/http.bzl:336:31: in <toplevel>
INFO: Analyzed target //mediapipe/examples/android/src/java/com/google/mediapipe/apps/wearableai:wearableai (180 packages loaded, 12170 targets configured).
INFO: Found 1 target...
ERROR: /home/cayden/.cache/bazel/_bazel_cayden/568d99cd4ee70f624f9d528fbac94170/external/maven/BUILD:898:11: Desugaring external/maven/_aar/com_alphacephei_vosk_android/classes_and_libs_merged.jar for Android failed: Worker process did not return a WorkResponse:

---8<---8<--- Start of log, file at /home/cayden/.cache/bazel/_bazel_cayden/568d99cd4ee70f624f9d528fbac94170/bazel-workers/worker-1-Desugar.log ---8<---8<---
Exception in thread "main" java.lang.NoClassDefFoundError: com/sun/jna/PointerType
	at java.base/java.lang.ClassLoader.defineClass1(Native Method)
	at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1016)
	at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:877)
	at com.google.devtools.build.android.desugar.io.HeaderClassLoader.findClass(HeaderClassLoader.java:67)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:588)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
	at java.base/sun.invoke.util.BytecodeDescriptor.parseSig(BytecodeDescriptor.java:95)
	at java.base/sun.invoke.util.BytecodeDescriptor.parseMethod(BytecodeDescriptor.java:60)
	at java.base/sun.invoke.util.BytecodeDescriptor.parseMethod(BytecodeDescriptor.java:45)
	at java.base/java.lang.invoke.MethodType.fromDescriptor(MethodType.java:1134)
	at java.base/java.lang.invoke.MethodType.fromMethodDescriptorString(MethodType.java:1113)
	at com.google.devtools.build.android.desugar.LambdaDesugaring$InvokedynamicRewriter.visitInvokeDynamicInsn(LambdaDesugaring.java:406)
	at org.objectweb.asm.MethodVisitor.visitInvokeDynamicInsn(MethodVisitor.java:462)
	at org.objectweb.asm.MethodVisitor.visitInvokeDynamicInsn(MethodVisitor.java:462)
	at com.google.devtools.build.android.desugar.strconcat.IndyStringConcatDesugaring$IndifiedStringConcatInvocationConverter.visitInvokeDynamicInsn(IndyStringConcatDesugaring.java:141)
	at org.objectweb.asm.MethodVisitor.visitInvokeDynamicInsn(MethodVisitor.java:462)
	at org.objectweb.asm.ClassReader.readCode(ClassReader.java:2451)
	at org.objectweb.asm.ClassReader.readMethod(ClassReader.java:1491)
	at org.objectweb.asm.ClassReader.accept(ClassReader.java:721)
	at com.google.devtools.build.android.desugar.Desugar.desugarClassesInInput(Desugar.java:549)
	at com.google.devtools.build.android.desugar.Desugar.desugarOneInput(Desugar.java:326)
	at com.google.devtools.build.android.desugar.Desugar.desugar(Desugar.java:246)
	at com.google.devtools.build.android.desugar.Desugar.processRequest(Desugar.java:1019)
	at com.google.devtools.build.android.desugar.Desugar.runPersistentWorker(Desugar.java:979)
	at com.google.devtools.build.android.desugar.Desugar.main(Desugar.java:957)
Caused by: java.lang.ClassNotFoundException: Class com.sun.jna.PointerType not found
	at com.google.devtools.build.android.desugar.io.HeaderClassLoader.findClass(HeaderClassLoader.java:53)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:588)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
	... 25 more
---8<---8<--- End of log ---8<---8<---
Target //mediapipe/examples/android/src/java/com/google/mediapipe/apps/wearableai:wearableai failed to build
INFO: Elapsed time: 208.971s, Critical Path: 124.50s
INFO: 5 processes: 3 internal, 2 linux-sandbox.
FAILED: Build did NOT complete successfully

Potential Solution

Update the com.alphacephei:vosk-android:0.3.32 Maven resource to include net.java.dev.jna:jna as a dependency.

[nshmyrev]

I released 0.3.33 with the fix. Please try and comment if needed.

[nshmyrev]

err, 0.3.34 actually

[CaydenPierce]

That worked, awesome. Good work and thanks @nshmyrev

Commit a5d788a solved this issue.

For others who want to build Vosk Android with Bazel, the results of my work will be Open Sourced here: https://github.com/emexlabs/WearableIntelligenceSystem

[CaydenPierce]

This has caused further errors.

While the above solution DID allow vosk to begin building with Bazel, it created issues down the line. I found these further issues working in Bazel and recreated in the original vosk-android-demo to ensure it wasn't a Bazel issue.

In vosk-android-demo, if we update to package 0.3.34, then we get Build error:

uplicate class com.sun.jna.AltCallingConvention found in modules jetified-jna-5.8.0 (net.java.dev.jna:jna:5.8.0) and jetified-jna-5.8.0-runtime (net.java.dev.jna:jna:5.8.0)
Duplicate class com.sun.jna.Callback found in modules jetified-jna-5.8.0 (net.java.dev.jna:jna:5.8.0) and jetified-jna-5.8.0-runtime (net.java.dev.jna:jna:5.8.0)
Duplicate class com.sun.jna.Callback$UncaughtExceptionHandler found in modules jetified-jna-5.8.0 (net.java.dev.jna:jna:5.8.0) and jetified-jna-5.8.0-runtime (net.java.dev.jna:jna:5.8.0)
Duplicate class com.sun.jna.CallbackParameterContext found in modules jetified-jna-5.8.0 (net.java.dev.jna:jna:5.8.0) and jetified-jna-5.8.0-runtime (net.java.dev.jna:jna:5.8.0)
Duplicate class com.sun.jna.CallbackProxy found in modules jetified-jna-5.8.0 (net.java.dev.jna:jna:5.8.0) and jetified-jna-5.8.0-runtime (net.java.dev.jna:jna:5.8.0)
Duplicate class com.sun.jna.CallbackReference found in modules jetified-jna-5.8.0 (net.java.dev.jna:jna:5.8.0) and jetified-jna-5.8.0-runtime (net.java.dev.jna:jna:5.8.0)
Duplicate class com.sun.jna.CallbackReference$AttachOptions found in modules jetified-jna-5.8.0 (net.java.dev.jna:jna:5.8.0) and jetified-jna-5.8.0-runtime (net.java.dev.jna:jna:5.8.0)
Duplicate class com.sun.jna.CallbackReference$DefaultCallbackProxy found in modules jetified-jna-5.8.0 (net.java.dev.jna:jna:5.8.0) and jetified-jna-5.8.0-runtime (net.java.dev.jna:jna:5.8.0)
Duplicate class com.sun.jna.CallbackReference$NativeFunctionHandler found in modules jetified-jna-5.8.0 (net.java.dev.jna:jna:5.8.0) and jetified-jna-5.8.0-runtime (net.java.dev.jna:jna:5.8.0)
...

To solve this, I comment out the jna dependency in app level build.gradle:

dependencies {
    implementation 'androidx.appcompat:appcompat:1.3.1'
    //implementation 'net.java.dev.jna:jna:5.8.0@aar'
    implementation group: 'com.alphacephei', name: 'vosk-android', version: '0.3.34'
    implementation project(':models')
}

This will now Gradle syncs and builds. However it raises the following runtime error (after Gradle or Bazel builds):

   java.lang.UnsatisfiedLinkError: Native library (com/sun/jna/android-aarch64/libjnidispatch.so) not found in resource path (.)
        at com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:962)
        at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:922)
        at com.sun.jna.Native.<clinit>(Native.java:190)
        at com.sun.jna.Native.register(Native.java:1641)
        at org.vosk.LibVosk.<clinit>(LibVosk.java:16)
        at org.vosk.LibVosk.setLogLevel(LibVosk.java:58)
        at org.vosk.demo.VoskActivity.onCreate(VoskActivity.java:72)
        at android.app.Activity.performCreate(Activity.java:8006)
        at android.app.Activity.performCreate(Activity.java:7990)
        at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1329)
        at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3584)
        at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3775)
        at android.app.servertransaction.LaunchActivityItem.execute(LaunchActivityItem.java:85)
        at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:135)
        at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:95)
        at android.app.ActivityThread$H.handleMessage(ActivityThread.java:2246)
        at android.os.Handler.dispatchMessage(Handler.java:106)
        at android.os.Looper.loop(Looper.java:233)
        at android.app.ActivityThread.main(ActivityThread.java:8010)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:631)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:978)

[CaydenPierce]

If I unzip the built apk and search for *.so files, I see:

 cayden@helterserver  ~/biggem/Tester/WearableIntelligenceSystem/android_smart_phone/main/tmp   localize ● ? ⍟9  find . | grep so                                                                           ✔
./lib/armeabi-v7a/libvosk.so
./lib/armeabi-v7a/libmediapipe_jni.so
./lib/armeabi-v7a/libopencv_java3.so
./res/drawable/abc_text_cursor_material.xml
./com/sun/jna/linux-armel/libjnidispatch.so
./com/sun/jna/linux-x86-64/libjnidispatch.so
./com/sun/jna/linux-aarch64/libjnidispatch.so
./com/sun/jna/linux-ppc/libjnidispatch.so
./com/sun/jna/linux-ppc64le/libjnidispatch.so
./com/sun/jna/sunos-sparcv9/libjnidispatch.so
./com/sun/jna/openbsd-x86/libjnidispatch.so
./com/sun/jna/linux-mips64el/libjnidispatch.so
./com/sun/jna/sunos-x86-64/libjnidispatch.so
./com/sun/jna/openbsd-x86-64/libjnidispatch.so
./com/sun/jna/freebsd-x86/libjnidispatch.so
./com/sun/jna/sunos-x86/libjnidispatch.so
./com/sun/jna/linux-riscv64/libjnidispatch.so
./com/sun/jna/linux-arm/libjnidispatch.so
./com/sun/jna/sunos-sparc/libjnidispatch.so
./com/sun/jna/linux-s390x/libjnidispatch.so
./com/sun/jna/freebsd-x86-64/libjnidispatch.so
./com/sun/jna/linux-x86/libjnidispatch.so
./resources.arsc

My architecture in Bazel is set for armeabi-v7a, and the Gradle system has abiFilters, so why are libjnidispatch.so artifacts only being packaged for desktop!?

[CaydenPierce]

For anyone wondering... I did eventually solve this issue. The code will be made public here: https://github.com/emexlabs/WearableIntelligenceSystem and you can reach out to me (caydenpierce.com) if you need a hand.

Thanks again @nshmyrev