Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MediaPipe Object Detection crashes on Android emulators if API > 30 #5134

Open
BoHellgren opened this issue Feb 10, 2024 · 12 comments
Open

MediaPipe Object Detection crashes on Android emulators if API > 30 #5134

BoHellgren opened this issue Feb 10, 2024 · 12 comments
Assignees
Labels
platform:android Issues with Android as Platform stat:awaiting googler Waiting for Google Engineer's Response task:object detection Issues related to Object detection: Track and label objects in images and video. type:bug Bug in the Source Code of MediaPipe Solution

Comments

@BoHellgren
Copy link

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

Yes

OS Platform and Distribution

Android 11, 12, 13

Mobile device if the issue happens on mobile device

Emulated Pixel 5

Browser and version if the issue happens on browser

No response

Programming Language and version

Kotlin

MediaPipe version

implementation 'com.google.mediapipe:tasks-vision:latest.release'

Bazel version

No response

Solution

Object Detection

Android Studio, NDK, SDK versions (if issue is related to building in Android environment)

No response

Xcode & Tulsi version (if issue is related to building for iOS)

No response

Describe the actual behavior

Fatal exception if API level > 30

Describe the expected behaviour

To work as it does on API level 30

Standalone code/steps you may have used to try to get what you need

D/AndroidRuntime( 7815): Shutting down VM
E/AndroidRuntime( 7815): FATAL EXCEPTION: main
E/AndroidRuntime( 7815): Process: se.ndssoft.mycamx_example, PID: 7815
E/AndroidRuntime( 7815): java.lang.UnsatisfiedLinkError: dlopen failed: library "libmediapipe_tasks_vision_jni.so" not found
E/AndroidRuntime( 7815):        at java.lang.Runtime.loadLibrary0(Runtime.java:1077)
E/AndroidRuntime( 7815):        at java.lang.Runtime.loadLibrary0(Runtime.java:998)
E/AndroidRuntime( 7815):        at java.lang.System.loadLibrary(System.java:1656)
E/AndroidRuntime( 7815):        at com.google.mediapipe.tasks.vision.core.BaseVisionTaskApi.<clinit>(BaseVisionTaskApi.java:37)
E/AndroidRuntime( 7815):        at com.google.mediapipe.tasks.vision.objectdetector.ObjectDetector.createFromOptions(ObjectDetector.java:174)
E/AndroidRuntime( 7815):        at se.ndssoft.mycamx.MycamxPlugin.startCamera$lambda-5(MycamxPlugin.kt:168)

MycamxPlugin.kt:168 follows
objectDetector = ObjectDetector.createFromOptions(mContext, options)

Other info / Complete Logs

My app works on a real Samsung Galaxy S10e, an emulated Pixel 5 API 30 and an emulated Nexus 5 API 30. But it crashes on Pixel 5 API 31 and Nexus 5 API 33 with the message dlopen failed: library "libmediapipe_tasks_vision_jni.so" not found
@BoHellgren BoHellgren added the type:bug Bug in the Source Code of MediaPipe Solution label Feb 10, 2024
@BoHellgren
Copy link
Author

BoHellgren commented Feb 10, 2024

The medipipe/examples/objectdetection app also crashes when run on an emulator with API > 30. It seems that x86 images are OK, but x86_64 images are not. Then this app crashes because libmediapipe_tasks_vision_jni.so cannot be found.
This crash comes at CameraFragment.kt:135

@BoHellgren BoHellgren changed the title ObjectDetector.createFromOptions crashes on Android API > 30 MediaPipe Object Detection crashes on Android emulators if API > 30 Feb 11, 2024
@kuaashish kuaashish added platform:android Issues with Android as Platform task:object detection Issues related to Object detection: Track and label objects in images and video. labels Feb 12, 2024
@kuaashish
Copy link
Collaborator

Hi @schmidt-sebastian,

It appears to be similar to the issue #5100, which has already been reported and is causing crashes on Android emulators. Could you please investigate this issue as well?

Thank you!!!

@kuaashish kuaashish added the stat:awaiting googler Waiting for Google Engineer's Response label Feb 12, 2024
@BoHellgren
Copy link
Author

BoHellgren commented Feb 12, 2024

I downloaded your sample program medipipe/examples/objectdetection and made only two small changes, to app build.gradle:

  • implementation 'com.google.mediapipe:tasks-vision:0.10.9'
  • compileSdkVersion 34 to compileSdk 34 (since compileSdkVersion is deprecated)
    and ran it in AndroidStudio on Pixel5API31 emulator. It still crashes with libmediapipe_tasks_vision_jni.so cannot be found.
    I also tried
  • upgrade AGP to 8.2
  • change ext.kotlin_version = '1.8.0' and ext.java_version = JavaVersion.VERSION_17
  • classpath 'androidx.navigation:navigation-safe-args-gradle-plugin:2.5.0'
    But it made no difference.
    Haven't you tested your own sample programs??
2024-02-12 12:52:47.477 32148-32180 AndroidRuntime          com...pipe.examples.objectdetection  E  FATAL EXCEPTION: pool-2-thread-1
Process: com.google.mediapipe.examples.objectdetection, PID: 32148
                                                                                                    java.lang.UnsatisfiedLinkError: dlopen failed: library "libmediapipe_tasks_vision_jni.so" not found
                                                                                                    	at java.lang.Runtime.loadLibrary0(Runtime.java:1077)
                                                                                                    	at java.lang.Runtime.loadLibrary0(Runtime.java:998)
                                                                                                    	at java.lang.System.loadLibrary(System.java:1656)
                                                                                                    	at com.google.mediapipe.tasks.vision.objectdetector.ObjectDetector.<clinit>(ObjectDetector.java:117)
                                                                                                    	at com.google.mediapipe.tasks.vision.objectdetector.ObjectDetector.createFromOptions(ObjectDetector.java:178)
                                                                                                    	at com.google.mediapipe.examples.objectdetection.ObjectDetectorHelper.setupObjectDetector(ObjectDetectorHelper.kt:125)
                                                                                                    	at com.google.mediapipe.examples.objectdetection.ObjectDetectorHelper.<init>(ObjectDetectorHelper.kt:53)
                                                                                                    	at com.google.mediapipe.examples.objectdetection.fragments.CameraFragment.onViewCreated$lambda-3(CameraFragment.kt:133)
                                                                                                    	at com.google.mediapipe.examples.objectdetection.fragments.CameraFragment.$r8$lambda$quxn3nvY7rt-pHwnK2X4H22loyQ(Unknown Source:0)
                                                                                                    	at com.google.mediapipe.examples.objectdetection.fragments.CameraFragment$$ExternalSyntheticLambda11.run(Unknown Source:2)

@BoHellgren
Copy link
Author

If I look at the APK file, both debug and release, there is a libmediapipe_tasks_vision_jni.so for x86, arm64-v8a, and arme-abiv7a, but not for x86_64. Have you for some reason not compiled MediaPipe for this architecture??

This bug stops me from releasing my new app since I cannot test it properly. I would be grateful if you fix this asap.

@BoHellgren
Copy link
Author

BoHellgren commented Feb 12, 2024 via email

@BoHellgren
Copy link
Author

@schmidt-sebastian When can you look at this?

@kuaashish
Copy link
Collaborator

Hi @BoHellgren,

We have identified that using an emulator to run this program might not be ideal due to hardware limitations and the program's reliance on specific hardware features for optimal performance. Using a physical device is highly recommended for the best experience and functionality.

Because, Machine learning programs often require direct interaction with real-world hardware resources to function effectively. Emulators, while valuable for development and testing, may not provide the necessary hardware capabilities for this program to run smoothly. Unfortunately, due to limitations, We can not do much about this issue.

Thank you!!

@kuaashish kuaashish added stat:awaiting response Waiting for user response and removed stat:awaiting googler Waiting for Google Engineer's Response labels Feb 22, 2024
@BoHellgren
Copy link
Author

BoHellgren commented Feb 22, 2024

I agree that testing ML apps in an emulator has its problems. For example, to test your Object Detection sample program, one has to use Extended Controls to mount an image on a wall in the emulated apartment, and then use mouse and QWERASDF keys to move the emulated camera to face the wall with the photo. This does not mean that testing on emulators is useless. On the contrary, unless you have a lot of physical devices, emulators are the only way to test that your app runs on all API levels. Testing on Firebase Test Lab is not an alternative for ML apps, since you then don't have any control of the camera whatsoever.

I am convinced that you can make MediaPipe run on emulators if you want to, and I urge you to fix this. I think many developers will say no thanks to MediaPipe if they hear that you only support testing on real devices.

@google-ml-butler google-ml-butler bot removed the stat:awaiting response Waiting for user response label Feb 22, 2024
@otro678
Copy link

otro678 commented Mar 26, 2024

Unfortunately, due to limitations, We can not do much about this issue.

@kuaashish
When added via gradle mediapipe literally provides x86 library and does not provide x86_64.

It would crash on a real hardware x64 device. This crash is not due emulator limitations, this is due the missing jni library, lol.

Have you even read the issue, or did you just generated your response using ChatGPT?

@kuaashish
Copy link
Collaborator

Hi @schmidt-sebastian,

Could you please have look into this issue?

Thank you!!

@kuaashish kuaashish added the stat:awaiting googler Waiting for Google Engineer's Response label Mar 27, 2024
@Hassaan69
Copy link

Did anyone find any solution to this problem ?

I have tried hand gesture example on s23 ultra running on android 14 and I get the same error basically any device having SDK > 30 gives same error i.e jni file not found.

@aartikov
Copy link

aartikov commented May 28, 2024

@kuaashish @schmidt-sebastian
There are real devices with x86_64 architecture, such as the ASUS ZenFone Series and Lenovo K900. Do you have plans to support these devices?

Additionally, the absence of libmediapipe_tasks_vision_jni.so for x86_64 can result in rejection when publishing to the Google Play Store. What solutions do you suggest for this issue?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
platform:android Issues with Android as Platform stat:awaiting googler Waiting for Google Engineer's Response task:object detection Issues related to Object detection: Track and label objects in images and video. type:bug Bug in the Source Code of MediaPipe Solution
Projects
None yet
Development

No branches or pull requests

6 participants