Running Android GPUDelegate in BG thread fails on API Level 31

[SusheelNath]

Environment:

• OS/OS Version: Android API Level 31
• Source Version: master
• Unity Version: 2021.3.13f1

Describe the bug

• Current Unity version for source (master) tf-lite-unity-sample is 2020.3.36f1
• Upgrading that to 2021.3.13f1 and deploying it on Android causes the error to arise.
• TensorflowLite BlazePose does not work anymore with Android API Level 31.
• It works with lower Android API levels.
• On API Level 31, It works with RunBackground Off however with poorer performance.
• This is specific to Android only. It works on iOS.

To Reproduce
Steps to reproduce the behavior:

1. Clone tf-lite-unity-sample master
2. Download Unity 2021.3.13f1
3. Upgrade project and set RunBackground On.
4. Set API Level 31 for Android
5. Deploy .apk and test on Android mobile phone

Expected behavior
TensorFlowLite BlazePose should work, with RunBackground toggle on, implying smoother experience.

Screenshots
(https://user-images.githubusercontent.com/33517815/203538438-22984ef1-a0f9-4987-a3c2-56a8c93a9a1a.jpeg)

[asus4]

Hi @SusheelNath, could you look at #269? Changing the API level from 31 to 30 might fix this issue.

[asus4]

I noticed the new app should be API level 31 or higher to submit an app to the Google Play Store.

Adding the following code to the AndroidManifect.xml might fix the issue

<uses-native-library android:name="libOpenCL.so" android:required="false" />
<uses-native-library android:name="libOpenCL-pixel.so" android:required="false" />

References:
https://stackoverflow.com/questions/71485371/android-12api-level-31-tflitegpudelegate-invoke-gpudelegate-must-run-on-the
https://developer.android.com/about/versions/12/behavior-changes-12#uses-native-library

[SusheelNath]

Hi Koki, Thank you for your response on this issue. I will try this out and keep you posted. Warm regards, Susheel

…

On Mon, Jan 9, 2023 at 10:55 AM Koki Ibukuro ***@***.***> wrote: I noticed the new app should be API level 31 or higher <https://developer.android.com/google/play/requirements/target-sdk> to submit an app to the Google Play Store. Adding the following code to the AndroidManifect.xml might fix the issue <uses-native-library android:name="libOpenCL.so" android:required="false" /> <uses-native-library android:name="libOpenCL-pixel.so" android:required="false" /> References: https://stackoverflow.com/questions/71485371/android-12api-level-31-tflitegpudelegate-invoke-gpudelegate-must-run-on-the https://developer.android.com/about/versions/12/behavior-changes-12#uses-native-library — Reply to this email directly, view it on GitHub <#263 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AH7XB5ZTFF6KO7JWHFCGW3DWRPVCHANCNFSM6AAAAAASI32QFU> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[SusheelNath]

Hi Koki,

We tried with adding the lines to the AndroidManifest.xml file.
The problem still persists, RunBackground = On with API Level 31 - > Not working.

With context as above, this is for Android and with RunBackground = Off with API Level 31 -> it works. On iOS, everything works as intended.

[karahanoguzhan]

Hi @SusheelNath, did you solve the issue?

[SusheelNath]

Hi @karahanoguzhan I wasn't able to unfortunately.

@asus4 Is there any other pathway that I can try out to solving this?

[asus4]

@SusheelNath Sorry for the wait, I'll look into it when I find the time.

[asus4]

Also, I would appreciate any detailed adb log including the Android model name, version etc to determine the issue.

[asus4]

Hi @SusheelNath @karahanoguzhan, I looked into this today.

On my Androids - tested with Google Pixel 6 and Sony Xperia XZ2, adding the <uses-native-library> tags in AndroidManifest.xml seems to fix the issues.

Steps:

In Unity 2021 LTS, the legacy way of creating a file at Plugins/Android/AndroidManifest.xml seems outdated. Here are detailed steps to build it:

1. In the Build Settings window, enable the "Export Project" option and export an Android project instead of building an apk directly from Unity
2. Open unityLibrary/manifests/AndroidManifest.xml and add the following lines:

    <uses-native-library android:name="libOpenCL.so" android:required="false" />
    <uses-native-library android:name="libOpenCL-car.so" android:required="false" />
    <uses-native-library android:name="libOpenCL-pixel.so" android:required="false" />

3. Maybe you need to upgrade the Gradle version to the latest 7.2.1 as I got an error " tag is unknown."
4. Remove the line 'android.enableR8' from the file Gradle Scripts/gradle.properties.

Then, I succeeded in running BlazePose with the background option.

---

I suppose the reason for this error is the fallback system in the Android GpuDelegate. The default priority of the GpuDelegate is OpenCL, then fallback to OpenGLES if OpenCL is not supported on your device. And as of Android API level 31, you have to add the uses-native-library tag to access the OpenCL library. So the GpuDelegate fallbacks to the OpenGLES, and it throws the error as the OpenGLES requires to run in the same thread.

Could you help us if the above steps also fix the issues on your devices? If it solves the error, we can probably make these changes automatically by implementing IPostGenerateGradleAndroidProject.

[SusheelNath]

Hi @asus4 @karahanoguzhan,

The above fix works. Tried it out on Samsung Galaxy S10FE & Samsung Galaxy S21.
Thank you for your efforts in this.

If you could kindly reflect this fix on your repository as well as your UPM packages? It would be a great help not only for our side but also to any other developer who comes across this issue.

Appreciate it and once again, thank you Koki.

Warm regards,
Susheel

[SusheelNath]

Hi @asus4, I have given the performance a try with the fix.

Issue: For API Level 31 and even lesser, the RunBackgroundOn performs worse than RunBackgroundOff.

Findings: This is the same findings with your base repository as well.

Background: Earlier in your repository (potentially before the Tensorflow package upgrades?), it used to be the other way around, where RunBackgroundOn used to perform much better than RunBackgroundOff.

Questions: 1) Is there any changes to be done to your code base for this?
2) Was this intended?
3) Is there any steps that can be done from our side to make the performance better? (with any, RunBackgroundOn/Off)

[asus4]

@SusheelNath Is it only happening on Android? If so, it might be related to #243.

There are no clues right now. If we could determine from which commit this slowdown issue happens, we would have a better indication of the cause.

Anyway, can you open a new issue for the performance issue? It looks like a different issue from the API Level.

[SusheelNath]

@asus4 Hi Koki, I will do so. I will create a new thread to mark the performance issue.

I will let you close this issue yourself so that you can keep track of the fix to be implemented in your repository and the packages.

Appreciate your help on this.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[asus4]

I have found a simpler way worth testing it.

https://lightship.dev/docs/ardk/ardk_fundamentals/building_android.html

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Im-Zapp]

Hi @asus4 , i was wondering if there was a simpler "within unity" solution available for it as I'm facing the same issue after setting target api to 33 as required by Google Play guidelines.

Issue: Works fine in editor but fails to detect body landmarks on Android

[asus4]

Right, would be nice if we could automate the above modification on the Android project in the post-process build.

[Im-Zapp]

That's great, eagerly waiting for it :)

[asus4]

• Support using OpenCL with Android API Level 33 #316

I simplified that process at the PR #316. You only need to do is adding the following tags in your custom AndroidManifest.xml 🎉

<uses-native-library android:name="libOpenCL.so" android:required="false"/>
<uses-native-library android:name="libOpenCL-car.so" android:required="false"/>
<uses-native-library android:name="libOpenCL-pixel.so" android:required="false"/>

[Im-Zapp]

Hi @asus4
Was able to get it to work. Thanks a lot :)