What phone support tensorflow lite GPU delegate? #60825

ldfandian · 2023-06-09T08:28:44Z

Click to expand!

Issue Type

Support

Have you reproduced the bug with TF nightly?

No

Source

binary

Tensorflow Version

2.12.0

Custom Code

No

OS Platform and Distribution

Android 13

Mobile device

Android 13

Python version

3.8.3

Bazel version

No response

GCC/Compiler version

No response

CUDA/cuDNN version

No response

GPU model and memory

No response

Current Behaviour?

I tested multiple mobile phones (including Xiaomi 12s Ultra, OnePlus 8, Honor Nova 10, Oppo Reno 9) using the example Android apk at https://github.com/tensorflow/examples/tree/master/lite/examples/image_classification/android . On all of the devices, the app tells "GPU is not supported", but NNAPI and CPU is OK.

Anything I can do to enable the GPU delegate on these Qualcomm Snapdragon devices?

Standalone code to reproduce the issue

the example Android apk at https://github.com/tensorflow/examples/tree/master/lite/examples/image_classification/android

Relevant log output

No response

pjpratik · 2023-06-09T13:00:56Z

Hi @ldfandian

Can you please add the following to the Android manifest in order to detect GPU delegate.

<uses-library android:name="libOpenCL.so"
           android:required="false"/>

<uses-library android:name="libOpenCL-pixel.so"
           android:required="false"/>

Thanks.

ldfandian · 2023-06-09T17:12:30Z

Hi @ldfandian

Can you please add the following to the Android manifest in order to detect GPU delegate.
<uses-library android:name="libOpenCL.so"
           android:required="false"/>

<uses-library android:name="libOpenCL-pixel.so"
           android:required="false"/>
Thanks.

Thanks for the quick response.

I tried to add the following code in AndroidManifest.xml, but it does not work.

    <application
        android:allowBackup="true"
        android:icon="@mipmap/ic_launcher"
        android:label="@string/app_name"
        android:roundIcon="@mipmap/ic_launcher_round"
        android:supportsRtl="true"
        android:taskAffinity=""
        tools:ignore="AllowBackup">

        <uses-native-library
            android:name="libOpenCL.so"
            android:required="false" />
        <uses-native-library
            android:name="libOpenCL-pixel.so"
            android:required="false" />
        <uses-native-library
            android:name="libGLESv2.so"
            android:required="false" />
        <uses-native-library
            android:name="libGLESv3.so"
            android:required="false" />
        <uses-native-library
            android:name="libvulkan.so"
            android:required="false" />
        <uses-native-library
            android:name="libneuralnetworks.so"
            android:required="false" />

        <uses-library
            android:name="libOpenCL.so"
            android:required="false" />
        <uses-library
            android:name="libOpenCL-pixel.so"
            android:required="false" />
        <uses-library
            android:name="libGLESv2.so"
            android:required="false" />
        <uses-library
            android:name="libGLESv3.so"
            android:required="false" />
        <uses-library
            android:name="libvulkan.so"
            android:required="false" />
        <uses-library
            android:name="libneuralnetworks.so"
            android:required="false" />
        ...

Btw, on the Xiaomi 12s Ultra, I checked that the file "/system/vendor/lib64/libOpenCL.so" does exist.
However, it looks like it does not show in the "/system/etc/public.libraries.txt".
Maybe it is the cause? Meaning that the mobile vendor blocks the GPU delegate?

The content of "/system/etc/public.libraries.txt" and "/etc/public.libraries.txt" is like:

# See https://android.googlesource.com/platform/ndk/+/master/docs/PlatformApis.md
libandroid.so
libaaudio.so
libamidi.so
libbinder_ndk.so
libc.so
libcamera2ndk.so
libdl.so
libEGL.so
libGLESv1_CM.so
libGLESv2.so
libGLESv3.so
libicu.so
libicui18n.so
libicuuc.so
libjnigraphics.so
liblog.so
libmediandk.so
libm.so
libnativehelper.so
libnativewindow.so
liblistenjni.qti.so
liblistensoundmodel2.qti.so
libneuralnetworks.so nopreload
libOpenMAXAL.so
libOpenSLES.so
libRS.so
libstdc++.so
libsync.so
libvulkan.so
libwebviewchromium_plat_support.so
libz.so

The content of "/vendor/etc/public.libraries.txt" and "/system/vendor/etc/public.libraries.txt" is like:

libqti-perfd-client.so
libadsprpc.so
libcdsprpc.so
libsdsprpc.so
libfastcvopt.so
libOpenCL.so
libSNPE.so
libmialgo_ai_vision.so
libmialgo_utils.so
libxmi_slow_motion_mein.so

Here is the logcat info:

2023-06-10 01:19:29.175 30415-22047 AdrenoGLES-0            org....examples.imageclassification  I  QUALCOMM build                   : 5e81ec0141, Icc5bd9b9d5
                                                                                                    Build Date                       : 11/07/22
                                                                                                    OpenGL ES Shader Compiler Version: EV031.36.08.11
                                                                                                    Local Branch                     : 
                                                                                                    Remote Branch                    : 
                                                                                                    Remote Branch                    : 
                                                                                                    Reconstruct Branch               : 
2023-06-10 01:19:29.175 30415-22047 AdrenoGLES-0            org....examples.imageclassification  I  Build Config                     : S P 12.1.1 AArch64
2023-06-10 01:19:29.175 30415-22047 AdrenoGLES-0            org....examples.imageclassification  I  Driver Path                      : /vendor/lib64/egl/libGLESv2_adreno.so
2023-06-10 01:19:29.175 30415-22047 AdrenoGLES-0            org....examples.imageclassification  I  Driver Version                   : 0615.50
2023-06-10 01:19:29.181 30415-22047 AdrenoGLES-0            org....examples.imageclassification  I  PFP: 0x01730155, ME: 0x00000000

...

2023-06-10 01:30:26.249  6749-25290 libEGL                  org....examples.imageclassification  E  pre_cache appList: com.sina.weibo,com.ss.android.article.news,com.taobao.taobao,com.smile.gifmaker,com.ss.android.ugc.aweme,com.tencent.mm,tv.danmaku.bili,
...
2023-06-10 01:26:11.511  6746-24015 libEGL                  org....examples.imageclassification  E  call to OpenGL ES API with no current context (logged once per thread)
...

ldfandian · 2023-06-10T07:00:46Z

An update, after adding those uses-library/uses-native-library settings in AndroidManifest.xml, OnePlus 8T works but the other three devices still don't work...

I compared the their content of /system/etc/public.libraries.txt, but find nothing different among these devices.

What's the magic here?

pkgoogle · 2023-06-15T17:15:01Z

Hi @ldfandian,

Have you tried using our acceleration service to see which configuration might be optimal for each of your phones? https://www.tensorflow.org/lite/android/acceleration_service

Have you checked the documentation here: https://www.tensorflow.org/lite/android/delegates/gpu_native ?

Also for more information, are you using an emulator or the actual phones?

ldfandian · 2023-06-16T03:28:29Z

Hi @ldfandian,

Have you tried using our acceleration service to see which configuration might be optimal for each of your phones? https://www.tensorflow.org/lite/android/acceleration_service

Have you checked the documentation here: https://www.tensorflow.org/lite/android/delegates/gpu_native ?

Also for more information, are you using an emulator or the actual phones?

Thanks for the advice.

Good to know it, but my phone does not have a Google play service installed~

pkgoogle · 2023-06-16T16:46:04Z

No worries, @ldfandian did the documentation help at all? Additionally please let us know if you are using emulators or actual phones or some combination of them. Thanks.

ldfandian · 2023-06-17T04:27:03Z

No worries, @ldfandian did the documentation help at all? Additionally please let us know if you are using emulators or actual phones or some combination of them. Thanks.

All of these (Xiaomi 12s Ultra, OnePlus 8, Honor Nova 10, Oppo Reno 9) are real phones sold in China.

pkgoogle · 2023-06-20T16:30:04Z

Hi @sirakiin, can you please take a look?

sirakiin · 2023-06-20T16:39:04Z

Triage to @grantjensen . Could you help on this? Thanks!

grantjensen · 2023-06-26T22:06:59Z

Hi @ldfandian, looking at the GPU's used in these devices (Adreno 730, Adreno 642L) we have full support. I was able to confirm this via the following commands:

bazel build -c opt --config=android_arm64 --copt=-DCL_DELEGATE_NO_GL third_party/tensorflow/lite/tools/benchmark:benchmark_model

adb push blaze-bin/third_party/tensorflow/lite/tools/benchmark/benchmark_model /data/local/tmp

adb push test.tflite /data/local/tmp

adb shell /data/local/tmp/benchmark_model --use_gpu=true --graph=/data/local/tmp/test.tflite

This way we can confirm whether your issue is confined to TFL code or something broader. Please run the above code & let me know how it goes, thanks.

github-actions · 2023-07-05T02:08:15Z

This issue is stale because it has been open for 7 days with no activity. It will be closed if no further activity occurs. Thank you.

github-actions · 2023-07-12T02:08:37Z

This issue was closed because it has been inactive for 7 days since being marked as stale. Please reopen if you'd like to work on this further.

google-ml-butler · 2023-07-12T02:08:42Z

Are you satisfied with the resolution of your issue?
Yes
No

ManishSahu53 · 2023-07-12T06:32:45Z

Actually, I have similar issue too. I cannot do GPU prediction on Phone.

Tensorflow : 2.9
Androind : 13
Phone - Samsung A73 5G

I am getting this flag as False
compatList.isDelegateSupportedOnThisDevice: false

google-ml-butler bot added the type:support Support issues label Jun 9, 2023

google-ml-butler bot assigned SuryanarayanaY Jun 9, 2023

SuryanarayanaY assigned pjpratik and unassigned SuryanarayanaY Jun 9, 2023

pjpratik added comp:lite TF Lite related issues TFLiteGpuDelegate TFLite Gpu delegate issue TF 2.12 For issues related to Tensorflow 2.12 labels Jun 9, 2023

pjpratik added the stat:awaiting response Status - Awaiting response from author label Jun 9, 2023

google-ml-butler bot removed the stat:awaiting response Status - Awaiting response from author label Jun 9, 2023

pjpratik assigned pkgoogle and unassigned pjpratik Jun 15, 2023

pkgoogle added the stat:awaiting response Status - Awaiting response from author label Jun 15, 2023

google-ml-butler bot removed the stat:awaiting response Status - Awaiting response from author label Jun 16, 2023

pkgoogle added the stat:awaiting response Status - Awaiting response from author label Jun 16, 2023

google-ml-butler bot removed the stat:awaiting response Status - Awaiting response from author label Jun 17, 2023

pkgoogle assigned sirakiin Jun 20, 2023

pkgoogle added the stat:awaiting tensorflower Status - Awaiting response from tensorflower label Jun 20, 2023

sirakiin assigned grantjensen Jun 20, 2023

pkgoogle added stat:awaiting response Status - Awaiting response from author and removed stat:awaiting tensorflower Status - Awaiting response from tensorflower labels Jun 27, 2023

github-actions bot added the stale This label marks the issue/pr stale - to be closed automatically if no activity label Jul 5, 2023

github-actions bot closed this as completed Jul 12, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

What phone support tensorflow lite GPU delegate? #60825

What phone support tensorflow lite GPU delegate? #60825

ldfandian commented Jun 9, 2023 •

edited by google-ml-butler bot

Issue Type

Have you reproduced the bug with TF nightly?

Source

Tensorflow Version

Custom Code

OS Platform and Distribution

Mobile device

Python version

Bazel version

GCC/Compiler version

CUDA/cuDNN version

GPU model and memory

Current Behaviour?

Standalone code to reproduce the issue

Relevant log output

pjpratik commented Jun 9, 2023

ldfandian commented Jun 9, 2023 •

edited

ldfandian commented Jun 10, 2023 •

edited

pkgoogle commented Jun 15, 2023

ldfandian commented Jun 16, 2023

pkgoogle commented Jun 16, 2023

ldfandian commented Jun 17, 2023

pkgoogle commented Jun 20, 2023

sirakiin commented Jun 20, 2023

grantjensen commented Jun 26, 2023

github-actions bot commented Jul 5, 2023

github-actions bot commented Jul 12, 2023

google-ml-butler bot commented Jul 12, 2023

ManishSahu53 commented Jul 12, 2023

What phone support tensorflow lite GPU delegate? #60825

What phone support tensorflow lite GPU delegate? #60825

Comments

ldfandian commented Jun 9, 2023 • edited by google-ml-butler bot

Issue Type

Have you reproduced the bug with TF nightly?

Source

Tensorflow Version

Custom Code

OS Platform and Distribution

Mobile device

Python version

Bazel version

GCC/Compiler version

CUDA/cuDNN version

GPU model and memory

Current Behaviour?

Standalone code to reproduce the issue

Relevant log output

pjpratik commented Jun 9, 2023

ldfandian commented Jun 9, 2023 • edited

ldfandian commented Jun 10, 2023 • edited

pkgoogle commented Jun 15, 2023

ldfandian commented Jun 16, 2023

pkgoogle commented Jun 16, 2023

ldfandian commented Jun 17, 2023

pkgoogle commented Jun 20, 2023

sirakiin commented Jun 20, 2023

grantjensen commented Jun 26, 2023

github-actions bot commented Jul 5, 2023

github-actions bot commented Jul 12, 2023

google-ml-butler bot commented Jul 12, 2023

ManishSahu53 commented Jul 12, 2023

ldfandian commented Jun 9, 2023 •

edited by google-ml-butler bot

ldfandian commented Jun 9, 2023 •

edited

ldfandian commented Jun 10, 2023 •

edited