[LiteRT Play Services] inference on NPU

[leeroyka]

Android
TFLite from Google Play Services
v16.4.0

C or C++ API

How can I inference on an NPU not on Qualciomm devices, how to use the NNAPI delegate? Example on TPU from Google Pixel devices.

only the GPU delegate is available in C++ directory

my gradle:

dependencies {
    implementation libs.play.services.tflite.java
    implementation libs.play.services.tflite.gpu
    implementation libs.play.services.tflite.acceleration.service
    implementation libs.play.services.tflite.impl
}

• directory "tensorflow/light/delegates" didnt exists in Play Services version
• available only TfLiteGpuDelegatePluginCApi

[gaikwadrahul8]

Hi, @leeroyka
Apologize for the delayed response, Thanks for filing this issue. Currently, LiteRT in Google Play Services C/C++ only provides the GPU and XNNPACK delegates at the moment. NNAPI has been deprecated as of Android 15 please refer this official documentation There’s no NNAPI delegate in the Play Services package.

NPU support is limited to Qualcomm AI Engine devices. Google Pixel Tensor delegate is coming in the coming months (Please refer this official documentation) and via the experimental Tensor ML SDK (sign up at https://ai.google.dev/edge/litert/next/tensor_ml_sdk). In the meantime, please use the GPU delegate (gpu_plugin.h) for acceleration. Thank you for your cooperation and patience.

[github-actions]

Marking this issue as stale since it has been open for 7 days with no activity. This issue will be closed if no further activity occurs.

[github-actions]

This issue was closed because it has been inactive for 14 days. Please post a new issue if you need further assistance. Thanks!

[nicman23]

Is there any news from the google tensor sdk?