Using TFLite GPU

[westracer]

Whenever I try to use tflite with gpu support ('org.tensorflow:tensorflow-lite:0.0.1-gpu-experimental') I get "GpuDelegate: java.lang.ClassNotFoundException" in runtime, though it compiles fine. flutter clean doesn't help.

E/flutter (25136): [ERROR:flutter/shell/platform/android/platform_view_android_jni.cc(40)] java.lang.NoClassDefFoundError: Failed resolution of: Lorg/tensorflow/lite/experimental/GpuDelegate;
E/flutter (25136): 	at sq.flutter.tflite.TflitePlugin.loadModel(TflitePlugin.java:139)
E/flutter (25136): 	at sq.flutter.tflite.TflitePlugin.onMethodCall(TflitePlugin.java:68)
E/flutter (25136): 	at io.flutter.plugin.common.MethodChannel$IncomingMethodCallHandler.onMessage(MethodChannel.java:201)
E/flutter (25136): 	at io.flutter.view.FlutterNativeView$PlatformMessageHandlerImpl.handleMessageFromDart(FlutterNativeView.java:188)
E/flutter (25136): 	at io.flutter.embedding.engine.FlutterJNI.handlePlatformMessage(FlutterJNI.java:202)
E/flutter (25136): 	at android.os.MessageQueue.nativePollOnce(Native Method)
E/flutter (25136): 	at android.os.MessageQueue.next(MessageQueue.java:323)
E/flutter (25136): 	at android.os.Looper.loop(Looper.java:142)
E/flutter (25136): 	at android.app.ActivityThread.main(ActivityThread.java:6238)
E/flutter (25136): 	at java.lang.reflect.Method.invoke(Native Method)
E/flutter (25136): 	at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:904)
E/flutter (25136): 	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:794)

[Cheren15]

Hi, @shaqian
It's a great work! I also tried to apply GPU delegate by adding gpudelegate to .\flutter.pub-cache\hosted\pub.flutter-io.cn\tflite-1.0.4, and also implemented tensorflow-lite-gpu:0.0.0-nightly in dependencies.
However it doesn't seems a huge difference occurs, looking forward to your suggestions!

[shaqian]

Sorry for the delay. I don't have a compatible phone to test GPU. 😂
Looking at the the error, I think the problem may be the path is wrong? Should it be "/org" rather "Lorg" ?
Failed resolution of: Lorg/tensorflow/lite/experimental/GpuDelegate;

@Cheren15 Are you able to get GPU work successfully? I saw discussions previously (if I remember correctly) the GPU version improves certain operations, if your model doesn't use those operations there won't be much difference.

[westracer]

I got GPU delegate to work by removing unnecessary import from my project (I don't remember exactly how, but I had made really silly mistake. I guess I had gradle tensorflow gpu import inside my project, not in plugin). But I couldn't run inference on tiny yolov2 because it has operations that are unsupported by tensorflow lite gpu delegate (if I recall correctly it was MAX operation).

[remcova]

I got GPU delegate to work by removing unnecessary import from my project (I don't remember exactly how, but I had made really silly mistake. I guess I had gradle tensorflow gpu import inside my project, not in plugin). But I couldn't run inference on tiny yolov2 because it has operations that are unsupported by tensorflow lite gpu delegate (if I recall correctly it was MAX operation).

How did you enable/add GPU delegate in your Flutter project?

[shaqian]

GPU delegate is now supported in the latest version 1.1.0. Set useGpuDelegate to true in loadModel.

Thanks,
Qian