Why ML Kit Pose Detection doesn't have GPU Delegate as option ?

[ArtanBerisha1]

I was able to run Pose Detection in my app , but I see that the model is running in CPU , and CPU Load goes to high.
I tried FAST and ACCURATE model , also tried different image size as input, but still they don't reduce the CPU Load to much.
I was wondering why we don't have a gpu delegate option, or how can I implement it ? because this is a tflite model running on background right ? and tflite models can be delegated in GPU (on Android devices).

Thanks

[chrisito]

We don't support GPU acceleration right now, but are working on it. Since the implementation consists of multiple models in a fairly complex pipeline, it is not as simple as enabling the GPU delegate. Also, we want to ensure that we only enable GPU acceleration on the devices that can reliably support it (some that support OpenGL are actually not compatible). Please stay tuned.

[ArtanBerisha1]

Thank you @chrisito for the reply,
I was thinking about this devices that are/aren't compatible with gpu delegate, can you give more details about the devices that are not compatible ? what is the reason they don't work ?

[wy16119]

@chrisito it's great that you are working on GPU acceleration. I wonder what the rough time frame would be for the release?

[boost-app]

is CoreMLDelegate/NNAPI coming for iOS/Android as well? If yes any ETA?

[ksulaimon]

Hey all,

GPU acceleration is currently on our roadmap and we're hoping to launch this early next year. We are exploring CoreMLDelegate/NNAPI, but currently have no plans we'd like to share. Please stay tuned.

[RCVZ]

Any news?

[yangcheng]

are there any update on NNAPI ?

[ai-plays]

Here is a response from another issue about GPU acceleration for Pose: #253 (comment)

"We are planning to enable GPU acceleration, although it is not in our immediate roadmap (likely in Q3)."

Similar case for NNAPI.

[wizecare]

@ksulaimon Hi! Is there any news with GPU acceleration for ML pose estimation?

[ai-plays]

We are actively working on Pose GPU acceleration now. Not guaranteed, but hopefully get something out in Q1 next year.

[wizecare]

Thanks for sharing. Good luck :)

[ai-plays]

Hi all,

We just released Pose GPU support V1 on Android.

On Pixel3XL, with CPU the Pose accurate SDK runs at ~23FPS, while with GPU, it runs ~30FPS.

By default, if you update to the latest SDK version(18.0.0-beta2), GPU will be enabled automatically when appropriate.
More details can be found here: https://developers.google.com/ml-kit/vision/pose-detection/android#hardware_config

Feel free to give it a try.
Any feedback about API, functionality, performance are welcomed.

[WCRohit]

@steven-chengji-yan Till when can we expect GPU support for Pose Detection?

[ai-plays]

Hi WCRohit, pose detection 18.0.0-beta2 uses GPU on applicable devices. (See my comment above for details).

Please give it a try. Looking forward to your feedback.

[WCRohit]

@steven-chengji-yan Sorry for the confusion, I was asking for GPU support for PoseDetection on iOS.

[ai-plays]

We already did some prototyping for iOS Pose GPU but don't have a concrete timeline for landing it at the moment.

The reason why we focus on Android first is because there are more lower or mid-end devices in the Android eco-system, which we believe could benefit more from hardware acceleration, while on iOS, the devices are relatively higher-end and already get decent performance numbers with CPU.

Could you tell us a bit more about your needs for iOS?
What is your use case?
What FPS are you looking for?
What is the current FPS you can get?

[WCRohit]

@steven-chengji-yan Thanks for the quick reply. The use case on our end is Pose Detection of Video Captured along with screen sharing during Video Call.
Normally without ScreenSharing we are getting 25 FPS but when we start screen sharing the CMSampleBuffer starts dropping and get reduced to around 16 FPS.
As ScreenSharing , Video Capture and PoseDetection all uses CPU that is why the drop starts.

That is why thought if Pose Detection can be handled in GPU, It will reduce the drops.
Thanks

[ai-plays]

Closing this issue as the title is about Pose GPU support in general and Android support has been added.

For iOS-specific Pose GPU support, we can follow up in #186

[smithlai]

Can I understand that as of now, only Pose Detection is supported with GPU acceleration?