Flutter Package?

[charlieforward9]

Hello, just learned about MediaPipe today! It's amazing how many developer resources are open-source. It's also crazy to realize that Google has empowered so much of it. With that being said, I was surprised to see the MediaPipe team has not created a package for Flutter developers since both were founded by Google and the MediaPipe is marketed as cross-platform

System information

• MediaPipe Solution: All solutions available on iOS and Android
• Programming language: Dart
• Are you willing to contribute it: Yes, but my team cannot prioritize this over other tasks.

Will this change the current API?

No. Creating a Flutter package will extend and merge the iOS/Android API such that the new, extended API can be used natively in the Dart language. Nothing about the current API will need to be changed as far as I am concerned.

Who will benefit with this feature?

• Developers and teams using the Flutter framework.
• Developers who want to build application to multiple platforms with one codebase
• Developers who want to do great things with great Google products
• Myself 🫶🏼

[kuaashish]

Hi @charlieforward9,
Thanks for raising this feature request. This looks like an interesting request. We will discuss this feature internally and prioritize it among others.

[szotp-lc]

I have integrated mediapipe into my flutter app but it was really cumbersome. It would be great at least have C api that can be called directly from dart, instead of channel communication separate for each platform.

There is https://github.com/homuler/MediaPipeUnityPlugin/tree/master/mediapipe_api, originally created for Unity but could be repurposed. The problem is - it's not pure C, so generating ffi bindings is not so straightforward, and it is not official.

[charlieforward9]

@szotp-lc

It would be great at least have C api that can be called directly from dart, instead of channel communication separate for each platform.

If it were done this way, would it work on all the mobile and desktop platforms that Flutter supports? If so, then the only separate implementation would be the web? Asking to learn. Thanks

[szotp-lc]

@charlieforward9 Correct.

[schmidt-sebastian]

Hi! We are having some discussion about this, but cannot provide a timeline at this point. We certainly recognize that better Flutter support is very important to large part of our community. We will provide updates if and when we can.

[charlieforward9]

@schmidt-sebastian Thank you for the response. Please do keep us updated in this thread, there seem to be a few who tagged along with interest based on the emoji response.

[charlieforward9]

@szotp-lc do you mind emailing me at charlesrichardsonusa@gmail.com, I would appreciate hearing more about how you integrated into your Flutter app (unless you didn't mind keeping the conversation in this thread). Thank you!

[szotp-lc]

Honestly I would recommend waiting for official support or using a different library (like ML Kit). It has been a painful process.

[charlieforward9]

Honestly I would recommend waiting for official support or using a different library (like ML Kit).

Official Support > .tflite embedding > MLKit MethodChannel

I built a PoseDetection MethodChannel for iOS using MLKit to compare it with my existing pose_detection_full.tflite file embedding using flutter-tflite. The MethodChannel inference delivered slower than the .tflite file embedding with my implementation.

I am sharing this to show findings, save other people time, and keep this relevant.

Goal

---

Actual (Embedded .tflite)

RPReplay_Final1687755288.mov

---

Actual (iOS MLKit Method Channel)

RPReplay_Final1687757878.mov

My implementation uses Dart Isolates to keep the model processing off of the main thread.

[charlieforward9]

Thanks to Google and their Mediapipe team, we are now able to witness development of flutter-mediapipe. It's great to follow along and see how Google would build a package from the ground up.

I have become a watcher of the repository to get a firsthand sight of the progress and changes, so if anyone else lands here and is interested in the progress of the package. I recommend doing the same.

[rukinish]

so as of now is it possible to have a mobile application that is developed using flutter that has mediapipe ?

[charlieforward9]

Yes, there is just a lot of manual setup. I recommend using this sample repo to help get started with it.

I didn't need Android config and I am actually moving to native Swift now for better performance. If you manage to get it up and running with all updated dependencies, feel free to create a PR to Open Source it.. I am waiting for the original creator @JaeHeee to merge it, so we can keep my fork current till it merges.

Happy Coding!