Converting BodyPix to TensorFlow Lite

[JakeWade3]

Hello, I was wondering if its possible to convert the BodyPix model to a TensorFlow Lite model for running on Android and iOS. I found a discussion on this GitHub about this

#1280

The issue was closed as it was not possible to convert this model back into a tensorflow saved model, I was wondering if this is now available/possible.

Thanks

[pyu10055]

@tylerzhu-github can you answer this question? thanks.

[tylerzhu-github]

Thanks for adding me Ping!
Thanks Jake for interested in using BodyPix for your project!

For PoseNet, we have released the model in .tflite format here:
Android: https://github.com/tensorflow/examples/tree/master/lite/examples/posenet/android
Coral TPU: https://github.com/google-coral/project-posenet

Here is a contribution from our community users:
https://medium.com/flutter-community/posenet-for-ios-android-and-flutter-using-tensorflow-lite-836788a110c7

You are right that currently tfjs_convertor doesn't support converting the TensorFlow.js GraphModel back to SavedModel format. We will try to release a BodyPix model in TensorFlow Lite format with demos.

Quick question for Ping is: can we host non TensorFlow.js model/assets (e.g. TensorFlow Lite file)?

[pyu10055]

@tylerzhu-github I think we can host the tflite file on TFHub storage.

[JakeWade3]

Thanks for the replies, so reading this there is a TensorFlow Lite model kicking around somewhere but it just needs hosting?

[gdgggithub]

I would also like to have this model data for my project if available pls inform thx a lot in advance!

[ReallyNotARussianSpy]

@tylerzhu-github I would like to use BodyPix in a Python project. Is it possible to get the original TensorFlow model?

[JakeWade3]

Why was this issue closed? As I am waiting on one of you guys / tensor-flow guys to respond, or at least point me where I can get the original model for use with TensorFlow Lite

[darcula1993]

@tylerzhu-github I would like to use BodyPix in a Python project. Is it possible to get the original TensorFlow model?

That would be great! Bodypix and Posenet are truly marvelous project. I would like to use them on other platform on which tfjs is not compatible.

[dallarosa]

I'd love to use this with TFLite for Android so if it would be great if you could release the original tensorflow model or at least the the tflite version

[tylerzhu-github]

Hi @JakeWade3, @gdgggithub, @sreed101, @darcula1993, @dallarosa,

Sorry for the delayed reply!
I agree with what @pyu10055 suggested about hosting the models on TFHub.
Currently I am in the middle of making the corresponding model available via TFHub together with TensorFlow and TensorFlow.js teams. This process might take time.

Thx for the patience!!

Best,
Tyler

[darcula1993]

@tylerzhu-github thank you very much for your reply.
@dallarosa @JakeWade3 @gdgggithub @sreed101
If you need a tf-format bodypix or posenet model, you can use
https://github.com/patlevin/tfjs-to-tf
to convert the tfjs-graph model into tf fronzen graph. You can also extract weights from it if you want to build your own posenet or bodypix.

I convert a mobilenetV1_float_100 posenet tfjs model into a keras h5 model here just in case anyone need it:
posenet.zip

[rthadur]

@JakeWade3 was your question answered ?

[JakeWade3]

@rthadur Not yet, still haven't had any links or information for when the body pix TensorFlow lite model ready.

[rthadur]

@tylerzhu-github gentle ping if you have any update ?

[jernsting]

@darcula1993 could you please explain how you converted the posenet to an hd5 file? I can only see the function to convert the graph model to a tensorflow graph model (.pb file).

[darcula1993]

@darcula1993 could you please explain how you converted the posenet to an hd5 file? I can only see the function to convert the graph model to a tensorflow graph model (.pb file).

Using https://github.com/patlevin/tfjs-to-tf, you can extract weight from tfjs model. Then I build the model architecture using keras and loading weights layer by layer.Since we have layer name, it is not that hard.

[dallarosa]

@tylerzhu-github thank you very much for your reply.
@dallarosa @JakeWade3 @gdgggithub @sreed101
If you need a tf-format bodypix or posenet model, you can use
https://github.com/patlevin/tfjs-to-tf
to convert the tfjs-graph model into tf fronzen graph. You can also extract weights from it if you want to build your own posenet or bodypix.

I convert a mobilenetV1_float_100 posenet tfjs model into a keras h5 model here just in case anyone need it:
posenet.zip

Thanks for this! It should probably do the job for me. I'll check it out :)

[KapoorHitesh]

Can we have some example usage of the boypix model in python?

[ajaichemmanam]

Can we have some example usage of the boypix model in python?

See https://github.com/ajaichemmanam/simple_bodypix_python

[JakeWade3]

Was wondering if there's any updates for this? For BodyPix on Tensorflow Lite?

[mahendran-narayanan]

@KapoorHitesh You can refer this ml5js and p5js example code.
https://github.com/ml5js/ml5-examples/tree/release/p5js/BodyPix

[hegman12]

For my personal project, i had to use bodypix in android so I ended up re implementing a part of bodypix in python( specifically SegmentMultiPersonParts using mobilenet).
You can find the tflite model here .
A demo app here
How to convert tfjs to tflite here
A flutter plugin for bodypix here

Sample results below

Hope this helps!

[josh-nathanson]

@tylerzhu-github Hi Tyler,

Has BodyPix model to a TensorFlow Lite model for iOS development been released yet? Would love to use it for mobile application development

Thanks

[josh-nathanson]

Does anyone have a solution for this?^

[mianaliasjad]

Guys i think instead of BodyPix TensorFlow Lite model we can use Segmentation from TensorFlow lite.

Is there any difference between these two? Because both are performing image segmentation.

[mishan]

@mianaliasjad from my (very limited) experience, BodyPix seems to perform more accurately

[rthadur]

Will be closing this issue as we will be using latest body segmentation model for same purpose.
Please check latest body segmentation model for latest bodypix model demo https://github.com/tensorflow/tfjs-models/tree/master/body-segmentation