posenet versions and resnet.

[vpenades]

Hi, first of all, thanks for this great repository of tensor flow models!, I am learning Tensor Flow and it's very useful!.

I am trying to use the models from Posenet, and the results I am getting don't look very good compared to what we can see in the online posenet demo.

Also, in the TensorFlow.JS repository, they say they're using the new PoseNet 2.0, which is only available for TensorFlow.JS.... and it comes in two modes: MobileNet and ResNet.

My questions are:

The Posenet models available in your repository, are based on the old Posenet models? or are they based on the new Posenet 2.0 advertised by tensorflow.js?

Could it be possible for you to include the new Posenet ResNet models in your repository?

Thanks in advance!

[PINTO0309]

My model is from old version 1. I didn't know that a version 2 model with high accuracy was released. Thank you for sharing. I need to investigate for a while to see if v2 can be converted and put into my repository. It will take some time.

[vpenades]

Here's the anouncement of the posenet 2.0 models:

https://github.com/tensorflow/tfjs-models/tree/master/posenet#note-weve-just-released-version-20-with-a-new-resnet-model-and-api-check-out-the-new-documentation-below

It seems there's an improved version for Mobilenet, and there's also the new resnet model, which is slower, but highly accurate... looking at the source code, it seems its downloading the models from here: https://github.com/tensorflow/tfjs-models/blob/master/posenet/src/checkpoints.ts#L18

There's also a similar demo for hand and finger tracking:

https://github.com/tensorflow/tfjs-models/tree/master/handpose

Which might be interesting to have too!

[vpenades]

Hi again... after further investigation, it seems some people already succeeded in converting the models to protobuf: https://github.com/atomicbits/posenet-python

[PINTO0309]

Awesome!! The information you share is very useful to other engineers. Thank you again!

[PINTO0309]

The skeleton detection of extraterrestrial life seems to be very difficult.
Test. 1

Test. 2

[vpenades]

Looks nice!, this is with the new posenet 2 models?

[PINTO0309]

The heaviest parameter, Resnet50.

--model resnet50 --stride 16 --quant_bytes 4

[vpenades]

For some reason, I am unable to run that python code.... I think I have a problem with some of the tensor flow dependencies. That's why I was using your repos for downloading the models.... I hope you can upload the .pb files soon!

[PINTO0309]

The .pb (saved_model) alone is ready to commit immediately. We are in the process of converting to tflite.

[vpenades]

Great!

[PINTO0309]

If you're in a hurry, you can download saved_model with this shell script.
https://github.com/PINTO0309/PINTO_model_zoo/blob/master/03_posenet/02_posenet_v2/01_float32/download_saved_model.sh

[vpenades]

Hi again; I've been having some trouble consuming the new posenet saved models produced by your script.

In particular, I'm trying to load them with Emgu.TF, but it complains about invalid format.

I've been inspecting the models with Netron, and I've found some subtle differences;

for example, this is what is typically expected:

And this is what I see in your saved models:

So, maybe setting the format as "TensorFlow Saved Model v1" instead of "TensorFlow Graph" may be causing some differences?

[PINTO0309]

I plan to commit saved_model, freeze_graph, tflite that will solve all your problems within a few hours.

[vpenades]

great!

[PINTO0309]

I have committed to all materials. Unfortunately, only Full Integer Quantization didn't work properly.

Posenet V2 ResNet50 225x225 - 513x513 - tflite

Weight Quantization, Integer Quantization, Float16 Quantization, saved_model, Freeze_graph
https://github.com/PINTO0309/PINTO_model_zoo/tree/master/03_posenet/02_posenet_v2

[vpenades]

Thanks!, the new models are working!

I have a question, though... I've noticed you've converted only the new RestNet models, which is great, since they're the most accurate ones...

But, what about the mobilenet models? The TensorFlow.JS online demo allowed to choose between mobilenet and resnet..... and with the online mobilenet models I got significantly better detection than with the old mobilenet versions.... so maybe the mobilenet versions have been retrained for posenet 2.0? maybe it was just my impression...

Anyway, even if Resnet is much more accurate, mobilenet is still very useful for low end machines.

And, did you take a look at the handpose models?

[PINTO0309]

I was so busy and tired from doing my main business on weekdays that I performed only minimal conversions. There is no other reason not to convert the MobileNet version. rest assured. I have plenty of time to work at home on Saturdays and Sundays, so I plan to convert MobileNet as well.

But, what about the mobilenet models? The TensorFlow.JS online demo allowed to choose between mobilenet and resnet..... and with the online mobilenet models I got significantly better detection than with the old mobilenet versions.... so maybe the mobilenet versions have been retrained for posenet 2.0? maybe it was just my impression...

really? I never expected the MobileNet version to be more accurate. In any case, the conversion process is easy, so I will try it.

And, did you take a look at the handpose models?

Yes. The MediaPipe demo is very interesting. I've been observing well for more than half a year.

[vpenades]

Don´t worry, I'm not in a hurry, actually I believe your responses have been incredibly fast!

And thanks a lot for the hard work!

[PINTO0309]

I plan to commit later today.

Btw, the video below is the result of my trial of 3D Pose Estimation, which is quite interesting.

3D PoseEstimation + OpenVINO + Corei7 CPU only + 720p(1280x720) USB Camera + Sync Real TIme
https://www.youtube.com/watch?v=DgKw0Ty22PE&feature=youtu.be

[PINTO0309]

I have committed a full version of MobileNet/ResNet model. Please update the cloned repository and execute the download script again.