having issues with the mnet3 branch

[justinjohn0306]

Here's a screenshot of the error i'm facing after the initialization of the mnet3 branch of the repo. The cmd line I used for the test: ./MocapNET2CSV --from con0014/2dJoints_v1.4.csv --ik 0.001 15 30 --mt --show 3 --hands --label s10-p01-c0014-f --seriallength 4 --gdpr

[AmmarkoV]

Unfortunately the error looks like something has internally changed in the way tensorflow restores and executes saved model signatures (?).

My MNET3 code relies on tensorflow CPP bindings :
https://www.tensorflow.org/api_docs/cc
using this wrapper :
https://github.com/FORTH-ModelBasedTracker/MocapNET/blob/master/src/Tensorflow2/tensorflow2.h

The MNET3 training data was produced in 2021 with TF 2.3.1 so this might mean that tensorflow has broken compatibility https://www.tensorflow.org/guide/versions ?

If that is the case the fix would be to :

1. dump the old models to a different format e.g. ONNX and add a wrapper for ONNX execution
2. Retrain the models from scratch and save them using the new convention, however these would be new models
3. somehow Upgrade TF2.3 models to the new version ?

The best I can offer at the time is the python MNET4 backend that has body + hands + face.. :(
https://colab.research.google.com/github/FORTH-ModelBasedTracker/MocapNET/blob/mnet4/mocapnet4.ipynb

Unfortunately if this is a compatibility problem it is hard to fix

[AmmarkoV]

I think there is a problem with the Google Drive model files..!
I reverted back to the ammar.gr version of the files and I think it fixes the issue, please confirm!

but also keep in mind the new version :
https://colab.research.google.com/github/FORTH-ModelBasedTracker/MocapNET/blob/mnet4/mocapnet4.ipynb

[AmmarkoV]

@justinjohn0306 I have also updated the Google Drive files and they work once again.
Will keep the issue open until you confirm!

[justinjohn0306]

@justinjohn0306 I have also updated the Google Drive files and they work once again. Will keep the issue open until you confirm!

It works now but the output joints from the bvh file is messed up, as you can see in the image below

[justinjohn0306]

I think there is a problem with the Google Drive model files..! I reverted back to the ammar.gr version of the files and I think it fixes the issue, please confirm!

but also keep in mind the new version : https://colab.research.google.com/github/FORTH-ModelBasedTracker/MocapNET/blob/mnet4/mocapnet4.ipynb

I did test the mnet4 branch and it wasn't accurate with the pose estimation with the mediapipe module

[ArEnSc]

we are running into the same issues as @justinjohn0306 here

[ArEnSc]

@AmmarkoV

[AmmarkoV]

Since the issue with model package is resolved, and the inverted arms is another issue
I am opening a new issue ticket ( #116 ) and closing this as resolved