OpenPose on Video
Based on the great presention from Medium. I've used the model from this Repo and weights from dropbox to create a complete pipeline to add the predicted skeleton from OpenPose on a complete video. I used Skibidi sound clip and this is the result (click to see the video):
I used this clip for several reasons :
- There is fast movements
- There is one or more peoples
- There is costumes
- There is animals
- It's a viral video :)
I used a model from 2017 predicting 17 keypoints (eyes, ears, nose, arms, legs, ...). There is multiple other ones existing more accurate, faster or with more keypoints like this one which propose 135 keypoints. I didn't find everytime how to use or weights that's why I used this one. Now in term or succes or fail we can see :
Missing arm due to constrast : Costume with too big difference with human skeleton : Complexe texture : Non human detection (the model should skip if it's not human) : Same image multiple time, different prediction (due to resolution) : 1 leg not detected due to texture : 2 body merged :
How to use
The code is now quite simple, you have to create some folders to store raw images and processed images. After you have to enter few parameters into variable.py. Once done, just run "create_video.bat".
Using other models more recent could be usefull to see how better they performs on previous images. In addition, this models can't be use on videos Live as it is too slow. It required around 20 minutes to render the 3.13 min of clip (4819 images) in a lower resolution (As it is now, it renders images 1 by 1 - like it it was a live video). Having batches may reduce a lot the process and is quite simple to implement. I'll probably add it in a future update.