Keypoints Optimization errors are not reduce

[Realdr4g0n]

Keypoint optimization errors are not reduced

Hello. Thanks for sharing your greate job.

However, when I inferenced keypoint fitting as a downstream task on my own dataset, I found that the reproject error and total loss did not decrease from a certain iteration.

To be precise, even when I output the video, I think that the output smpl pose of Pro_HMR and the optimized output pose are so similar that it is difficult to distinguish them with the naked eye.

Is this a limitation of this model? Or am I missing something?

I look forward to your reply.
thank you

[nkolot]

What type of keypoints are you using for fitting? Is it OpenPose? Because I remember you asked a few days ago in Issue #2 .
I suspect that there might be an error there. Also what is the image resolution?

[Realdr4g0n]

Thanks to your reply.

As you think, what I used keypoints format is openpose. And images resolustion are 1280x720 (Images are extracted frame from youtube)

I wonder what "I suspect that there might be an error there" means. Is it not recommended to use openpose?

[nkolot]

Ok, just to confirm: you are using the default code, the one that we provide in this repo that constructs the dataset on-the-fly using a folder with images and a folder with OpenPose.

In this case what I want from you is to send pictures with the regressed shape, the optimized shape and corresponding OpenPose detections. Or if the data is not sensitive you could upload your image/OpenPose folders to Google drive so that I can run it on my system and compare results.

As a sidenote, if the regression results are very close to the solution, then it doesn't take more that 2 or 3 LBFGS iterations for the optimization to converge.

[Realdr4g0n]

All of the confirms you wrote are the same as mine.

However, the result of the regression is not the same as the solution. The meaning of this word is that it produced an unnatural result. Folding of arms or folding of legs occurs, nonetheless the same or similar to the optimization result. (optimization result and the regression result cannot be identified with the naked eye)

Thanks for the reply and sorry for the late reply.

[nkolot]

Unfortunately, without having extra information about your particular testing scenario I cannot help you with the issues you are seeing. It could be because the OpenPose detection are bad, or there could be a very large domain gap with the images that you are testing this on. So I’m closing this issue for now.