No 3D plots in my own data

[sklf]

Hello guys, I'm trying to use your code to estimate 3D poses. My dataset caontains frames from 12 cameras and camera parameters. I think my dataset is bigger than Campus and Shelf. So could you tell me what should I do to improve the performance? I just tried a little data. But the code seems very slow. Besides I can't get 3D pose plot at all. Could you help me?

[JiangWenPL]

First, the inference speed highly depends on 2D detector and pose estimator. You could switch to other 2D detectors if the speed is your concern.
Second, the 3D plot is expected to be appeared in the second window as I remember it. You could set a breakpoint into our code to find what the problem is. You may need to tune the hyper parameters discussed in our paper though we find our method isn't very sensitive to them empirically.

[ahsan3803]

@sklf I think you are running this project in command line interface. As @JiangWenPL said in above comment that 3D plot will appear in separate window.
You should run this project in PyCharm by setting parameters, as in PyCharm you can get better SciView in PyCharm with 3D plots.

[sklf]

First, the inference speed highly depends on 2D detector and pose estimator. You could switch to other 2D detectors if the speed is your concern.
Second, the 3D plot is expected to be appeared in the second window as I remember it. You could set a breakpoint into our code to find what the problem is. You may need to tune the hyper parameters discussed in our paper though we find our method isn't very sensitive to them empirically.

Thanks for your apply. I find the varient 'multi_pose3d' is an empty list in my dataset, which is the reason that I can't see the 3D plot. So I checked the code and found the triangulated points didn't pass the following check:
https://github.com/zju3dv/mvpose/blob/2e61d2845e5d2c9b80ae12be09b8c4b201761df7/src/m_lib/pictorial.pyx#L196

I found the points before the check are very huge so it can't be smaller than the max_length. I'll try to fix this issue. And I will appreciate if you can give me more suggestions.

[siehlema]

I have the same problem with my own custom dataset. If you find a solution, I would really appreciate if you could share it.

Could the cause maybe be a problem with the camera_parameter.pickle? I recorded my extrinsic parameters and had to transfer from Quaternion to Rot+Trans format.

Best,
Martin

[sklf]

I have the same problem with my own custom dataset. If you find a solution, I would really appreciate if you could share it.

Could the cause maybe be a problem with the camera_parameter.pickle? I recorded my extrinsic parameters and had to transfer from Quaternion to Rot+Trans format.

Best,
Martin

You can refer to my answer above https://github.com/zju3dv/mvpose/issues/42#issuecomment-572846853. Maybe it can help you. Besides，I think the limb length is measured in meter. So check your camera parameter scale may help.

[siehlema]

Supplement to my question:

Like I assumed, the problem with my setup were the extrinsic camera parameters. Wrong RT values led to the problem @sklf described above (bone_length check).

[lisa676]

@sklf @siehlema for multi-camera calibration which toolbox you used?

[sklf]

@sklf @siehlema for multi-camera calibration which toolbox you used?

I use this toolbox.

[ahsan3803]

@sklf @siehlema I'm also facing same problem. I also used same calibration tool as @sklf. I read above solution but unable to solve my problem. Can you elaborate more about solution that what should I do to solve it?

[jellyfish1456]

@sklf
Can you elaborate more about solution that what should I do to solve it? The link you provide cannot open.

[sklf]

@sklf
Can you elaborate more about solution that what should I do to solve it? The link you provide cannot open.

After you git clone this project, you can check the 196th line of the $mvposePATH$/src/m_lib/pictorial.pyx. Make sure your results can pass the backbone check. ^_^

[jellyfish1456]

@sklf @JiangWenPL if it does not pass, how to solve it?

I also have another question is that "half body picture(no full length body)" will also affect the result cannot produce 3D plots?

Thank you so much~

[cv1995]

First, the inference speed highly depends on 2D detector and pose estimator. You could switch to other 2D detectors if the speed is your concern.
Second, the 3D plot is expected to be appeared in the second window as I remember it. You could set a breakpoint into our code to find what the problem is. You may need to tune the hyper parameters discussed in our paper though we find our method isn't very sensitive to them empirically.

Thanks for your apply. I find the varient 'multi_pose3d' is an empty list in my dataset, which is the reason that I can't see the 3D plot. So I checked the code and found the triangulated points didn't pass the following check:
https://github.com/zju3dv/mvpose/blob/2e61d2845e5d2c9b80ae12be09b8c4b201761df7/src/m_lib/pictorial.pyx#L196

I found the points before the check are very huge so it can't be smaller than the max_length. I'll try to fix this issue. And I will appreciate if you can give me more suggestions.

Have you solved this problem?

[ghost]

@sklf
can you help make file:

save('intrinsic.mat','K');
save('m_RT.mat', 'm_RT');
save('P.mat', 'P');
save('prjectionMat','P');

when i run
python ./src/tools/mat2pickle.py /parameter/dir ./datasets/CampusSeq1
get error :
No such file or directory: '/parameter/dir/intrinsic.mat'

[sklf]

@sklf
can you help make file:

save('intrinsic.mat','K');
save('m_RT.mat', 'm_RT');
save('P.mat', 'P');
save('prjectionMat','P');

when i run
python ./src/tools/mat2pickle.py /parameter/dir ./datasets/CampusSeq1
get error :
No such file or directory: '/parameter/dir/intrinsic.mat'

Sorry，I‘ve forgotten about the specific details for this repo. But those matrix are the camera parameters which can be obtained by camera calibration. You can refer to author’s code or search for details on wikipedia. I used a toolbox (mentioned above) to get those matrices for my own data. Good luck!

[sklf]

@sklf
can you help make file:

save('intrinsic.mat','K');
save('m_RT.mat', 'm_RT');
save('P.mat', 'P');
save('prjectionMat','P');

when i run
python ./src/tools/mat2pickle.py /parameter/dir ./datasets/CampusSeq1
get error :
No such file or directory: '/parameter/dir/intrinsic.mat'

Besides, you should check whether the intrinsic matrix is in this path. If not, you should modify those matlab code to ensure you can save it to correct path.

[ghost]

@sklf
thank you
i dont know must run .datasets/CampusSeq1/Calibration/producePmat.m
to get instrinsic matrix in matlab