How can I train on a dataset without lidar2cam matrix?

[Italian-PAO]

Hi filaPro,
Thank you for your brilliant work in 3D detection.
I'm trying to train imvoxelnet on my own dataset which only has world2cam matrix, but no lidar2cam matrix compared with kitti dataset. Is lidar2cam matrix is necessary for training? If so, can I train with world2cam matrix?
Thank you!

[Italian-PAO]

btw, I think there is a R0 matrix (3x3) in our custom dataset just like kitti, however it's in a vector format, like this

and as we can see, the translation matrix is extremely larger than that in kitti

[filaPro]

Yes, I don't see difference between lidar2img and world2img. We only compute projection matrix here and use it here. Just check that your points from world coordinate system are projected to something reasonable in this function. Also be careful with world coordinate ranges for your custom dataset 1, 2.

Btw, we also implement ImVoxelNet in original mmdetection3d repo. The code may be more clean there.

[Italian-PAO]

Thank you so much for your reply!
Actually, I am using mmdet3d repo, and I have used the roation (without the translation) above to formulate "world2img" matrix, but when I browse my dataset in mmdet3d, the dir angle seems to have pi/4 offset, like this

I think maybe it's caused by "LiDARInstance3DBoxes" in mmdet3d, because when I browse the dataset with other model's config using "CameraInstance3DBoxes", the visualization result is right.
I also tried to change the “dir_offset” in anchor3d_head.py, but it still does not help.
What could be the reason of this problem？Or will this problem affect the detection results except the wrong angle.
Thanks again