Shouldn't you project the 3D coordinates for KITTI 2D MOT evaluation ?

[Florijan-Iljazovic]

Hello,

your paper and code are fantastic! I learned a lot.
Quick question:
In additional_info variables, among other things, you store 2D bounding box that is obtained directly from detector module (point-rcnn).
You then use this 2D box directly and pass it to output files to be evaluated on KITTI server.

On the other hand, 3D coordinates obtained from the detector are used in your tracking pipeline to associate detections and tracks, and are therefore updated in the process (Kalman filter update, orientation correction etc.)

Shouldn't you project those 3D coordinates obtained from your tracking pipeline onto image plane and use this 2D bounding box for evaluation on KITTI 2D mot benchmark ?
Instead of using given bounding box from the detection.

Thanks!

[xinshuoweng]

Thanks for your question!

So both should work, either using the original 2D box's coordinate or projecting the updated 3D coordinates to 2D. However, there is a slight difference in performance. I remember I have tried both a long time ago and if I remember correctly, projecting 3D coordinates to 2D yields a slightly worse performance. This is quite common to see due to the 3D-2D box definition inconsistency. Normally, the projected 2D box from 3D box is not tight enough compared to detection directly in 2D space. You can try visualization for a few 3D detection results and see if this is the case.