Missed/shifted boxes problem

[eslambakr]

Thanks for your amazing work.

I have noticed two main problems in your module:

1. The first one is the shifted boxes as shown below
   
   I think this mainly because of CARLA server itself, what do you think?

2. The second problem is missing Bboxes due to the area that you have chosen becomes out of field of view.
   I think we could easily solve it by using the segmentation mask instead of generating smaller box from the original one.
   

[MukhlasAdib]

Hi again,

1. I also think that it is caused by unsolved error in CARLA synchronization. I have encountered the same problem quite often when collecting multiple sensors' data in CARLA and it seems that it has become a long issue in CARLA community itself. However, my module is not responsible for the data generation process. So, it might require some tricky coding but maybe it is something that can be solved in client side. (Note: I haven't solved it either)

2. Thanks for informing me about this. I used the vehicle's centre coordinate to determine whether it is inside the camera FOV or not. So, even tough that most of vehicle's body is visible, it will be filtered out if its centre point is not inside the FOV. And also thanks for your suggestion. I'll fix it as soon as I can.

Anyway, I have just read CARLA 0.9.10 update info and I found that this version provides LIDAR instance segmentation. I think it will be more efficient to use it for BBox extraction rather than using depth image. It might also solve the problem No. 2 that you mentioned. I'm working on it.

[eslambakr]

Hello @MukhlasAdib

I have solved the first problem as we said it was because of synchronization problem.
I noticed that you began your code "test_draw_bb.py" from CARLA example code "client_bounding_boxes.py" which is suffering from synchronization problem, so I re-base the code to start from "synchronous_mode.py" instead, which illuminate the shifted boxes problem.

I am working now on solving the missing boxes problem but using the same version 9.8 to avoid the hassle of upgrading CARLA version.
I will keep the issue open and post the updates here, as it may be useful for others

[MukhlasAdib]

@eslambakr, thanks a lot for your help! I will try it and perhaps update the code after I am done building the code that uses segmentation LIDAR. Feel free to share anything that you find here

[MukhlasAdib]

@eslambakr Thanks for your suggestion, I have updated the code to fix the shift problem. In the other hand, I also have added a function to extract bounding boxes using semantic LIDAR, which seems to solve the missed box problem.