Investigate why OBBs are not aligning correctly with the clusters

[sophtsang]

Observing that even for static clusters that don't really move spatially, the generated oriented bounding boxes around these clusters sometimes differ by 90 degrees in y-axis angle. I checked the y-axis angle and found that for these cases the difference in angle would differ by one sign with similar magnitude, which is causing this rotation.

[sophtsang]

Given that I am using OBB matching with IoU-based cost function for cluster matching over consecutive LiDAR scans, I need to make sure that the OBBs generated are accurate representations of the shape and spatial orientation of the corresponding obstacle clusters. When testing the previous implementation of OBB generation, I noticed that for static obstacles, sometimes the orientation of boxes would deviate by ~90 degrees, which may be a factor in the reason why cluster matching right now does not produce consistent cluster_ids for static obstacles.

First I had to out the correct xyz-axes orientation of the sample LiDAR scan I was using for testing, just to keep track the orientations are as follows, which is pretty conventional:

• x - red - front, back
• y - green - left, right
• z - red - up, down

Didn't pinpoint the exact problem for why there would sometimes be ~90 deg deviations in the yaw of the bounding boxes when using OpenCV's minAreaRect to first predict 2D bounding boxes on BEV projections and then adding height in the z-axis via z_min and z_max of point cloud clusters. I switched to directly to using cpp::open3d to generate bounding boxes, set pitch and roll to 0 (here the assumption is that the ground plane is the xy-plane, later I will have to find a way to actually get the ground plane), keep only the yaw, and generate bounding boxes that actually represent the orientation of the obstacle clusters pretty well.

working.mp4

[lucaslibshutz]

@sophtsang this is sweet! is it possible that the cost function you mentioned is not really taking orientation into account, or that there might be a more direct way to penalize instantaneous orientation changes after detecting a box?

[sophtsang]

Cluster matching is now a mix of predefined matches and max bipartite matching:

• The loss function takes into account differences in box centroid, orientation (for now just yaw), and volume, which could be like a rough estimation of IoU. Predefined matches are past, current cluster matches that have loss < 0.05f: this is a hyperparameter, I just observed that for static obstacles, typically loss is less than this threshold.
• Max bipartite matching is implemented via Hungarian algorithm.

I'm going to claim that cluster matching is working pretty well for most cases -> I'll tweak around more w/ loss functions and predefined matches but I don't want to overfit.

Should probably start focusing on speedup and optimization of cluster matching, there's observable latency and delay between lidar scans oops.

[sophtsang]

Should I close this specific issue? Not sure there's more work needed to be done on generating the OBBs.

Also here is a demo, as you can see, cluster matching is kinda slow :(

[lucaslibshutz]

@sophtsang I guess so - as long as the OBB alignment problem has been fixed then ye