In this project, obstacles were detected and tracked with a camera and lidar sensors mounted on a vehicle.
- Image keypoints and keypoints descriptors are extracted using the best of detector+ descriptor combination based on performance analysis.
- Keypoints are matched across two consecutive frames.
- Images are processed through YOLO as an out-of-the-box obstacle detector providing a bounding box dimension and the center of the detected obstacle.
- Lidar points are cropped to focus on the preceding vehicle.
- Then, Lidar points are clustered by associating lidar points enclosed by a bounding box to that box.
Finally, 3d objects are matched across successive frames, and the TTC (time to collision) is calculated based on camera and lidar data. For simplicity, the TTC estimation is done only for the preceding vehicle.
- To install dependencies, follow the instructions provided here.
- Find more details about implementation here.
- In LidarTTC you can find the performance analysis of the TTC Lidar on a frame-by-frame basis, although the manual calculation is not error-free.
- In Camera TTC performance analysis on a frame-by-frame basis can be found. Red cells represent a poor estimation of the TTC compared to the TTC frame average.
- Camera images with TTC estimation are located here.
