pointcloud_object_detection.rst

Point cloud object detection is currently in Beta.

Point Cloud Object Detection

The Task

A Point Cloud is a set of data points in space, usually describes by x, y and z coordinates.

PointCloud Object Detection is the task of identifying 3D objects in point clouds and their associated classes and 3D bounding boxes.

The current integration builds on top Open3D-ML.

---

Example

Let's look at an example using a data set generated from the KITTI Vision Benchmark. The data are a tiny subset of the original dataset and contains sequences of point clouds.

The data contains:

• one folder for scans
• one folder for scan calibrations
• one folder for labels
• a meta.yaml file describing the classes and their official associated color map.

Here's the structure:

data
├── meta.yaml
├── train
│   ├── scans
|   |    ├── 00000.bin
|   |    ├── 00001.bin
|   |    ...
│   ├── calibs
|   |    ├── 00000.txt
|   |    ├── 00001.txt
|   |   ...
│   ├── labels
|   |    ├── 00000.txt
|   |    ├── 00001.txt
│   ...
├── val
│   ...
├── predict
    ├── scans
    |   ├── 00000.bin
    |   ├── 00001.bin
    |
    ├── calibs
    |   ├── 00000.txt
    |   ├── 00001.txt
    ├── meta.yaml

Learn more: http://www.semantic-kitti.org/dataset.html

Once we've downloaded the data using ~flash.core.data.download_data, we create the ~flash.image.detection.data.PointCloudObjectDetectorData. We select a pre-trained randlanet_semantic_kitti backbone for our ~flash.image.detection.model.PointCloudObjectDetector task. We then use the trained ~flash.image.detection.model.PointCloudObjectDetector for inference. Finally, we save the model. Here's the full example:

../../../flash_examples/pointcloud_detection.py

To learn how to view the available backbones / heads for this task, see backbones_heads.

---

Flash Zero

The point cloud object detector can be used directly from the command line with zero code using flash_zero. You can run the above example with:

flash pointcloud_detection

To view configuration options and options for running the point cloud object detector with your own data, use:

flash pointcloud_detection --help