Skip to content

Commit

Permalink
Add label propagation
Browse files Browse the repository at this point in the history
 - add a feature to carry over labels to the next point cloud
 - can be activated via config and menu action
 - will only copy labels if the next point cloud has no bounding boxes so far
 - add checkbox to settings dialog
 - add to documentation

Closes: #120
  • Loading branch information
ch-sa committed Dec 11, 2022
1 parent 0ca6868 commit ee1f238
Show file tree
Hide file tree
Showing 8 changed files with 263 additions and 182 deletions.
2 changes: 2 additions & 0 deletions config.ini
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,8 @@ std_rotation = 0.5
std_scaling = 0.03
; minimum value for the length, width and height of a bounding box
min_boundingbox_dimension = 0.01
; propagate labels to next point cloud if it has no labels yet
propagate_labels = False

[USER_INTERFACE]
; only allow z-rotation of bounding boxes. set false to also label x- & y-rotation
Expand Down
1 change: 1 addition & 0 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ The following parameters can be changed:
| `std_rotation` | Standard step for rotating the bounding box (with key press). | *0.5* |
| `std_scaling` | Standard step for scaling the bounding box (with button press). | *0.03* |
| `min_boundingbox_dimension` | Minimum value for the length, width and height of a bounding box. | *0.01* |
| `propagate_labels` | Copy all bounding boxes of the current point cloud to the next point cloud (only forward). | *False* |
| **[USER_INTERFACE]** |
| `z_rotation_only` | Only allow z-rotation of bounding box; deactivate to also label x- & y-rotation. | *True* |
| `show_floor` | Visualizes the floor (x-y-plane) as a grid. | *True* |
Expand Down
49 changes: 32 additions & 17 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,12 @@ labelCloud is a lightweight tool for labeling 3D bounding boxes in point clouds.
It is written in Python and can be installed via `pip` (see [Setup](setup.md)).

## Labeling
labelCloud supports two different ways of labeling (*picking* & *spanning*) as well as multiple mouse and keyboard options for subsequent correction.
labelCloud supports two different ways of labeling (*picking* & *spanning*) as well as multiple
mouse and keyboard options for subsequent correction.

![Screencast of the Labeling Methods](assets/screencast_small.gif)
(There is also a [short YouTube-Video](https://www.youtube.com/watch?v=8GF9n1WeR8A) that introduces the tool.)
(There is also a [short YouTube-Video](https://www.youtube.com/watch?v=8GF9n1WeR8A) that introduces
the tool.)

### Picking Mode

Expand All @@ -24,16 +26,23 @@ labelCloud supports two different ways of labeling (*picking* & *spanning*) as w

### Correction

* Use the buttons on the left-hand side or shortcuts to correct the *translation*, *dimension* and *rotation* of the bounding box
* Use the buttons on the left-hand side or shortcuts to correct the *translation*, *dimension* and
*rotation* of the bounding box
* Resize the bounding box by holding your cursor above one side and scrolling with the mouse wheel

By default the x- and y-rotation of bounding boxes will be prohibited.
For labeling **9 DoF-Bounding Boxes** deactivate `z-Rotation Only Mode` in the menu, settings or `config.ini` file.
Now you will be free to rotate around all three axes.
For labeling **9 DoF-Bounding Boxes** deactivate `z-Rotation Only Mode` in the menu, settings or
`config.ini` file.
Now you will be able to rotate around all three axes.

If you have a point clouds with objects that keep their positions over multiple frames, you can
activate the *Propagate Labels* feature in the Labels menu or `config.ini`.

## Import & Export Options
labelCloud is built for a versatile use and aims at supporting all common point cloud file formats and label formats for storing 3D bounding boxes.
The tool is designed to be easily adaptable to multiple use cases. To change the settings, simply edit the corresponding line in the `config.ini` (see [Configuration](configuration.md)) for a description of all parameters).
labelCloud is built for a versatile use and aims at supporting all common point cloud file formats
and label formats for storing 3D bounding boxes.
The tool is designed to be easily adaptable to multiple use cases. To change the settings, simply
edit the corresponding line in the `config.ini` (see [Configuration](configuration.md)) for a description of all parameters).

### Supported Point Cloud Formats

Expand All @@ -44,21 +53,22 @@ The tool is designed to be easily adaptable to multiple use cases. To change the

### Supported Label Formats

| Label Format | Description |
| --------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `centroid_rel` | Centroid `[x, y, z]`; Dimensions `[length, width, height]`; <br> Relative Rotations as Euler angles in radians (-pi..+pi) `[yaw, pitch, roll]` |
| `centroid_abs` | Centroid `[x, y, z]`; Dimensions `[length, width, height]`; <br> Absolute Rotations as Euler angles in degrees (0..360°) `[yaw, pitch, roll]` |
| `vertices` | 8 Vertices of the bounding box each with `[x, y, z]` (see [Conventions](conventions.md) for order) |
| `kitti` | Centroid; Dimensions; z-Rotation (See [specification](https://github.com/bostondiditeam/kitti/blob/master/resources/devkit_object/readme.txt)) |
| `kitti_untransformed` | See above, but without transformations. |
| Label Format | Description |
| --------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
| `centroid_rel` | Centroid `[x, y, z]`; Dimensions `[length, width, height]`; <br> Relative Rotations as Euler angles in radians (-pi..+pi) `[yaw, pitch, roll]` |
| `centroid_abs` | Centroid `[x, y, z]`; Dimensions `[length, width, height]`; <br> Absolute Rotations as Euler angles in degrees (0..360°) `[yaw, pitch, roll]` |
| `vertices` | 8 Vertices of the bounding box each with `[x, y, z]` (see [Conventions](conventions.md) for order) |
| `kitti` | Centroid; Dimensions; z-Rotation (See [specification](https://github.com/bostondiditeam/kitti/blob/master/resources/devkit_object/readme.txt)) |
| `kitti_untransformed` | See above, but without transformations. |

You can easily create your own exporter by subclassing the abstract [BaseLabelFormat](https://github.com/ch-sa/labelCloud/blob/master/labelCloud/label_formats/base.py#L10).
All rotations are counterclockwise (i.e. a z-rotation of 90°/π is from the positive x- to the negative y-axis!).



## Usage & Attribution
When using the tool feel free to drop me a mail with feedback or a description of your use case (christoph.sager[at]tu-dresden.de).
When using the tool feel free to drop me a mail with feedback or a description of your use case
(christoph.sager[at]tu-dresden.de).
If you are using the tool for a scientific project please consider citing our publications:


Expand Down Expand Up @@ -104,5 +114,10 @@ If you are using the tool for a scientific project please consider citing our pu
```

## Acknowledgment
I would like to thank the [Robotron RCV-Team](https://www.robotron.de/rcv) for the support in the preparation and user evaluation of the software.
The software was developed as part of my diploma thesis titled "labelCloud: Development of a Labeling Tool for 3D Object Detection in Point Clouds" at the [Chair for Business Informatics, especially Intelligent Systems](https://tu-dresden.de/bu/wirtschaft/winf/isd) of the TU Dresden. The ongoing research can be followed in our [project on ResearchGate](https://www.researchgate.net/project/Development-of-a-Point-Cloud-Labeling-Tool-to-Generate-Training-Data-for-3D-Object-Detection-and-6D-Pose-Estimation).
I would like to thank the [Robotron RCV-Team](https://www.robotron.de/rcv) for the support in the
preparation and user evaluation of the software.
The software was developed as part of my diploma thesis titled "labelCloud: Development of a
Labeling Tool for 3D Object Detection in Point Clouds" at the
[Chair for Business Informatics, especially Intelligent Systems](https://tu-dresden.de/bu/wirtschaft/winf/isd)
of the TU Dresden. The ongoing research can be followed in our
[project on ResearchGate](https://www.researchgate.net/project/Development-of-a-Point-Cloud-Labeling-Tool-to-Generate-Training-Data-for-3D-Object-Detection-and-6D-Pose-Estimation).
6 changes: 6 additions & 0 deletions labelCloud/control/controller.py
Original file line number Diff line number Diff line change
Expand Up @@ -64,9 +64,15 @@ def next_pcd(self, save: bool = True) -> None:
if save:
self.save()
if self.pcd_manager.pcds_left():
previous_bboxes = self.bbox_controller.bboxes
self.pcd_manager.get_next_pcd()
self.reset()
self.bbox_controller.set_bboxes(self.pcd_manager.get_labels_from_file())

if not self.bbox_controller.bboxes and config.getboolean(
"LABEL", "propagate_labels"
):
self.bbox_controller.set_bboxes(previous_bboxes)
else:
self.view.update_progress(len(self.pcd_manager.pcds))
self.view.button_next_pcd.setEnabled(False)
Expand Down
35 changes: 35 additions & 0 deletions labelCloud/resources/interfaces/interface.ui
Original file line number Diff line number Diff line change
Expand Up @@ -1546,25 +1546,35 @@
<property name="title">
<string>File</string>
</property>
<property name="toolTipsVisible">
<bool>true</bool>
</property>
<addaction name="act_set_pcd_folder"/>
<addaction name="act_set_label_folder"/>
</widget>
<widget class="QMenu" name="menuLabels">
<property name="title">
<string>Labels</string>
</property>
<property name="toolTipsVisible">
<bool>true</bool>
</property>
<widget class="QMenu" name="act_set_default_class">
<property name="title">
<string>Set Default Object Class …</string>
</property>
</widget>
<addaction name="act_set_default_class"/>
<addaction name="act_delete_all_labels"/>
<addaction name="act_propagate_labels"/>
</widget>
<widget class="QMenu" name="menuSettings">
<property name="title">
<string>Settings</string>
</property>
<property name="toolTipsVisible">
<bool>true</bool>
</property>
<addaction name="act_z_rotation_only"/>
<addaction name="act_show_floor"/>
<addaction name="act_show_orientation"/>
Expand Down Expand Up @@ -1603,6 +1613,9 @@
<property name="text">
<string>Z-Rotation Only Mode</string>
</property>
<property name="toolTip">
<string>Only allows bounding box rotation around the z-axis.</string>
</property>
</action>
<action name="act_color_with_label">
<property name="checkable">
Expand Down Expand Up @@ -1642,6 +1655,9 @@
<property name="text">
<string>Show Floor</string>
</property>
<property name="toolTip">
<string>Shows a grid along the x-y-plane (z=0).</string>
</property>
</action>
<action name="act_show_orientation">
<property name="checkable">
Expand All @@ -1664,6 +1680,10 @@
<property name="text">
<string>Keep Perspective</string>
</property>
<property name="toolTip">
<string>Saves the last perspective and reuses it,
when opening that point cloud again.</string>
</property>
</action>
<action name="act_align_pcd">
<property name="checkable">
Expand All @@ -1675,6 +1695,9 @@
<property name="text">
<string>Align Point Cloud</string>
</property>
<property name="toolTip">
<string>Transforms the point cloud so that the floor is the x-y-plane.</string>
</property>
</action>
<action name="act_change_settings">
<property name="text">
Expand All @@ -1686,6 +1709,18 @@
<string>test</string>
</property>
</action>
<action name="act_propagate_labels">
<property name="checkable">
<bool>true</bool>
</property>
<property name="text">
<string>Propagate Labels</string>
</property>
<property name="toolTip">
<string>Propagate Labels to the next Point Cloud
if it does not have labels yet.</string>
</property>
</action>
</widget>
<customwidgets>
<customwidget>
Expand Down
Loading

0 comments on commit ee1f238

Please sign in to comment.