Propagating boxes between point clouds

[LouisSF]

Hi there, thanks for the nice work.

I am working on a custom sequential LiDAR point cloud dataset and wanted to know if there is a way within labelCloud to propagate annotated bounding boxes from one pointcloud to the next in a numbered sequence.
Right now I have to recreate boxes every time, I would like to reuse boxes from a previous frame and just slightly translate/rotate them.

Thanks !

[ch-sa]

Hi @LouisSF,

at the moment there is no such feature. But it would be relatively easy to add.

So, what you want is something, like this?

• copy over the bounding boxes from the previous point cloud
• use the exact same location
• only when the next point cloud does not have bounding boxes yet

[bdytx5]

I agree. This would be very helpful

[bdytx5]

Should be a way to create a "saved" box so that its usable on the next cloud

[ch-sa]

I implemented a prototype. You can check it out on branch ch-sa/bbox-propagation:

It works as follows:

• activate-able via config or menu
• checks if next point cloud already has labels (from label file)
• if not, carries over labels from previous point cloud
• the feature only works forwardly, i.e. going backward won't try carry over to the previous one

Please test it using the branch and provide feedback.

[bdytx5]

will give it a spin. Also, something Ive noticed is that creating the bounding boxes seems to be very difficult and glitchy. Im on mac, so I dont know if this is related. Its just a very poor experience in my opinion. What are your thoughts on this? Also, btw this tool has the potential to be the top 3d bounding box tool just due to its simplicity, easy installation etc, and the lack of other good tools available, so great job thus far

[bdytx5]

Yeah, unfortunately the bounding box spanning logic is just not usable for me yet.

[bdytx5]

Another idea -- if you could allow custom data logging code (eg the user overrides the log function so they can write the data in any format they choose) this would make things easier on your end as well

[LouisSF]

This looks very nice, will try it thanks ! So far haven't had many issues on Linux, I will keep you updated

[ch-sa]

Hi @bdytx5,

thanks for the feedback.

Also, something Ive noticed is that creating the bounding boxes seems to be very difficult and glitchy.

I know that especially the spanning mode takes some time to get used to. But could you be a bit more specific about what parts are difficult to use, like the positioning or the correction.

For the picking mode the logic is just to select any point and it will add a bounding box with the selected point being the front top center of it. Scrolling is the z-rotation of that box.

Another idea -- if you could allow custom data logging code (eg the user overrides the log function so they can write the data in any format they choose) this would make things easier on your end as well

You can quite easily extend labelCloud to export bounding boxes in whichever format you like. There is an abstract exporter class and you just have to implement the import and export method:
https://github.com/ch-sa/labelCloud/blob/master/labelCloud/io/labels/base.py#L66

Will try to add this to the documentation later, so keep an eye on: https://ch-sa.github.io/labelCloud/

[bdytx5]

Ok. Not sure if maybe im just not understanding how to use the box spanning, or if there are bugs on mac, or what. I guess my logic is that if I cant easily jump into the program and quickly make boxes, its not sufficient in terms of UX. I had lots of problems where the box expands straight up to infiniti, or i try to make a box within the point cloud and it just doesnt work at all. My suggestions is if you can replicate the way Fusion 360 (leading 3d printer cad program) uses sketches followed by extrusions to make the 3d shape, I think this would be much better. Here is a simple explanation if you've never used fusion youtube.com/watch?v=w70NHAS74_E

[bdytx5]

Fusion also has great logic for moving object. You tap 'M' and it basically allows you to drag the object in either x, y, or z direction. Its great and I would seriously recommend just imitating everything they do.

[LouisSF]

Okay I tested I bit, I have to agree with @bdytx5 that spanning mode is quite hard to use. I can understand the idea of using points as vertices, but I wish that that configuring the height after was easier, in practice I often face the issue of having flat boxes or some that go to infinity. Maybe just using the mouse vertical movement as a way to change the height of the box (since the other attributes are locked) would be easier ? Or at least have an option for this.

Furthermore, due to view angles it can be hard to adjust bounding box dimensions with the mouse scroll (especially since the box expands so you have to constantly readjust mouse position if your camera isn't facing the expanding box side). Maybe adding a shortcut or buttons in the UI to expand left/right, front/back, top/bottom would help ?

The box propagation seems to work well though, thank you !