3D object detection Transformation issue

[imdsafi09]

Hi,
I am trying to run the stack with only D435 camera. As I dont have the base_link, I change the target_frame into camera_link. However, I am still getting the following error even though the camera_link frame exists.

"[detect_3d_node-3] [ERROR] [1709019322.086835673] [yolo.detect_3d_node]: Could not transform: "camera_link" passed to lookupTransform argument target_frame does not exist.
"

[mgonzs13]

Hey @imdsafi09, I couldn't reproduce your error. I may be a problem with your tf. Which ROS distro are you using?

[imdsafi09]

Hi @mgonzs13
Thank you for your response. The reason for this was that my camera had not previously published the/tf_static frame.
However, I have another problem, I am using Ouster-lidar signal image topic "/ouster/signal_image" to subscribe it into as a camera. The topic is working with similar yolov8 based ros package ultralytics_ros but when I am trying to subscribe it here it has the following error:

[yolov8_node-1] File "/home/imad/.local/lib/python3.10/site-packages/ultralytics/engine/predictor.py", line 125, in preprocess
[yolov8_node-1] im = im[..., ::-1].transpose((0, 3, 1, 2)) # BGR to RGB, BHWC to BCHW, (n, 3, h, w)

Could you please let me know what could be the possible solution.

Ubuntu:22.04
ROS-Humble

Best regards,

[mgonzs13]

Hi @imdsafi09, it seems to be a problem with the encoding of the image. Can you give me more info about the error, the error message? If you record a rosbag with your data, I can test it better.

[imdsafi09]

Thank you for your response, @mgonzs13. The issue was caused by a mismatch in the input image channel, which was resolved when I converted mono16 to rgb8.

[mgonzs13]

Hey @imdsafi09, how is this going?

[imdsafi09]

Hi @mgonzs13 , sorry for the late response. I resolved the previous issue, and it's working with the Ouster-lidar signal image topic. However, 3D bounding visualization is the same for all the classes. Would it be possible to have a different visualization based on the class type, similar to a 2D one?

[mgonzs13]

Hi @imdsafi09, do you mean changing the color of the 3D bounding box? I think that can be easily included.

[mgonzs13]

Check out the new version that includes the 3D bounding boxes with the same color as the 2D boxes.

[imdsafi09]

Hi @mgonzs13,
Thank you for your excellent work. I implemented it successfully, and the 2D and 3D versions work fine. In my case, I am integrating 3D object detection with 3D mapping to save the object's location within the map. I tried several approaches. However, they cause redundancy and make the overall process computationally more expensive. Could you please give any suggestions?
Regards,

[mgonzs13]

I have never used semantic localization. Which works have you checked? From a fast search, I have found these 1, 2, 3. Do you have a public repo to take a look at your work?

[imdsafi09]

I had no known issues with mapping or localization. I am looking for options on how to keep/save the detected 3D object position on the map. I am attaching the script I am following for you to look at. It would be something like this.
1
2

Like the image below, once the bounding box is detected, it should not disappear, but its location should be added to the map.

[mgonzs13]

So, If I understand the code in 1, markers from door detections are used to modify the occupancy grid map of the localization. As a result, you will have the detections included in the map. Or do you also want to visualize the detection markers? You can add a publisher to publish the markers included in the map so that the don't desappear.

[imdsafi09]

Yes, you understood it correctly. They used markers from objects of interest to modify the occupancy grid map. Technically, the markers visualization does not add value to the overall system. Only the localization of the object within the map is essential for my case. However, for visual understanding, I am trying to add detected markers as well. Thank you for all your insightful comments. I will look into it as you suggested.

[HHX0607]

Hi, I am trying to run the stack with only D435 camera. As I dont have the base_link, I change the target_frame into camera_link. However, I am still getting the following error even though the camera_link frame exists.

"[detect_3d_node-3] [ERROR] [1709019322.086835673] [yolo.detect_3d_node]: Could not transform: "camera_link" passed to lookupTransform argument target_frame does not exist. "

@imdsafi09 Hello！I am facing the same problem.Could you tell me how to create a camera_link and what is necessary to set. I just use D435i and i want to test 3d detection on yolov8_ros. Thanks!

[imdsafi09]

Hi @HHX0607,

In my case, I changed the default value from default_value="base_link", to "camera_link" and it worked fine. Hopefully, it will work for you as well. If you have already made changes in this file and are still facing the problem, then try checking the 'tf_tree'.