isaac_ros_stereo_image_proc: Subscribed topics should be image_raw? #19
Comments
|
Which node in What seems off about the disparity view? Are the images rectified on the way into DisparityNode? Could the calibration on the camera be off perhaps? |
|
Hi hemalshahNV, thanks for your support. I am using the EA3 release of the repository with JP 4.6.1 on a Jetson Xavier AGX. Since the hawk does not publish rectified images natively but raw images instead the disparity looks bad. You can see an example here, where I used the raw images with the isaac::stereo_image_proc plugin. Terminal 1: (Start Camera and remap topics to "fake" rectified images) Terminal 2: (Use stereo_image_proc plugin with raw images) Terminal 3: (Show the image)
You can see that the image is raw and distorted and the disparity looks bad. I understood from your last comment that I need to make use of the composable nodes in order to rectify images before using the isaac::stereo_image_proc plugin since it is subscribed to rectified images. Basically the pipeline looks as follows:
I have created a startup script based on composable nodes. All the image processing runs in a container. It is attached here. It makes some remapping of nodes in order to match with the nodes subscriptions: def generate_launch_description():
"""Launch file which brings up argus monocular camera node with rectify node."""
rectify_node_left = ComposableNode(
name='isaac_ros_rectify',
package='isaac_ros_image_proc',
plugin='isaac_ros::image_proc::RectifyNode',
remappings=[('image', 'left/image_raw'),
('camera_info', 'left/camera_info'),
('image_rect', 'left/image_rect')])
rectify_node_right = ComposableNode(
name='isaac_ros_rectify',
package='isaac_ros_image_proc',
plugin='isaac_ros::image_proc::RectifyNode',
remappings=[('image', 'right/image_raw'),
('camera_info', 'right/camera_info'),
('image_rect', 'right/image_rect')])
disparity_node = ComposableNode(
name='disparity',
package='isaac_ros_stereo_image_proc',
plugin='isaac_ros::stereo_image_proc::DisparityNode',
parameters=[{
'backends': 'CUDA',
'max_disparity': 256,
}])
argus_camera_node = Node(
package='isaac_ros_argus_camera_stereo',
executable='isaac_ros_argus_camera_stereo',
parameters=[{
'device': 0,
'sensor': 0,
'output_encoding': "rgb8",
'camera_frame_id': "test_frame_id",
}],
remappings=[('stereo/left/image_raw', 'left/image_raw'),
('stereo/right/image_raw', 'right/image_raw'),
('stereo/right/camera_info', 'right/camera_info'),
('stereo/left/camera_info', 'left/camera_info')])
disparity_container = ComposableNodeContainer(
name='disparity_container',
namespace='',
package='rclcpp_components',
executable='component_container',
composable_node_descriptions=[
rectify_node_left,
rectify_node_right,
disparity_node
],
output='screen'
)
return launch.LaunchDescription([disparity_container, argus_camera_node])I can see that the following topics are created when executing the script above: So everything seems to be fine in terms of needed topics. I can now observe rectified images on /left/image_rect and /right/image_rect:
But the disparity view does not generate any data or shows any image. When launching the script, there are some warnings that I do not understand. Maybe you can help to debug this issue, or maybe there is a mistake in the launch script? Best Regards and thanks |
|
Thanks for the helpful information in your post! We suspect that the nodes are having trouble processing the large images from HAWK at full resolution. Rectify node seems to be complaining that not enough images and camera_infos are appearing close enough in time for it to match them. This may be causing left and right images to be published soon enough to be time synchronized causing no disparity to be emitted. You may want to try a smaller image size and a smaller max_disparity value to confirm. In more recent Isaac ROS releases, we introduced NITROS which eliminates much of the memory overhead with large images which should significantly improve your framerate. Finally, the launch file seems correct but it may be worth verifying using rQt to make sure everything is actually connected. The warning is a concern but you should likely be seeing at least a slow rate of disparity images coming through. |
hemalshahNV
added
needs info
|
Thanks hemalshahNV, reducing the CPU load by adjusting the image size did make the trick. The Xavier AGX seems not to have enough computing power to process the image pipeline at full resolution. I will try to update to newest JP release to make use of NITROS acceleration in order to see if this makes a difference. Best Regards |
Hi team,
Regarding the current documentation the isaac_ros_stereo_image_proc package is subscribed to left/right rectified images (/image_rect)? But the "original" package from ros (stereo_image_proc) is subscribed to raw images, see http://wiki.ros.org/stereo_image_proc?distro=noetic.
Is this an documentation issue or am I wrong? I have tried passing remapped nodes (left/image_raw /right/image_raw to left/image_rect right/image_rect) to this package and it seems to work with raw inputs. Nevertheless, the disparity view is not looking as good as I would expect, so I am bit struggling.
I am using the GMSL stereo camera (Hawk) from LI, which is not publishing rectified images natively.
Thanks
millertheripper
The text was updated successfully, but these errors were encountered: