Isaac ROS Image Pipeline

NVIDIA-accelerated Image Pipeline.

Overview

Isaac ROS Image Pipeline is a metapackage of functionality for image processing. Camera output often needs pre-processing to meet the input requirements of multiple different perception functions. This can include cropping, resizing, mirroring, correcting for lens distortion, and color space conversion. For stereo cameras, additional processing is required to produce disparity between left + right images and a point cloud for depth perception.

This package is accelerated using the GPU and specialized hardware engines for image computation, replacing the CPU-based image_pipeline metapackage. Considerable effort has been made to ensure that replacing image_pipeline with isaac_ros_image_pipeline on a Jetson or GPU is as painless a transition as possible.

Note

Some image pre-processing functions use specialized hardware engines, which offload the GPU to make more compute available for other tasks.

Rectify corrects for lens distortion from the received camera sensor message. The rectified image is resized to the input resolution for disparity, using a crop before resizing to maintain image aspect ratio. The image is color space converted to YUV from RGB using the luma channel (the Y in YUV) to compute disparity using SGM. This common graph of nodes can be performed without the CPU processing a single pixel using isaac_ros_image_pipeline; in comparison, using image_pipeline, the CPU would process each pixel ~3 times.

The Isaac ROS Image Pipeline metapackage offloads the CPU from common image processing tasks so it can perform robotics functions best suited for the CPU.

Isaac ROS NITROS Acceleration

This package is powered by NVIDIA Isaac Transport for ROS (NITROS), which leverages type adaptation and negotiation to optimize message formats and dramatically accelerate communication between participating nodes.

Performance

Sample Graph	Input Size	AGX Thor T5000	AGX Thor T4000	AGX Orin	Orin Nano Super 8GB	DGX Spark	x86_64 w/ RTX 5090	x86_64 w/ RTX 5070
Rectify Node	1080p	1690 fps 0.48 ms @ 30Hz	1140 fps 0.52 ms @ 30Hz	886 fps 0.51 ms @ 30Hz	296 fps 1.4 ms @ 30Hz	2500 fps 0.15 ms @ 30Hz	2490 fps 0.43 ms @ 30Hz	1300 fps 0.46 ms @ 30Hz
Stereo Disparity Node	1080p	187 fps 5.8 ms @ 30Hz	128 fps 27 ms @ 30Hz	86.5 fps 25 ms @ 30Hz	45.5 fps 22 ms @ 30Hz	198 fps 5.0 ms @ 30Hz	478 fps 2.4 ms @ 30Hz	281 fps 2.9 ms @ 30Hz
Stereo Disparity Graph	1080p	168 fps 6.1 ms @ 30Hz	115 fps 30 ms @ 30Hz	83.0 fps 25 ms @ 30Hz	41.2 fps 24 ms @ 30Hz	174 fps 6.0 ms @ 30Hz	413 fps 2.8 ms @ 30Hz	237 fps 3.2 ms @ 30Hz

---

Documentation

Please visit the Isaac ROS Documentation to learn how to use this repository.

---

Packages

• isaac_ros_depth_image_proc
  • API
• isaac_ros_image_pipeline
  • Replacing image_pipeline with isaac_ros_image_pipeline
• isaac_ros_image_proc
  • Quickstart
• isaac_ros_stereo_image_proc
  • Quickstart
  • Try More Examples
  • API

Latest

Update 2026-08-18: Compatibility and integration updates for the Isaac ROS 4.6.0 release