Isaac ROS Visual SLAM

Overview

This repository provides a ROS2 package that performs stereo visual simultaneous localization and mapping (VSLAM) and estimates stereo visual inertial odometry using the Isaac Elbrus GPU-accelerated library. It takes in a time-synced pair of stereo images (grayscale) along with respective camera intrinsics to publish the current pose of the camera relative to its start pose.

Elbrus is based on two core technologies: Visual Odometry (VO) and Simultaneous Localization and Mapping (SLAM).

Visual SLAM is a method for estimating a camera position relative to its start position. This method has an iterative nature. At each iteration, it considers two consequential input frames (stereo pairs). On both the frames, it finds a set of keypoints. Matching keypoints in these two sets gives the ability to estimate the transition and relative rotation of the camera between frames.

Simultaneous Localization and Mapping is a method built on top of the VO predictions. It aims to improve the quality of VO estimations by leveraging the knowledge of previously seen parts of a trajectory. It detects if the current scene was seen in the past (i.e. a loop in camera movement) and runs an additional optimization procedure to tune previously obtained poses.

Along with visual data, Elbrus can optionally use Inertial Measurement Unit (IMU) measurements. It automatically switches to IMU when VO is unable to estimate a pose; for example, when there is dark lighting or long solid featureless surfaces in front of a camera. Elbrus delivers real-time tracking performance: more than 60 FPS for VGA resolution. For the KITTI benchmark, the algorithm achieves a drift of ~1% in localization and an orientation error of 0.003 degrees per meter of motion. Elbrus allows for robust tracking in various environments and with different use cases: indoor, outdoor, aerial, HMD, automotive, and robotics.

To learn more about Elbrus SLAM click here.

Latest Update

Update 2022-08-31: Update to be compatible with JetPack 5.0.2

Supported Platforms

This package is designed and tested to be compatible with ROS2 Humble running on Jetson or an x86_64 system with an NVIDIA GPU.

Note: Versions of ROS2 earlier than Humble are not supported. This package depends on specific ROS2 implementation features that were only introduced beginning with the Humble release.

Platform	Hardware	Software	Notes
Jetson	Jetson Orin Jetson Xavier	JetPack 5.0.2	For best performance, ensure that power settings are configured appropriately.
x86_64	NVIDIA GPU	Ubuntu 20.04+ CUDA 11.6.1+

Docker

To simplify development, we strongly recommend leveraging the Isaac ROS Dev Docker images by following these steps. This will streamline your development environment setup with the correct versions of dependencies on both Jetson and x86_64 platforms.

Note: All Isaac ROS Quickstarts, tutorials, and examples have been designed with the Isaac ROS Docker images as a prerequisite.

Coordinate Frames

This section describes the coordinate frames that are involved in the VisualSlamNode. The frames discussed below are oriented as follows:

input_base_frame: The name of the frame used to calculate transformation between baselink and left camera. The default value is empty (''), which means the value of base_frame_ will be used. If input_base_frame_ and base_frame_ are both empty, the left camera is assumed to be in the robot's center.
input_left_camera_frame: The frame associated with left eye of the stereo camera. Note that this is not the same as the optical frame. The default value is empty (''), which means the left camera is in the robot's center and left_pose_right will be calculated from the CameraInfo message.
input_right_camera_frame: The frame associated with the right eye of the stereo camera. Note that this is not the same as the optical frame. The default value is empty (''), which means left and right cameras have identity rotation and are horizontally aligned, so left_pose_right will be calculated from CameraInfo.
input_imu_frame: The frame associated with the IMU sensor (if available). It is used to calculate left_pose_imu.

Quickstart

Set up your development environment by following the instructions here.

Clone this repository and its dependencies under ~/workspaces/isaac_ros-dev/src.

cd ~/workspaces/isaac_ros-dev/src

git clone https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_visual_slam

git clone https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common

Pull down a ROS Bag of sample data:

cd ~/workspaces/isaac_ros-dev/src/isaac_ros_visual_slam && \ 
  git lfs pull -X "" -I isaac_ros_visual_slam/test/test_cases/rosbags/

Launch the Docker container using the run_dev.sh script:

cd ~/workspaces/isaac_ros-dev/src/isaac_ros_common && \
  ./scripts/run_dev.sh

Inside the container, build and source the workspace:

cd /workspaces/isaac_ros-dev && \
  colcon build --symlink-install && \
  source install/setup.bash

(Optional) Run tests to verify complete and correct installation:
```
colcon test --executor sequential
```

Run the following launch files in the current terminal:

ros2 launch isaac_ros_visual_slam isaac_ros_visual_slam.launch.py

In a second terminal inside the Docker container, prepare rviz to display the output:

source /workspaces/isaac_ros-dev/install/setup.bash && \
  rviz2 -d src/isaac_ros_visual_slam/isaac_ros_visual_slam/rviz/default.cfg.rviz

In an another terminal inside the Docker container, run the following ros bag file to start the demo:
```
source /workspaces/isaac_ros-dev/install/setup.bash && \
  ros2 bag play src/isaac_ros_visual_slam/isaac_ros_visual_slam/test/test_cases/rosbags/small_pol_test/
```
Rviz should start displaying the point clouds and poses like below:

Next Steps

Try More Examples

To continue your exploration, check out the following suggested examples:

Tutorial with Isaac Sim

Customize your Dev Environment

To customize your development environment, reference this guide.

Package Reference