This repo contains deep learning inference nodes for ROS with support for NVIDIA Jetson TX1/TX2/Xavier and TensorRT.
The nodes use the image recognition and object detection vision objects from the
jetson-inference library and NVIDIA Two Days to a Demo tutorial, which come with several built-in pretrained network models and the ability to load customized user-trained models.
ROS Kinetic (for TX1/TX2) and ROS Melodic (for Xavier) are supported.
First, install the latest JetPack on your Jetson (JetPack 3.3 for TX1/TX2 and JetPack 4.1.1 for Xavier).
Then, build and install
$ cd ~ $ sudo apt-get install git cmake $ git clone https://github.com/dusty-nv/jetson-inference $ cd jetson-inference $ git submodule update --init $ mkdir build $ cd build $ cmake ../ $ make $ sudo make install
Before proceeding, it's worthwhile to test that
jetson-inference is working properly on your system by following this step of the Two Days to a Demo tutorial:
ros-base package on your Jetson following these directions:
Then, create a Catkin workspace (
~/catkin_ws) using these steps:
Depending on which Jetson you're using, install some additional dependencies:
TX1/TX2 (ROS Kinetic)
$ sudo apt-get install ros-kinetic-image-transport $ sudo apt-get install ros-kinetic-image-publisher $ sudo apt-get install ros-kinetic-vision-msgs
Xavier (ROS Melodic)
$ sudo apt-get install ros-melodic-image-transport $ sudo apt-get install ros-melodic-image-publisher $ sudo apt-get install ros-melodic-vision-msgs
Next, navigate into your Catkin workspace and clone and build
$ cd ~/catkin_ws/src $ git clone https://github.com/dusty-nv/ros_deep_learning $ cd ../ $ catkin_make
The inferencing nodes should now be built and ready to use.
Before proceeding, make sure that
roscore is running first:
First, to stream some image data for the inferencing node to process, open another terminal and start an
image_publisher, which loads a specified image from disk. We tell it to load one of the test images that come with jetson-inference, but you can substitute your own images here as well:
$ rosrun image_publisher image_publisher __name:=image_publisher ~/jetson-inference/data/images/orange_0.jpg
Next, open a new terminal, overlay your Catkin workspace, and start the
$ source ~/catkin_ws/devel/setup.bash $ rosrun ros_deep_learning imagenet /imagenet/image_in:=/image_publisher/image_raw _model_name:=googlenet
Here, we remap imagenet's
image_in input topic to the output of the
image_publisher, and tell it to load the GoogleNet model using the node's
model_name parameter. You can substitute
googlenet-12 here, with the
googlenet model being loaded by default.
In another terminal, you should be able to verify the
vision_msgs/Classification2D message output of the node, which is published to the
$ rostopic echo /imagenet/classification
Kill the other nodes you launched above, and start publishing a new image with people in it for the
detectnet node to process:
$ rosrun image_publisher image_publisher __name:=image_publisher ~/jetson-inference/data/images/peds-004.jpg $ rosrun ros_deep_learning detectnet /detectnet/image_in:=/image_publisher/image_raw _model_name:=pednet
See here for the built-in detection models available. Here's an example of launching with the model that detects dogs:
$ rosrun image_publisher image_publisher __name:=image_publisher ~/jetson-inference/data/images/dog_0.jpg $ rosrun ros_deep_learning detectnet /detectnet/image_in:=/image_publisher/image_raw _model_name:=coco-dog
To inspect the
vision_msgs/Detection2DArray message output of the node, subscribe to the
$ rostopic echo /detectnet/detections