Skip to content
Deep learning inference nodes for ROS with support for NVIDIA Jetson TX1/TX2/Xavier and TensorRT
Branch: master
Clone or download
Latest commit 126e561 Jan 4, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
config ros parameter server is working nicely Oct 25, 2016
launch works with jetson-video nodelet Oct 28, 2016
src added segNet node Jan 4, 2019
CMakeLists.txt added segNet node Jan 4, 2019
README.md update readme Dec 20, 2018
package.xml nodelet implementation is functional Oct 25, 2016
ros_deep_learning_nodelets.xml forgot to add a file earlier Oct 26, 2016

README.md

ros_deep_learning

This repo contains deep learning inference nodes for ROS with support for NVIDIA Jetson TX1/TX2/Xavier and TensorRT.

The nodes use the image recognition and object detection vision objects from the jetson-inference library and NVIDIA Two Days to a Demo tutorial, which come with several built-in pretrained network models and the ability to load customized user-trained models.

ROS Kinetic (for TX1/TX2) and ROS Melodic (for Xavier) are supported.

Installation

jetson-inference

First, install the latest JetPack on your Jetson (JetPack 3.3 for TX1/TX2 and JetPack 4.1.1 for Xavier).

Then, build and install jetson-inference

$ cd ~
$ sudo apt-get install git cmake
$ git clone https://github.com/dusty-nv/jetson-inference
$ cd jetson-inference
$ git submodule update --init
$ mkdir build
$ cd build
$ cmake ../
$ make
$ sudo make install

Before proceeding, it's worthwhile to test that jetson-inference is working properly on your system by following this step of the Two Days to a Demo tutorial:

ROS

Install the ros-base package on your Jetson following these directions:

Then, create a Catkin workspace (~/catkin_ws) using these steps:

http://wiki.ros.org/ROS/Tutorials/InstallingandConfiguringROSEnvironment#Create_a_ROS_Workspace

Depending on which Jetson you're using, install some additional dependencies:

TX1/TX2 (ROS Kinetic)

$ sudo apt-get install ros-kinetic-image-transport
$ sudo apt-get install ros-kinetic-image-publisher
$ sudo apt-get install ros-kinetic-vision-msgs

Xavier (ROS Melodic)

$ sudo apt-get install ros-melodic-image-transport
$ sudo apt-get install ros-melodic-image-publisher
$ sudo apt-get install ros-melodic-vision-msgs

ros_deep_learning

Next, navigate into your Catkin workspace and clone and build ros_deep_learning:

$ cd ~/catkin_ws/src
$ git clone https://github.com/dusty-nv/ros_deep_learning
$ cd ../
$ catkin_make

The inferencing nodes should now be built and ready to use.

Testing

Before proceeding, make sure that roscore is running first:

$ roscore

imageNet

First, to stream some image data for the inferencing node to process, open another terminal and start an image_publisher, which loads a specified image from disk. We tell it to load one of the test images that come with jetson-inference, but you can substitute your own images here as well:

$ rosrun image_publisher image_publisher __name:=image_publisher ~/jetson-inference/data/images/orange_0.jpg

Next, open a new terminal, overlay your Catkin workspace, and start the imagenet node:

$ source ~/catkin_ws/devel/setup.bash
$ rosrun ros_deep_learning imagenet /imagenet/image_in:=/image_publisher/image_raw _model_name:=googlenet

Here, we remap imagenet's image_in input topic to the output of the image_publisher, and tell it to load the GoogleNet model using the node's model_name parameter. You can substitute alexnet and googlenet-12 here, with the googlenet model being loaded by default.

In another terminal, you should be able to verify the vision_msgs/Classification2D message output of the node, which is published to the imagenet/classification topic:

$ rostopic echo /imagenet/classification

detectNet

Kill the other nodes you launched above, and start publishing a new image with people in it for the detectnet node to process:

$ rosrun image_publisher image_publisher __name:=image_publisher ~/jetson-inference/data/images/peds-004.jpg 
$ rosrun ros_deep_learning detectnet /detectnet/image_in:=/image_publisher/image_raw _model_name:=pednet

See here for the built-in detection models available. Here's an example of launching with the model that detects dogs:

$ rosrun image_publisher image_publisher __name:=image_publisher ~/jetson-inference/data/images/dog_0.jpg
$ rosrun ros_deep_learning detectnet /detectnet/image_in:=/image_publisher/image_raw _model_name:=coco-dog

To inspect the vision_msgs/Detection2DArray message output of the node, subscribe to the detectnet/detections topic:

$ rostopic echo /detectnet/detections
You can’t perform that action at this time.