Dense RGB-D SLAM system RGB-D SLAM articulated around a supersurfel-based 3D representation for fast, lightweight and compact mapping in indoor environment.
Check out the video:
The system is strongly based on the following paper:
"Speed and Memory Efficient Dense RGB-D SLAM in Dynamic Scenes, Bruce Canovas, Michele Rombaut, Amaury Negre, Denis Pellerin and Serge Olympieff, Proc. IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems (IROS 2020), Las Vegas, USA (2020)."
Please cite this publication when using SupersurfelFusion.
Note that the software is experimental and some changes have been done since the publication of the paper. For instance in the part performing moving object detection and removal, we add the possibility to use a lightweight deep learning based object detection to extract humans and improve the robustness against dynamic elements.
The system has been tested both on Ubuntu 16.04 and 18.04.
-
ROS Kinetic or Melodic
Use ROS Kinetic on Ubuntu 16.04 and Melodic on 18.04.
-
CUDA >= 9.0
The system has been tested on platforms with CUDA 9.0 and CUDA 10.2.
-
OpenCV 3.4
We use OpenCV 3.4 but any version of OpenCV 3 should work. OpenCV has to be installed with CUDA support and with contrib modules.
Note that, to be able to use with ROS a version of OpenCV different from the default ROS OpenCV version (which doesn't have CUDA support), you might have to rebuild all ROS packages that require OpenCV against your specific version, particularly the vision_opencv package that provides cv_bridge. If you have differents OpenCV versions installed on your computer, you can specify the one you want to use at build time by calling catkin_make like this:
$ catkin_make -DOpenCV_DIR=<your OpenCV path>
or by using this:
find_package(OpenCV REQUIRED
NO_MODULE #Should be optional, tells CMake to use config mode
PATHS <your OpenCV path># Tells CMake to look here
NO_DEFAULT_PATH #and don't look anywhere else)
inside your CMakeLists.txt instead of:
find_package(OpenCV 3 REQUIRED)
to be sure to link against the desired version.
This dependency can be resolved by installing the following package: libsuitesparse-dev.
Our implementation integrates parts of codes from external libraries.
For features extraction and associated descriptors computation.
We use Grid-based Motion Statistics for robust feature correspondence.
We credit ElasticFusion as a significant basis for our deformation graph and loop closure implementation.
We based the design of our feature-based visual odometry on their method and our Perspective-n-Point solver has been implemented following their code.
We integrated tiny YOLOv4 in our pipeline for improving robustness in dynamic scenes by detecting persons. To see how to enable or disable its use please refer to this section.
SupersurfelFusion is released under a GPLv3 license (see supersurfel_fusion/licenses/LICENSE-SupersurfelFusion.txt).
SupersurfelFusion includes differents third-party open-source software, which themselves include third-party open-source software. Each of these components have their own license.
You can find the licenses in the repository supersurfel_fusion/licenses/.
The system is provided as a ROS package which can be copied or cloned into your workspace and built directly using catkin.
$ cd ~/catkin_ws/src
$ git clone https://gricad-gitlab.univ-grenoble-alpes.fr/canovasb/supersurfel_fusion.git
First install darknet to use the YOLOv4 object detector using make in the darknet repository (supersurfel_fusion/third_party/darknet).
In the Makefile set:
GPU=1to build enabling CUDA support (OPTIONAL)OPENCV=1to build with OpenCVLIBSO=1to build the librarydarknet.so
Once darknet installed just go to your catkin workspace root directory and build using catkin_make.
$ cd ~/catkin_ws
$ catkin_make
Our system takes as input registered RGB-D frames. It is interfaced with a ROS node.
We provide a lauch file to start our system along an Intel RealSense cameras (D400 series SR300 camera and T265 Tracking Module) and an Rviz GUI. ROS drivers for Intel Realsense Devices can be found here. To use SupersurfelFusion, open a terminal and execute:
$ roslaunch supersurfel_fusion supersurfel_fusion_realsense_rviz.launch
To use SupersurfelFusion with other devices you just need to remap the /camera_info, /image_color and /image_depth topics in the supersurfel_fusion_rviz.launch to the topics published by your sensor or bagfile. The launch file is set assuming depth data in meter. If your depth is given in millimeter you will need to change the parameter depth_scale from 1.0 to 0.001. Then start your device on ROS with registered RGB-D stream, or play your rosbag and execute:
$ roslaunch supersurfel_fusion supersurfel_fusion_rviz.launch
$ roslaunch supersurfel_fusion supersurfel_fusion_realsense.launch
or
$ roslaunch supersurfel_fusion supersurfel_fusion.launch
Run with TUM RGB-D Dataset sequences
$ roslaunch supersurfel_fusion supersurfel_fusion_rgbd_benchmark.launch
We provide two sequences in supersurfel_fusion/rgbd_benchmark that can be specified in the supersurfel_fusion_rgbd_benchmark.launch launch file. When using this launch file, SupersurfelFusion processes every frame and can be played/paused anytime by checking the "stop" boxe of the rqt_reconfigure window that popped up. Estimated and ground truth trajectories are displayed in Rviz and the estimation is saved (location can be specified in the launch file) so it can be used for evaluation with tools provided by the TUM.
The supersurfel_fusion_node, executed by the differents launchfile, is the node that allows to run SupersurfelFusion under ROS.
There are differents parameters you can play with in the different launch files. For instance you can enable/disable the loop closure, enable/disable the moving object detection with or without YOLO...
\image_color\image_depth\camera_info
\superpixels: image of superpixels\slanted_plane: image of slanted plane associated to superpixels\mod_mask: binary mask of detected moving elements\model_supersurfel_marker: global map\frame_supersurfel_marker: frame supersurfels\nodes_marker: nodes of the deformation graph\edges_marker: edges of the deformation graph\constraints_marker: deformation constraints\trajectory: camera path\vo: camera odometry\local_map: sparse vo local map point cloud