Object-Guided-Relocalisation

Introduction

This is the open source implementation of [1]. Demo video can be found here

[1] Object-Augmented RGB-D SLAM for Wide-Disparity Relocalisation. Y. Ming, X. Yang & A. Calway. IROS, 2021

Detection Network

NOCS network is used here for object detection and pose estimation.

TF1 implementation is the official implementation and tested with CUDA 10.0 & cuDNN 7.41, Python 3.5, Tensorflow 1.14.0 and Keras 2.3.0.

TF2 modification is tested with CUDA 11.1.1 & cuDNN 8.0.5, Python 3.7, Tensorflow nightly 2.5.0 and Keras 2.4.3.

RGB-D SLAM

Install the following dependencies:

• OpenCV, tested on 3.4.13
• CUDA, tested on 11.1.1
• Eigen, tested on 3.3.4
• Pangolin
• Sophus
• (Optional) OpenNI2
• (Optional) Azure Kinnet

Dataset

Dataset can be download here.

Repo Usage

In "main" branch: executable data_path folder_name sequence_number display_or_not

eg.

./bin/vil_const ~/SLAMs/datasets/ BOR 5 true

In "dev-submap" branch: executable sequence_number

eg.

./bin/vil_const 5

Future Improvement

• ICP. Now ICP is done using Depth + Centroids, consider change it to Background + Objects + Centroids;
• Submaps. After introducing submaps, handle track & fuse on different submaps;
• Submaps. Manage the submaps, when to fuse multiple submaps (object and dense) into a single map.
• Loading pose. Orthogonal issue when loading poses from text files.