PointMBF: A Multi-scale Bidirectional Fusion Network for Unsupervised RGB-D Point Cloud Registration

This repository represents the official implementation of the paper:

PointMBF: A Multi-scale Bidirectional Fusion Network for Unsupervised RGB-D Point Cloud Registration

Instructions

This code has been tested on

• Python 3.8, PyTorch 1.7.1, CUDA 11.1, GeForce RTX 3090

Requirements

To create a virtual environment and install the required dependences please run:

git clone https://github.com/phdymz/PointMBF.git
conda create --name PointMBF python=3.8
conda activate PointMBF
pip install -r requirements.txt

Make dataset

You need to download the RGB-D version of 3DMatch dataset and ScanNet dataset in advance. Details can refer to URR.

3DMatch

python create_3dmatch_rgbd_dict.py --data_root 3dmatch_train.pkl train
python create_3dmatch_rgbd_dict.py --data_root 3dmatch_valid.pkl valid
python create_3dmatch_rgbd_dict.py --data_root  3dmatch_test.pkl test

ScanNet

python create_scannet_dict.py --data_root scannet_train.pkl train
python create_scannet_dict.py --data_root scannet_valid.pkl valid
python create_scannet_dict.py --data_root scannet_test.pkl test 

Train on 3DMatch

python train.py --name RGBD_3DMatch  --RGBD_3D_ROOT 

Train on ScanNet

python train.py --name ScanNet  --SCANNET_ROOT 

Inference

python test.py --checkpoint --SCANNET_ROOT

Pretrained Model

We provide the pre-trained model of PointMBF in BaiDuyun, Password: pmbf.

Acknowledgments

In this project we use (parts of) the official implementations of the followin works:

• URR (Trainer and dataset)
• D3Feat (Geometric network)
• ScanNet (Make dataset)
• 3DMatch (Make dataset)

We thank the respective authors for open sourcing their methods.

Citation

If you find this code useful for your work or use it in your project, please consider citing:

@article{yuan2023pointmbf,
  title={PointMBF: A Multi-scale Bidirectional Fusion Network for Unsupervised RGB-D Point Cloud Registration},
  author={Yuan, Mingzhi and Fu, Kexue and Li, Zhihao and Meng, Yucong and Wang, Manning},
  journal={arXiv preprint arXiv:2308.04782},
  year={2023}
}