Siting Zhu · Renjie Qin · Guangming Wang · Jiuming Liu · Hesheng Wang
Table of Contents
sem_guass_slam has been benchmarked with Python 3.10, Pytorch 1.12.1 & CUDA=11.6. The simplest way to install all dependences is to use anaconda and pip in the following steps:
# conda create -n sem_gauss python=3.10
# conda activate sem_gauss
# conda install -c "nvidia/label/cuda-11.6.0" cuda-toolkit
#conda install pytorch==1.12.1 torchvision==0.13.1 torchaudio==0.12.1 cudatoolkit=11.6 -c pytorch -c conda-forge
conda env create --file environment.yml
conda activate semgauss
pip install torch==2.7.1 torchvision==0.22.1 torchaudio==2.7.1 --index-url https://download.pytorch.org/whl/cu128
pip install -r requirements.txt
pip install diff-gaussian-rasterization-w-depth_sem_gauss
# pip install segmentation/facebookresearch_dinov2_mainWe will use the replica dataset as an example to show how to use sem_guass_slam. The following steps are similar for other datasets.
Please download the corresponding pth file of the dinov2 in here and put it as: checkpoints/dinov2_replica.
To run sem_guass_slam, please use the following command:
python sem_gauss.py configs/replica/replica.pyTo see the evaluation of the reconstructed mesh, please use the following command:
python eval_mesh/mesh_eval.pyYou should rewrite the path of the reconstructed mesh flie and true mesh file in eval_mesh/mesh_eval.py. And you can download the true mesh file in here.
Dataroot is ./data0/replica by default. Please change the input_folder path in the scene-specific config files if datasets are stored somewhere else on your machine.
Download the replica data on this website:replica. Note that the Replica data is generated by the authors of iMAP (but hosted by the authors of NICE-SLAM). Please cite iMAP if you use the data.
Please follow the data downloading procedure on the ScanNet website, and extract color/depth frames from the .sens file using this code.
We thank the authors of the following repositories for their open-source code:
- 3D Gaussians
- Dataloaders
- Baselines
If you find our paper and code useful, please cite us:
@article{zhu2024semgauss,
title={Semgauss-slam: Dense semantic gaussian splatting slam},
author={Zhu, Siting and Qin, Renjie and Wang, Guangming and Liu, Jiuming and Wang, Hesheng},
journal={arXiv preprint arXiv:2403.07494},
year={2024}
}