Official implementation of Splatt3R: Zero-shot Gaussian Splatting from Uncalibarated Image Pairs
[Project Page], [ArXiv Submission], [Demo]
- Clone Splatt3R
git clone https://github.com/btsmart/splatt3r.git
cd splatt3r- Setup Anaconda Environment
conda env create -f environment.yml
pip install git+https://github.com/dcharatan/diff-gaussian-rasterization-modified- (Optional) Compile the CUDA kernels for RoPE (as in MASt3R and CroCo v2)
cd src/dust3r_src/croco/models/curope/
python setup.py build_ext --inplace
cd ../../../../../We train our model using the pretrained MASt3R_ViTLarge_BaseDecoder_512_catmlpdpt_metric checkpoint from the MASt3R authors, available from the MASt3R GitHub repo. This checkpoint is placed at the file path checkpoints/MASt3R_ViTLarge_BaseDecoder_512_catmlpdpt_metric.pth.
A pretrained Splatt3R model can be downloaded here.
We use ScanNet++ to train our model. We download the data from the official ScanNet++ homepage and process the data using SplaTAM's modified version of the ScanNet++ toolkit. We save the processed data to the 'processed' subfolder of the ScanNet++ root directory.
Our generated test coverage files, and our training and testing splits, can be downloaded here, and placed in data/scannetpp.
The Gradio demo can be run using python demo.py, which loads our trained checkpoint from Hugging Face.
This demo generates a .ply file that represents the scene, which can be downloaded and rendered using online 3D Gaussian Splatting viewers such as here or here.
Our example images and .ply files are available for download here.
Our training run can be recreated by running python main.py configs/main.yaml. Other configurations, such as those for the ablations, can be found in the configs folder.
Forthcoming arXiv citation