Pytorch implementation of our method for learning neural radiance fields that takes advantage of depth supervised by 3D point clouds. It can be used to train NeRF models given only very few input views.
Depth-supervised NeRF: Fewer Views and Faster Training for Free
arXiv 2107.02791, 2021
Kangle Deng1, Andrew Liu2, Jun-Yan Zhu1, Deva Ramanan1,3,
1CMU, 2Google, 3Argo AI
We propose DS-NeRF (Depth-supervised Neural Radi-ance Fields), a model for learning neural radiance fields that takes advantage of depth supervised by 3D point clouds.
NeRF trained with 2 views:
DS-NeRF trained with 2 views:
Install requirements:
pip install -r requirements.txt
You will also need COLMAP installed to compute poses if you want to run on your data.
Download data for the example scene: fern_2v
bash download_example_data.sh
To play with other scenes presented in the paper, download the data here.
You can download the pre-trained models here. Place the downloaded directory in ./logs in order to test it later. See the following directory structure for an example:
├── logs
│ ├── fern_2v # downloaded logs
│ ├── flower_2v # downloaded logs
This step is necessary only when you want to run on your data.
First, place your scene directory somewhere. See the following directory structure for an example:
├── data
│ ├── fern_2v
│ ├── ├── images
│ ├── ├── ├── image001.png
│ ├── ├── ├── image002.png
To generate the poses and sparse point cloud:
python imgs2poses.py <your_scenedir>
Once you have the experiment directory (downloaded or trained on your own) in ./logs,
- to render a video:
python run_nerf.py --config configs/fern_dsnerf.txt --render_only
To train a DS-NeRF on the example fern dataset:
python run_nerf.py --config configs/fern_dsnerf.txt
You can create your own experiment configuration to try other datasets.
If you find this repository useful for your research, please cite the following work.
@article{kangle2021dsnerf,
title={Depth-supervised NeRF: Fewer Views and Faster Training for Free},
author={Kangle Deng, Andrew Liu, Jun-Yan Zhu, and Deva Ramanan},
journal={arXiv preprint arXiv:2107.02791},
year={2021}
}
This code borrows heavily from nerf-pytorch.