PyTorch implementation of Neural Distance-Density Field (NeDDF), a 3D representation that reciprocally constrains the distance and density fields.
Neural Density-Distance Fields
Itsuki Ueda1,
Yoshihiro Fukuhara2,
Hirokatsu Kataoka3,
Hiroaki Aizawa4,
Hidehiko Shishido1,
Itaru Kitahara1
1University of Tsukuba, 2Waseda University, 3National Institute of Advanced Industrial Science and Technology (AIST), 4Hiroshima University
in ECCV 2022(poster)
- docker
- docker-compose
- nvidia-docker2
This repository is based on Ascender project. Please refer to Ascender for detailed instructions on how to set up the host environment.
## Move to the directory where docker-compose.yaml exists.
$ cd /path/to/neddf/environments/gpu
## build and start docker container
$ docker compose up -d core
## Enter docker container with bash
$ docker compose exec core bash
You can download the nerf_synthetic_dataset and nerf_llff_dataset proposed in original NeRF paper here.
Please place downloaded data to data/
In the default configuration, the bunny_smoke
dataset is set to learn.
Run
$ poetry run python neddf/scripts/run.py
To use other dataset with data format of nerf_synthetic
, please select other dataset directory like follow:
$ poetry run python neddf/scripts/run.py dataset.dataset_dir="data/nerf_synthetic/drums/"
Logs are saved in outputs/{datatime}/
.
Model parameters are saved to outputs/{datatime}/models/
.
Rendering results are saved to outputs/{datatime}/render/
, images rendered from a camera is in iteration number directory, and visualization of field slices are in fields
directory.
Run
$ poetry run python neddf/scripts/run_eval.py {pretrained files directory}
For example, use following commands to evaluate neddf model in bunny_smoke scene.
$ poetry run python neddf/scripts/run_eval.py pretrained/bunny_smoke/
You can download pretrained models of nerf_synthetic_dataset from this link
To visualize trained distance, density, aux.gradient and color fields, please run following command.
$ poetry run python neddf/scripts/fields_visualizer.py {pretrained files directory}
This visualizer draw fields by 2D slices. Following video is visualization example in lego scene(pretrained/lego
)
visualizer_fields.mp4
To visualize dataset, please run following command (override of dataset config is optional):
$ poetry run python neddf/scripts/dataset_visualizer.py dataset.dataset_dir=data/bunny_smoke/