Skip to content

ueda0319/neddf

Repository files navigation

Neural Density-Distance Fields

PyTorch implementation of Neural Distance-Density Field (NeDDF), a 3D representation that reciprocally constrains the distance and density fields.

Neural Density-Distance Fields
Itsuki Ueda1, Yoshihiro Fukuhara2, Hirokatsu Kataoka3, Hiroaki Aizawa4, Hidehiko Shishido1, Itaru Kitahara1
1University of Tsukuba, 2Waseda University, 3National Institute of Advanced Industrial Science and Technology (AIST), 4Hiroshima University
in ECCV 2022(poster)

Prerequisite

  • docker
  • docker-compose
  • nvidia-docker2

How to setup docker container

This repository is based on Ascender project. Please refer to Ascender for detailed instructions on how to set up the host environment.

## Move to the directory where docker-compose.yaml exists.
$ cd /path/to/neddf/environments/gpu

## build and start docker container
$ docker compose up -d core

## Enter docker container with bash
$ docker compose exec core bash

How to run code

You can download the nerf_synthetic_dataset and nerf_llff_dataset proposed in original NeRF paper here. Please place downloaded data to data/

Train NeDDF

In the default configuration, the bunny_smoke dataset is set to learn.

Run

$ poetry run python neddf/scripts/run.py

To use other dataset with data format of nerf_synthetic, please select other dataset directory like follow:

$ poetry run python neddf/scripts/run.py dataset.dataset_dir="data/nerf_synthetic/drums/"

Logs are saved in outputs/{datatime}/. Model parameters are saved to outputs/{datatime}/models/. Rendering results are saved to outputs/{datatime}/render/, images rendered from a camera is in iteration number directory, and visualization of field slices are in fields directory.

Evaluate NeDDF

Run

$ poetry run python neddf/scripts/run_eval.py {pretrained files directory}

For example, use following commands to evaluate neddf model in bunny_smoke scene.

$ poetry run python neddf/scripts/run_eval.py pretrained/bunny_smoke/

You can download pretrained models of nerf_synthetic_dataset from this link

Visualize pretrained fields

To visualize trained distance, density, aux.gradient and color fields, please run following command.

$ poetry run python neddf/scripts/fields_visualizer.py {pretrained files directory}

This visualizer draw fields by 2D slices. Following video is visualization example in lego scene(pretrained/lego)

visualizer_fields.mp4

Visualize for check dataset

To visualize dataset, please run following command (override of dataset config is optional):

$ poetry run python neddf/scripts/dataset_visualizer.py dataset.dataset_dir=data/bunny_smoke/