Few shot 3D reconstruction using Score Distillation Sampling

An implementation using NVIDIA Kaolin Wisp and Stable Diffusion that applies the Text-to-3D model DreamFusion to the task of Few-shot / single-view 3D object reconstruction.

Examples

Few-shot 3D object reconstruction

Input RGB views (only 3 sparse views, omitting the right side of the car):

1	2	3

L2 reconstruction loss on the input views + Score Distillation Sampling with the guiding prompt "a yellow sports car":

car_reconstruction_1.webm

Text-to-3D

Score Distillation Sampling with a guiding prompt:

1	2	3

Installation

Follow the instrutions here to install NVIDIA Kaolin Wisp.

Run setup.sh to install the prerequisites.

Usage

Few-shot 3D object reconstruction

python main.py --config configs/diffusion_nerf.yaml --dataset-path /path/to/car/dataset --prompt "a car"

Text-to-3D

python main.py --config configs/diffusion_nerf.yaml --prompt "a DSLR photo of a blue car"