Official repository for "Sync-NeRF: Generalizing Dynamic NeRFs to Unsynchronized Videos"
enabling dynamic NeRFs to successfully reconstruct the scene from unsynchroznied dataset.
We provide an integrated requirements file for Sync-MixVoxels and Sync-K-Planes.
pip install -r requirements.txt
You can download our Unsynchronized Dynamic Blender Dataset from this link
We provide example configs for the Unsynchronized Plenoptic Video Dataset and Unsynchronized Dynamic Blender Dataset. You can train the model using the following command:
python train.py --config path/to/config.txt
We also propose a method for optimizing time offsets during test time. You can execute this test-time optimization using the following command:
python train.py --config path/to/config.txt --test-optim
After completing model training, you can perform evaluation using the --render_only 1 flags.
python train.py --config path/to/config.txt --render_only 1 --ckpt path/to/checkpoint.pt
K-Planes offers two versions of config: hybrid and explicit. We provide example configs for the Unsynchronized Plenoptic Video Dataset and Unsynchronized Dynamic Blender Dataset. You can train the model using the following command:
PYTHONPATH='.' python plenoxels/main.py --config-path path/to/config.py
We also propose a method for optimizing time offsets during test time. You can execute this test-time optimization using the following command:
PYTHONPATH='.' python plenoxels/main.py --config-path path/to/config.py --log-dir path/to/logfolder --test_optim
After completing model training, you can perform evaluation using the --validate-only or --rendering-only flags.
@article{Kim2024Sync,
title={Sync-NeRF: Generalizing Dynamic NeRFs to Unsynchronized Videos},
author={Seoha Kim, Jeongmin Bae, Youngsik Yun, Hahyun Lee, Gun Bang, Youngjung Uh},
booktitle={AAAI},
year={2024}}The codes are based on MixVoxels and K-Planes, many thanks to the authors.