PyTorch implementation of paper "RecolorNeRF: Layer Decomposed Radiance Fields for Efficient Color Editing of 3D Scenes" by Bingchen Gong*, Yuehao Wang*, Xiaoguang Han, and Qi Dou.
A novel user-friendly color editing approach for neural radiance fields.
[Paper] [Project Website] [Selected Results]
recolornerf_teaser.mp4
- Sep 18, 2023: The camera-ready version is uploaded to Arxiv.
- Jul 16, 2023: Our paper is accepted by ACM MM 2023! 🎉
- Feb 05, 2023: Initial release of the code.
Tested on Ubuntu 18.04 with PyTorch 1.12.1 and CUDA 11.1.
Type the commands below to set up the running environment.
conda create -n recolornerf python=3.8
conda activate recolornerf
# PyTorch (may need to adapt to your environment)
pip install torch torchvision
# PyTorch3D
conda install pytorch3d -c pytorch3d
# Essentials
pip install tqdm scikit-image opencv-python configargparse lpips imageio-ffmpeg kornia Pillow lpips tensorboard trimesh
conda install -c conda-forge einops
# Palette extraction
conda install -c conda-forge scipy
conda install -c conda-forge cython
CVXOPT_BUILD_GLPK=1 pip install cvxoptDownload our customized initial palettes to data_palette/ for reproducing our layer decomposition results.
For scenes other than the provided ones, you can use the jupyter notebook tools/get_palette.ipynb to generate and customize new palettes.
Type the command below to train a RecolorNeRF model:
python run_recolornerf.py --config configs/chair.txtWe provide our configurations for 15 scenes in the configs/ directory. Remember to change the datadir option to your dataset path. For more options, please refer to the utils/opt.py file.
We create a jupyter notebook tools/color_edit.ipynb for recoloring an optimized RecolorNeRF in a quasi-interactive way. In this jupyter notebook, some simple GUI widgets (like color pickers) and visualization & rendering scripts are provided.
If you find our code or paper is helpful, please consider citing:
@article{gong2023recolornerf,
title={RecolorNeRF: Layer Decomposed Radiance Fields for Efficient Color Editing of 3D Scenes},
author={Gong, Bingchen and Wang, Yuehao and Han, Xiaoguang and Dou, Qi},
journal={arXiv preprint arXiv:2301.07958},
year={2023}
}
- Our code is based on TensoRF and nerf-pytorch (re-implemented by @peihaowang).
- Some of the palette extraction code is borrowed from this repo.