Skip to content
/ PhySG Public

Code for PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Relighting and Material Editing

License

Notifications You must be signed in to change notification settings

Kai-46/PhySG

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Relighting and Material Editing

Quick start

  • Create conda environment
conda env create -f environment.yml
conda activate PhySG
  • Download example data from google drive.

  • Optimize for geometry and material given a set of posed images and object segmentation masks

cd code
python training/exp_runner.py --conf confs_sg/default.conf \
                              --data_split_dir ../example_data/kitty/train \
                              --expname kitty \
                              --nepoch 2000 --max_niter 200001 \
                              --gamma 1.0
  • Render novel views, relighting and mesh extraction, etc.
cd code
# use same lighting as training
python evaluation/eval.py --conf confs_sg/default.conf \
                              --data_split_dir ../example_data/kitty/test \
                              --expname kitty \
                              --gamma 1.0 --resolution 256 --save_exr
# plug in new lighting                              
python evaluation/eval.py --conf confs_sg/default.conf \
                              --data_split_dir ../example_data/kitty/test \
                              --expname kitty \
                              --gamma 1.0 --resolution 256 --save_exr \
                              --light_sg ./envmaps/envmap3_sg_fit/tmp_lgtSGs_100.npy

Tips: for viewing exr images, you can use tev hdr viewer.

Some important pointers

  • code/model/sg_render.py: core of the appearance modelling that evaluates rendering equation using spherical Gaussians.
    • code/model/sg_envmap_convention.png: coordinate system convention for the envmap.
  • code/model/sg_envmap_material.py: optimizable parameters for the material part.
  • code/model/implicit_differentiable_renderer.py: optimizable parameters for the geometry part; it also contains our foward rendering code.
  • code/training/idr_train.py: SGD optimization of unknown geometry and material.
  • code/evaluation/eval.py: novel view rendering, relighting, mesh extraction, etc.
  • code/envmaps/fit_envmap_with_sg.py: represent an envmap with mixture of spherical Gaussians. We provide three envmaps represented by spherical Gaussians optimized via this script in the 'code/envmaps' folder.

Prepare your own data

  • Organize the images and masks in the same way as the provided data.
  • Make sure object of interest is inside the unit sphere by properly normalizing your camera parameters.
  • As to camera parameters, we follow the same convention as NeRF++ to use OpenCV conventions.

Acknowledgements: this codebase borrows a lot from the awesome IDR work; we thank the authors for releasing their code.

About

Code for PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Relighting and Material Editing

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages