Differentiable Light Transport with Gaussian Surfels via Adapted Radiosity for Efficient Relighting and Geometry Reconstruction
_{Official PyTorch implementation of the ACM SIGGRAPH Asia 2025 paper}

Differentiable Light Transport with Gaussian Surfels via Adapted Radiosity for Efficient Relighting and Geometry Reconstruction
Kaiwen Jiang, Jia-Mu Sun, Zilu Li, Dan Wang, Tzu-Mao Li, Ravi Ramamoorthi

Paper | Project | Video | Data&Results

Abstract: Radiance fields have gained tremendous success with applications ranging from novel view synthesis to geometry reconstruction, especially with the advent of Gaussian splatting. However, they sacrifice modeling of material reflective properties and lighting conditions, leading to significant geometric ambiguities and the inability to easily perform relighting. One way to address these limitations is to incorporate physically-based rendering, but it has been prohibitively expensive to include full global illumination within the inner loop of the optimization. Therefore, previous works adopt simplifications that make the whole optimization with global illumination effects efficient but less accurate. In this work, we adopt Gaussian surfels as the primitives and build an efficient framework for differentiable light transport, inspired from the classic radiosity theory. The whole framework operates in the coefficient space of spherical harmonics, enabling both diffuse and specular materials. We extend the classic radiosity into non-binary visibility and semi-opaque primitives, propose novel solvers to efficiently solve the light transport, and derive the backward pass for gradient optimizations, which is more efficient than auto-differentiation. During inference, we achieve view-independent rendering where light transport need not be recomputed under viewpoint changes, enabling hundreds of FPS for global illumination effects, including view-dependent reflections using a spherical harmonics representation. Through extensive qualitative and quantitative experiments, we demonstrate superior geometry reconstruction, view synthesis and relighting than previous inverse rendering baselines, or data-driven baselines given relatively sparse datasets with known or unknown lighting conditions.

Requirements

• We have done all the experiments on the Linux platform with an NVIDIA 6000 Ada GPU. We have also tested on NVIDIA A10 GPUs. Generally, an NVIDIA GPU with 24GB+ memory is recommended.
• CUDA toolkit 12.1 or later. (Why is a separate CUDA toolkit installation required? We use the OptiX 7.7 in our custom CUDA extensions.)
• Python libraries: see environment.yml for exact library dependencies. You can use the following commands with Miniconda3 to create and activate your Python environment.

conda env create -f environment.yml
conda activate radiosity_gs
./install.sh

Getting started

Dataset Preparation

We currently only support the format as specified in GS^3 where the location and intensity of point lights are provided, or the standard Blender/COLMAP format where a fixed environment map is assumed.

We require scaling the scene within the unit cube (as we limit the maximum scaling of Gaussian primitives) and using the HDR images for supervision (as we do not apply tone mapper). Exceptions to these requirements may demand hyper-parameters tuning and applying a tone mapper.

To synthesize a dataset using the Blender, we use the script from NRHints. Important: Synthetic scene sometimes features an non-zero background strength. Please set that to 0 as we do not model an ambient light in our model.

Training

Please use the following command to train a scene:

python train.py -s <path to dataset>

Commandline arguments for regularizations:

--lambda_dist            <number>       # hyperparameter for depth distortion
--solver_type            <PR | hybrid>  # solver for light transport.
                                        # - PR      => Direct Illumination
                                        # - hybrid  => Global Illumination

Tip: For a new scene, it is recommended to start without the depth distortion loss and only introduce it gradually once the baseline results are reasonable.

Testing

Please use the following command to synthesize novel views, perform relighting, or extract the mesh:

python render.py -m <path to pre-trained model>

Commandline arguments for adjusting the TSDF fusion:

--voxel_size                            # voxel size
--depth_trunc                           # depth truncation
--solver_type            <PR | hybrid>  # solver for light transport.
                                        # - PR      => Direct Illumination
                                        # - hybrid  => Global Illumination

If voxel_size and depth_trunc are not specified, the script will automatically estimate them using the camera information.

For Stanford-ORB dataset, the script will automatically trigger the mesh quality evaluation.

Inference

For pure inference without re-calculating the light transport, please first call the renderGI function as usual to calculate the light transport once, and then pass in the returned radiosity to the override_radiosities argument when the viewpoint changes.

An example is provided here:

# Calculate the light transport once
render_pkg = renderGI(viewpoint_cam, gaussians, light_sources, ...)
radiosity = render_pkg['radiosity']
# Pure inference witout re-calculating the light transport
renderGI(viewpoint_cam, gaussians, light_sources, ..., override_radiosities=radiosity)

Full Evaluation

Dataset Preparation

We provide our sparse-view relighting dataset at here. Stanford-ORB dataset can be found at here. We have re-implemented the algorithm for improved efficiency, which has slightly impacted performance compared to the original paper. The re-implemented codebase is optimized towards sparse-view relighting, while slightly affecting the performance on Stanford-ORB dataset.

You can report either the numbers from the paper or from this implementation, as long as they are discussed in a comparable setting.

For sparse-view relighting, please use the following command:

python scripts/view_synthesis_eval.py --dataset <path to the sparse-view relighting dataset>

Table Results

	PSNR (25 Views)	SSIM (25 Views)	LPIPS (25 Views)	PSNR (50 Views)	SSIM (50 Views)	LPIPS (50 Views)
Paper (Global Illumination)	25.29	0.8950	0.0985	27.43	0.9151	0.0830
Paper (Direct Illumination)	24.36	0.8794	0.1085	26.24	0.8982	0.0942
Reproduced (Global Illumination)	25.55	0.9001	0.0922	27.65	0.9186	0.0789
Reproduced (Direct Illumination)	24.46	0.8828	0.1035	26.37	0.9007	0.0894

For Stanford-ORB dataset, please use the following command:

python scripts/stanford_orb_eval.py --dataset <path to the Stanford-ORB dataset>

Table Results

	CD (x 10^3)	PSNR (Test Views)	SSIM (Test Views)	LPIPS (Test Views)	PSNR (Relighting)	SSIM (Relighting)	LPIPS (Relighting)
Paper	0.173	39.30	0.9894	0.0251	32.72	0.9721	0.0365
Reproduced	0.181	39.09	0.9890	0.0254	32.37	0.9706	0.0356

Acknowledgements

Besides GFSGS and 2DGS, this project also utilizes/takes inspire from NRHints, GS^3, ndg-fitting and torchoptix. Many thanks to the authors!

Citation

@article{jiang2025radiositygs,
  author  = {Jiang, Kaiwen and Sun, Jia-Mu and Li, Zilu and Wang, Dan and Li, Tzu-Mao and Ramamoorthi, Ravi},
  title   = {Differentiable Light Transport with Gaussian Surfels via Adapted Radiosity for Efficient Relighting and Geometry Reconstruction},
  year    = {2025},
  journal = {ACM Transactions on Graphics (TOG)}, 
  number  = {6}, 
  volume  = {44}, 
  month   = {December}
}

Local extensions in this fork

The README content above is from the RadiosityGS authors. This section documents functionality added in this fork.

Local-light rigs and fitting

This fork adds fixed and trainable point-light and spotlight rigs. A rig may contain any number of lights, provided every light has the same type. Use optimize_light.py to fit a frozen trained scene to one or more local lights. For OLAT datasets, fit a single image with --target_view because each image can have a different input light.

# Fit two point lights and render the saved rig.
python optimize_light.py -m output/50_Hotdog --light_type point --num_lights 2 --target_view 0
python render.py -m output/50_Hotdog \
    --lights_file output/50_Hotdog/light_optimization/<run>/point_lights.json \
    --skip_novel --skip_mesh

# Fit two spotlights. Their cone cutoff and falloff are fixed during fitting.
python optimize_light.py -m output/50_Hotdog --light_type spot --num_lights 2 \
    --cutoff_deg 20 --sigma_deg 12 --target_view 0

Use relight_spotlight.py to place and render fixed spotlights. Repeat each per-light option for multiple lights:

python relight_spotlight.py -m output/50_Hotdog --num_lights 2 \
    --light_pos 2 1 3 --target 0 0 0 --intensity 10 8 6 \
    --light_pos -2 1 3 --target 0 0 0 --intensity 6 8 10 --all_views

Both workflows write a human-readable JSON rig; render.py --lights_file loads it. Intensities are linear RGB:

{
  "type": "point",
  "lights": [
    {"position": [2, 1, 3], "intensity": [10, 8, 6]},
    {"position": [-2, 1, 3], "intensity": [6, 8, 10]}
  ]
}

For spotlights, use "type": "spot" and add direction (or target), cutoff_deg, and sigma_deg to each light. Mixed point/spot rigs are not supported. Spotlights are represented with band-limited spherical harmonics, so very narrow cones are necessarily softer than their requested shape.

Creating Blender datasets

scripts/render_blender_radiositygs.py renders a Blender scene into the GS^3-compatible HDR dataset format used here. It normalizes the scene to the unit cube, removes unmodelled illumination, and stores per-frame point-light metadata. Run it through Blender (not CPython):

blender -b ~/data/Cycles.blend --python scripts/render_blender_radiositygs.py -- \
    --output ~/data/Cycles_radiositygs

Useful options include --layout cornell for Cornell-box camera/light placement, --train-views, --test-views, --resolution, --samples, and --dry-run to inspect normalization without rendering.

Depth-backed point initialization

For synthetic Blender scenes, scripts/render_blender_depth_init.py can render depth maps for an existing generated dataset and back-project them into points3d.ply. This is optional geometry-assisted initialization, not an image-only/SfM baseline:

blender -b ~/data/Cycles.blend --python scripts/render_blender_depth_init.py -- \
    --dataset ~/data/Cycles_radiositygs

Use --points-per-view, --max-views, and --resolution to control the initializer. The source .blend file is not modified by either Blender tool.