PIXLRelight: Controllable Relighting via Intrinsic Conditioning

Perceptual Intelligence and Extended Reality Lab, University of Oxford

Miguel Farinha   Ronald Clark

@article{farinha2026pixlrelight,
  title   = {PIXLRelight: Controllable Relighting via Intrinsic Conditioning},
  author  = {Farinha, Miguel and Clark, Ronald},
  journal = {arXiv preprint arXiv:2605.18735},
  year    = {2026}
}

PIXLRelight is a feed-forward model for physically controllable single-image relighting. Given a source photograph and a target lighting condition — specified either as an RGB image or as a path-traced Blender Cycles render — it produces a relit version of the source under the new illumination in under a tenth of a second.

Clone the repository and install the dependencies in a fresh environment:

git clone git@github.com:mlfarinha/pixlrelight.git
cd pixlrelight

conda create -n pixlrelight python=3.11 -y
conda activate pixlrelight
pip install -r requirements.txt

The model weights and config are hosted on Hugging Face and downloaded automatically on first run.

PIXLRelight supports two ways to specify the target lighting condition.

The bundled intrinsic decomposer (Marigold-IID-Lighting) extracts albedo, diffuse shading, and non-diffuse residuals from a reference photograph, which then condition the relighting model. Use this mode when you have a real photograph of the target illumination.

Lay out each sample as a folder containing a source.* and a target.* image:

samples/
├── room00/
│   ├── source.jpg
│   └── target.jpg
├── room01/
│   ├── source.jpg
│   └── target.jpg
└── ...

For physically controllable lighting, condition on a path-traced render of a coarse 3D reconstruction of the scene under user-specified PBR lights. The render passes are loaded directly from disk and converted into the same intrinsic representation the model is trained on. Use this mode when you want full PBR control over the target illumination.

Drop the Cycles passes (auto-discovered by filename) into the sample folder alongside source.*:

samples/
├── ...
├── room02/
│   ├── source.jpg
│   ├── diffuse_color.exr
│   ├── diffuse_direct.exr
│   ├── diffuse_indirect.exr
│   ├── glossy_color.exr
│   ├── glossy_direct.exr
│   ├── glossy_indirect.exr
│   ├── transmission_color.exr        # optional group
│   ├── transmission_direct.exr
│   ├── transmission_indirect.exr
│   ├── volume_direct.exr             # optional group
│   ├── volume_indirect.exr
│   ├── emission.exr                  # optional
│   └── environment.exr               # optional
└── ...

The passes must be HDR EXR files in linear colour space — Cycles' default. If you're exporting through Blender's compositor, make sure the output node is set to OpenEXR with the colour space set to Linear (not sRGB). Filename matching is case- and separator-insensitive, so Diffuse Color.exr, diffuse_color.exr, and Diffuse-Color.exr all resolve to the same pass.

The same command handles both modes, single samples, and batches:

python infer.py --input_dir samples/ --output_dir outputs/

• Layout A (single sample): --input_dir is a sample directory; outputs land directly in --output_dir.
• Layout B (batch): --input_dir contains sample subdirectories; outputs mirror the subdir structure under --output_dir.

Each input is auto-classified per sample by what's on disk: if target.* is present, Mode 1 is used; if Cycles passes are present, Mode 2 is used (and takes precedence if both are available). For each sample, a single <source_stem>_relit.png is written next to (or under) --output_dir.

At a high level, PIXLRelight bridges physically based rendering (PBR) and learned image synthesis through a shared intrinsic conditioning that can be obtained from either real photographs (via Marigold-IID-Lighting) or PBR renders (via Cycles). A transformer-based neural renderer applies the target illumination to the source photograph, preserving fine image detail through a per-pixel affine modulation applied at the source's native resolution.

For full details — training data, architecture, ablations, and additional qualitative results — see the paper and project page.

Thanks to these great repositories: VGGT, Depth Anything 3, DINOv3, Marigold, and many other inspiring works in the community.

Code in this repository is released under the MIT License. The model weights, hosted on Hugging Face, are released under the Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) license.