SD Webui Diffusion Color Grading

Alpha

^{Experimental... Expect feature-breaking changes~}

This is an Extension for the Automatic1111 Webui, which performs Color Grading during the generation, producing a more neutral and balanced, but also vibrant and contrasty color.

This is the fruition of the joint research between TimothyAlexisVass with their findings, and me with my experience in developing Vectorscope CC

This Extension comes with two main features, Recenter and Normalization:

Recenter

Abstract

TimothyAlexisVass discovered that, the value of the latent noise Tensor often starts off-centered, and the mean of each channel tends to drift away from 0. Therefore, I tried to write an Extension to guide the mean back to 0. However after some trial and error, it was found out that pushing the mean of every channel to 0 often produces a green tint, suggesting that the "center" of each channel might not simply be the same at 0. After experimenting with hundreds of images, with both Anime and Realistic checkpoints, I have located a set of rather suitable values for a neutral and balanced tone.

Effects

When you enable the feature, the output images will not have a biased color tint, and all colors will distribute evenly; Additionally, the brightness will be adjusted so that bright areas are not overblown and dark areas are not clipped, producing a similar effect like the HDR photos taken by smartphones.

Samples

Off | On

Off | On

Normalization

Abstract

By encoding images into latent noise with VAE, TimothyAlexisVass discovered that the values for VAE to decode are usually within a certain range, and thus theorized that if the final latent noise has a smaller value range, then some precision is essentailly wasted. This gave me an idea to write a function that can make the latent noise utilize the full color depth, making the final output more vibrant and contrasty.

Effects

When you enable the feature, the latent noise will attempt to span across the value ranges if possible, before getting decoded by the VAE. As a result, bright areas will get brighter and dark areas will get darker; Additional details may also be introduced in these areas.

This feature is currently disabled during Hires. fix pass

Samples

Off | On

Off | On

_{*Works better when Recenter is also on}

Combined

Finally, you can also enable both features at the same time, creating some really stunning results!

SDXL

Off | On

Some sample images were generated using older versions of the Extension. I will update all of them again when the Extension is finally out of experimental stage.

To Do

• Parameter Settings
  • Such as effect strength
  • However, I also want this to be a simple plug-and-use Extension
• Better SDXL Support

  The current values may produce a pink tint for SDXL

• Better Algorithms

  Currently, for extreme cases (eg. a bowl of oranges), the overall colors will be overcompensating

• Generation InfoText

---

Checkpoints Used:

• UHD-23
• Realistic Vision V5.1
• SDXL Base 1.0 w/ 0.9 VAE

---

Debug

1. This also comes with the script tensor_debug.py
2. You can enable it in the System section of Settings tab
3. It will plot out the values of the Tensor during the generation, and save to the debug folder in output
4. Perhaps you can make some interesting discoveries too?
5. The folder will be deleted whenever the UI is reloaded/closed