Skip to content

Commit

Permalink
Add HDR Script
Browse files Browse the repository at this point in the history
Implement #16
  • Loading branch information
Haoming02 committed Sep 20, 2023
1 parent c981c9b commit 83a9485
Show file tree
Hide file tree
Showing 6 changed files with 151 additions and 21 deletions.
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
### v1.4.6 - 2023 Sep.19
- Add **HDR** Script

### v1.4.5 - 2023 Sep.13
- Bug Fix for Color Wheel in `img2img`
- Minor Formatting
Expand Down
51 changes: 31 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,23 +1,20 @@
# SD Webui Vectorscope CC
This is an Extension for the [Automatic1111 Webui](https://github.com/AUTOMATIC1111/stable-diffusion-webui), which performs *a kind of* **Offset Noise**[*](#offset-noise-tldr) natively.
This is an Extension for the [Automatic1111 Webui](https://github.com/AUTOMATIC1111/stable-diffusion-webui), which performs a kind of **Offset Noise**[*](#offset-noise-tldr) natively,
allowing you to adjust the brightness, contrast, and color of the generations.

> [Sample Images](#sample-images)
## How to Use
After installing this Extension, you will see a new section in both **txt2img** and **img2img** tabs,
refer to the parameters and sample images below and play around with the values.
After installing this Extension, you will see a new section in both **txt2img** and **img2img** tabs.
Refer to the parameters and sample images below and play around with the values.

**Note:** Since this modifies the underlying latent noise, the composition may change drastically.
**Note:** Since this modifies the underlying latent noise, the composition may change drastically. Using the **Ones** scaling seems to reduce the variations.

#### Parameters
- **Enable:** Turn on & off this Extension
- **Enable:** Turn on/off this Extension
- **Alt:** Modify an alternative Tensor instead, causing the effects to be significantly stronger
- **Skip:** Skip the last percentage of steps and only process the first few steps
- *Not as useful now that you can tune the [Scaling Settings](#scaling-settings)*

<p align="center"><img src="samples/Skip.jpg" width=512></p>
<p align="center">When <code>Alt.</code> is enabled, the image can get distorted at high value<br>Increase <code>Skip</code> to still achieve a stronger effect but without distortion</p>

- *Not as useful now since you can tune the [Scaling Settings](#scaling-settings) instead*
- **Brightness:** Adjust the overall brightness of the image
- **Contrast:** Adjust the overall contrast of the image
- **Saturation:** Adjust the overall saturation of the image
Expand Down Expand Up @@ -65,10 +62,8 @@ refer to the parameters and sample images below and play around with the values.
- Click **Refresh Style** to update the `Dropdown` if you edited the `styles.json` directly

#### Advanced Settings

- **Process Hires. fix:** By default, this Extension only functions during the **txt2img** phase, so that **Hires. fix** may "fix" the artifacts introduced during **txt2img**. Enable this to process **Hires. fix** phase too.
- This option does not affect **img2img**
- **Note:** Keep the **txt2img** base `steps` higher than **Hires. fix** `steps` if you enable this

##### Noise Settings
> let `x` denote the Tensor ; let `y` denote the operations
Expand All @@ -95,7 +90,7 @@ refer to the parameters and sample images below and play around with the values.

##### Scaling Settings
Previously, this Extension offsets the noise by the same amount each step.
But due to the denoising process , this may produce undesired outcomes such as blurriness at high **Brightness** or noises at low **Brightness**.
But due to the denoising process, this may produce undesired outcomes such as blurriness at high **Brightness** or noises at low **Brightness**.
Thus, I added a scaling option to modify the offset amount.

> Essentially, the "magnitude" of the default Tensor gets smaller every step, so offsetting by the same amount will have stronger effects at later steps. This is reversed on the `Alt.` Tensor however.
Expand Down Expand Up @@ -174,19 +169,20 @@ Thus, I added a scaling option to modify the offset amount.
- [X] Fix the **Brightness** issues *~~kinda~~*
- [X] Add API Docs
- [X] Add Infotext Support *(by. [catboxanon](https://github.com/catboxanon))*
- [X] ADD **HDR** Script
- [ ] Add Gradient features

<p align="center"><img src="samples/XYZ.jpg" width=768></p>
<p align="center"><code>X/Y/Z Plot Support</code><br><i>(Outdated Contrast Value)</i></p>
<p align="center"><img src="samples/XYZ.jpg" width=768></p>

<p align="center"><code>X/Y/Z Plot w/ Randomize</code></p>
<p align="center"><img src="samples/Random.jpg" width=768></p>
<p align="center">For <b>Randomize</b> in <code>X/Y/Z Plot</code>, the value is used as the random seed<br>
You can refer to the console to see the randomized values</p>
<p align="center">The value is used as the random seed<br>You can refer to the console to see the randomized values</p>

## API
You can use this Extension via [API](https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/API) by adding an entry in the `alwayson_scripts` of your payload.
An [example](samples/api_example.json) is provided.
The `args` are the sent in the following order:
The `args` are sent in the following order:

- **[Enable, Alt, Brightness, Contrast, Saturation, R, G, B, Skip, Process Hires. Fix, Noise Settings, Scaling Settings]**
> `bool`, `bool`, `float`, `float`, `float`, `float`, `float`, `float`, `float`, `bool`, `str`, `str`
Expand All @@ -195,9 +191,24 @@ The `args` are the sent in the following order:
- Does not work with `DDIM`, `UniPC` samplers
- Has little effect when used with certain **LoRA**s

## HDR
<p align="right"><i><b>BETA</b></i></p>

> [Discussion Thread](https://github.com/Haoming02/sd-webui-vectorscope-cc/issues/16)
- In the **Script** `Dropdown` at the bottom, there is now a new option: **`High Dynamic Range`**
- This script will generate multiple images *("Brackets")* of varying brightness, then merge them into 1 HDR image
- *Do provide feedback in the thread!*

#### Settings
- **Brackets:** The numer of images to generate
- **Gaps:** The brightness difference between each image
- **Automatically Merge:** When enabled, this will merge the images using a `OpenCV` algorithm and save to the `HDR` folder in the `outputs` folder; When disabled, this will return all images to the result section
- All the images are still saved to the `outputs` folder regardless

<hr>

#### Offset Noise TL;DR
### Offset Noise TL;DR
The most common *version* of **Offset Noise** you may have heard of is from this [blog post](https://www.crosslabs.org/blog/diffusion-with-offset-noise),
where it was discovered that the noise functions used during **training** were flawed, causing `Stable Diffusion` to always generate images with an average of `0.5`.

Expand All @@ -211,7 +222,7 @@ Though, the results may not be as good as using properly trained models.

<hr>

#### What is Under the Hood
### What is Under the Hood
After reading through and messing around with the code,
I found out that it is possible to directly modify the Tensors
representing the latent noise used by the Stable Diffusion process.
Expand All @@ -228,7 +239,7 @@ hence why you can control the brightness as well as the colors.

<hr>

#### Vectorscope?
### Vectorscope?
The Extension is named this way because the color interactions remind me of the `Vectorscope` found in **Premiere Pro**'s **Lumetri Color**.
Those who are experienced in Color Correction should be rather familiar with this Extension.

Expand Down
6 changes: 6 additions & 0 deletions javascript/vec_cc.js
Original file line number Diff line number Diff line change
Expand Up @@ -126,5 +126,11 @@ onUiLoaded(async () => {
row2.style.alignItems = 'end'
row2.style.gap = '1em'

// ----- HDR UIs -----
const hdr_settings = document.getElementById('vec-hdr-' + mode)
const buttons = hdr_settings.getElementsByTagName('label')

for (let i = 0; i < buttons.length; i++)
buttons[i].style.borderRadius = '0.5em'
})
})
1 change: 1 addition & 0 deletions scripts/cc.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
class VectorscopeCC(scripts.Script):
def __init__(self):
clean_outdated('cc.py')
clean_outdated('cc_hdr.py')

self.xyzCache = {}
xyz_support(self.xyzCache)
Expand Down
109 changes: 109 additions & 0 deletions scripts/cc_hdr.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
import modules.scripts as scripts
import gradio as gr
import numpy as np
import cv2 as cv

from modules.processing import process_images, get_fixed_seed
from copy import copy

# https://docs.opencv.org/4.8.0/d2/df0/tutorial_py_hdr.html
def merge_HDR(imgs:list, path:str, depth:str, fmt:str, gamma:float):
import datetime
import math
import os

output_folder = os.path.join(path, 'hdr')
if not os.path.exists(output_folder):
os.makedirs(output_folder)

imgs_np = [np.array(img, dtype=np.uint8) for img in imgs]

merge = cv.createMergeMertens()
hdr = merge.process(imgs_np)
hdr += math.ceil(0 - np.min(hdr) * 1000) / 1000

#print(f'{np.min(hdr)}, {np.max(hdr)}')

target = (65535 if depth == '16bpc' else 255)
precision = ('uint16' if depth == '16bpc' else 'uint8')

hdr = np.power(hdr, (1 / gamma))

ldr = np.clip(hdr * target, 0, target).astype(precision)
rgb = cv.cvtColor(ldr, cv.COLOR_BGR2RGB)

cv.imwrite(os.path.join(output_folder, f'{datetime.datetime.now().strftime("%H-%M-%S")}{fmt}'), rgb)

class VectorHDR(scripts.Script):

def title(self):
return "High Dynamic Range"

def show(self, is_img2img):
return True

def ui(self, is_img2img):
with gr.Row():
count = gr.Slider(label="Brackets", minimum=3, maximum=9, step=2, value=7)
gap = gr.Slider(label="Gaps", minimum=0.50, maximum=2.50, step=0.25, value=1.50)

with gr.Accordion("Merge Options", elem_id='vec-hdr-' + ('img' if is_img2img else 'txt'), open=False):
auto = gr.Checkbox(label="Automatically Merge", value=True)

with gr.Row():
depth = gr.Radio(['16bpc', '8bpc'], label="Bit Depth", value="16bpc")
fmt = gr.Radio(['.tiff', '.png'], label="Image Format", value=".tiff")

gamma = gr.Slider(label="Gamma", info='Lower: Darker | Higher: Brighter',minimum=0.2, maximum=2.2, step=0.2, value=1.2)

return [count, gap, auto, depth, fmt, gamma]

def run(self, p, count:int, gap:float, auto:bool, depth:str, fmt:str, gamma:float):
center = count // 2

p.seed = get_fixed_seed(p.seed)
p.scripts.script('vectorscope cc').xyzCache.update({
'Enable':'True',
'Alt':'True',
'Skip': 0,
'Brightness': 0,
'DoHR': 'False',
'Method': 'Ones',
'Scaling': '1 - Cos'
})

baseline = process_images(p)
pc = copy(p)

imgs = [None] * count
imgs[center] = baseline.images[0]

brackets = brightness_brackets(count, gap)

for it in range(count):
if it == center:
continue

pc.scripts.script('vectorscope cc').xyzCache.update({
'Enable':'True',
'Alt':'True',
'Skip': 0,
'Brightness': brackets[it],
'DoHR': 'False',
'Method': 'Ones',
'Scaling': '1 - Cos'
})

proc = process_images(pc)
imgs[it] = proc.images[0]

if not auto:
baseline.images = imgs
return baseline
else:
merge_HDR(imgs, p.outpath_samples, depth, fmt, gamma)
return baseline

def brightness_brackets(count, gap):
half = count // 2
return [gap * (i - half) for i in range(count)]
2 changes: 1 addition & 1 deletion scripts/cc_version.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import modules.scripts as scripts
import json

VERSION = 'v1.4.5'
VERSION = 'v1.4.6'

def clean_outdated(EXT_NAME:str):
with open(scripts.basedir() + '/' + 'ui-config.json', 'r') as json_file:
Expand Down

0 comments on commit 83a9485

Please sign in to comment.