img2img outputs have posterized effect, Codeformer causing artifacts

[cvar66]

Img2img outputs have added posterized effect. The higher the denoise setting the worse it seems to get. Most visible in the shadow areas. After disabling the latest img2img color correction fix, the problem goes away.

more extreme example

Also when I add face correction using Codeformer with default settings weight at 0.5, it causes ugly noise artifacts. Most visible in the highlights. This seems to happen with txt2img too. GFPGAN seems fine.

[RandomLegend]

I have the same issue with colorcorrection. So i second this bugreport

[AUTOMATIC1111]

it's off by default now

[nahojjjen]

I'm still having this issue. Tested changing jpeg save quality from 80 to 100, and saving as png instead, but did not fix the issue.

[cvar66]

The color correction in this fork is broken. Best to just turn it off in the settings tab. It fixes the magenta hue problem but introduces a bunch of other image quality issues.

It's been brought up several times before but from what I have seen, the devs don't seem very interested in fixing it, which to me is surprising as img2img is one of the most popular uses.

[nahojjjen]

Given the popularity and the amount of other features and issues popping up, I think its pretty fair that the devs haven't fixed this issue which is off by default. :) I mean, this issue thread is only 3 days old. I think it's pretty low priority.

[cvar66]

Sorry this wasn't meant as a dig at the devs, I realize they have a ton on their plate and are very busy. I just meant that this issues has been brought up in multiple threads by multiple users and the response from the devs just seems like they don't think it's a problem or is very low priority just like you said.

Seems to me that the users of this fork think it's a bit higher priority then the devs do. It essentially makes loopback unusable, and forces users to manually edit every output that img2img spits out. That to me seems somewhat above "pretty low priority."

[C43H66N12O12S2]

@AUTOMATIC1111 I tested this and can confirm it happens.

[rewbs]

I've raised a PR here that reduces some of the loopback artefacts by allowing you to save before color is applied, but still passing the color-corrected output into the loopback to avoid the magenta shift: #847 – see the PR for examples of the types of improvements it yields, and for why this isn't an easy problem to solve fully.

I also have an alternative loopback script here to allow experimenting with intermitted color correction, periodic recalibration, and static LUTs: https://github.com/rewbs/stable-diffusion-webui/blob/rewbs-cc-experiments-script/scripts/loopback-cc-experiments.py .

That said, all of this is a workaround for the magenta shift issue which recreates on all forks including Dream Studio on the 1.5 model. Based on my discussions so far on the SD discord there doesn't seem to be a solution for this.

[rewbs]

FWIW, here is a 6-way loopback comparison using different colour correction strategies (generated using https://github.com/rewbs/stable-diffusion-webui/blob/rewbs-cc-experiments-script/scripts/loopback-cc-experiments.py ). To really drive the point home, the initial image is black&white, and the prompt is requesting colour. In an ideal world, a loopback would progressively introduce nice colour.

6way.mp4

Config:

• Initial image: Input image: https://i.imgur.com/1d6ib3F.png
• Prompt: Crazy color studio photo portrait of a mad person against an interesting background.
• Params: Steps: 20, Sampler: Euler a, CFG scale: 7, Seed: 8-58, Size: 512x512, Denoising strength: 0.5 (fixed)

The different strategies are:

• no-cc: no colour correction, as expected the magenta skew attacks.
• original-cc: colour correction as it is today on master. There's no major posterisation showing up, probably because the input and output are both b&w.
• cc-1-r-0: the pre-cc save strategy proposed in Add option to save before color correction. This helps with some posterisation issues in img2img loopback. #847. This strategy reduces posterisation in many cases, but it's not visible here (see the PR for examples where it is). Here you can see individual frames sometimes expressing some colour that is not carried back into the loopback. The input into the loopback is rest to the initial image's histogram on every loop.
• cc-5-r0: colour correct every 5 frame back to the initial image's histogram. You can see colour emerging over 5 frames then being wiped back to b&w every 5th frame.
• cc-2-r3: colour correct every 2 frames, and resample the target colour histogram every 3 frames. Here we see colour slowly being allowed in, but the skew to magenta takes hold quickly because the color correction target is picking up the magenta skew.
• cc-5-r9: colour correct every 5 frames, and resample the target colour histogram every 9 frames. Here we see colour being allowed in, with the magenta skew taking hold but more slowly.

So for some scenarios, intermittent colour correction and periodic resampling will yield better results, but we can't avoid the magenta skew altogether.

Next I'd like to experiment with a colour correction target that is an average of N past frames.

[Jonseed]

I just spent hours trying to fix the color banding in all my images before I realized it was the color correction setting. I thought it was something I was putting in the prompts that was causing it... I put many things into the negative prompt to try to fix it, like "posterize," "duotone," "low color depth," and nothing worked, of course.