Bad renders after last ControlNet update --

[makeoo1]

i have been using controlnet since a while without any problem. After recently updating all i am getting is "half-renders", like if SD was not completing the steps. I tested it with all the controlnet model but the result is the same

I haven't touched anything in the models folder, just added the new models to the one i already had and always worked.

I am just guessing but I think it has to do with yaml file. Problem is that i am also not sure which yaml file could be casuing the problem, since there a few.

Any help would be very appreciated. Thx a lot!

[wcarletsdrive]

I am getting horrible renders too and I did a fresh install of stable diffusion webui as a friend instructed me, only changed controlnet model cache size to 2 and added an extra controlnet slot and I have this type of quality no matter what. I even followed this tutorial exactly the same and mine looks awful.. What the hell😔 I need a solution ASAP for this extension because I need to use it😔 https://youtu.be/wDM8iDK-yng

[lllyasviel]

ok let us look into it

[wcarletsdrive]

Thank you for looking into it

[lllyasviel]

can you check your setting and see if you mistakenly checked "cfg-based controlnet"?

[wcarletsdrive]

When I reinstalled webui for the fresh install, that setting was off. The only setting checked in controlnet is use mid-control on highres pass. My other install of webui previously before the fresh install is in this picture below. Either way I get bad looking renders that don't represent the style at all. Even if I am using canny or depth mode

l, it all looks horrible

[lllyasviel]

1. uncheck the cfg-based guidance and restart your webui completely, is the problem solved?
2. does this option checked by default when fresh install?

[wcarletsdrive]

1. No, and I used this on the previous install of webui
2. On the fresh install of webui, this isn't checked and I still get the same bad quality

[lllyasviel]

it seems that the "cfg-based guidance" is broken. we will further look into it

[wcarletsdrive]

Thank you so much once again 🙏

[lllyasviel]

are u willing to share the input image that you tested?

[makeoo1]

here's mine

[makeoo1]

i just saw the updates of the issue.

I also followed your instruction but the problem stays

[wcarletsdrive]

First image is the input

Second image is for the style

I also tried using an image that wasn't at 512 x 512, used innerfit in controlnet too.

[lllyasviel]

it seems that the building image is a difficult case for ControlNet and the model fail to recognize it. it is not a bug.
The girl case is likely to be caused by t2i adaper, which is not a part of ControlNet. let me know if you can reproduce the problem without using t2i adaper

[wcarletsdrive]

Sadly on looks I am getting and I got the style adapter from here https://huggingface.co/TencentARC/T2I-Adapter/tree/main/models

The canny, and depth models etc are from your huggingface

[lllyasviel]

Can you get similar quality in webui?

[lllyasviel]

can you send me your screenshot with exactly same setting with me?

[lllyasviel]

For the girl image case, please let me know if the problem exist without using t2i adapter. If the problem exist with pure text prompts, we will look into it deeper.

[wcarletsdrive]

All below are with having the weight in controlnet range from 0.5-1. Also, using your models instead and no adapter. If I have the weight low like 0.15, of course I see my prompt but this defeats the point of me using your extension :( My prompts look beautiful without it, but with the extension, they look like below..

[lllyasviel]

From the result it seems that the model works as intended. If no additional bugs are reported (especially that building image), we will close this soon.
The t2i-adapter is from a 3rd team (Tencent-ARC), and the webui plugin's support for t2i-adapter is experimental - because we are not sure if that model is robust enough for a production environment.
All standard controlnets are well-tested and should be robust.

If you want to transfer the prompt contents into your wanted coarse shape, a better choice could be using the depth model and tune the parameters a bit.

Thank you for your supports as always.

[wcarletsdrive]

Well I appreciate you taking the time looking into this for us at least, thank you for trying, I really wish this would be totally different. But before you go just to make sure, I am doing everything correctly on my end?

[lllyasviel]

Besides, you are encouraged to direct your issue to Tencent-ARC
https://github.com/TencentARC/T2I-Adapter
Nevertheless, please note that they are an independent team and may not have the same way as us to interact with end users.

[lllyasviel]

I closed the bug issue in webui Mikubill/sd-webui-controlnet#555
let me know if the problem still exist with that building image and I will reopen it

[lllyasviel]

And before you report to Tencent-ARC you are encouraged to see if your problem can be reproduced by their official demo https://huggingface.co/spaces/Adapter/T2I-Adapter

[wcarletsdrive]

Thanks for that recommendation. I tried it just now and used a different input image and the issue still proceeds so there is something going on. Its weird because this is a fresh install of webui. SMH..

[lllyasviel]

Unfortunately, this is because that model is not robust enough. The problem even exist in their official demos.
We are not able to do anything more about this. This model belongs to Tencent-ARC

[wcarletsdrive]

Thank God it isn't just me, THAT is good news, so within the two days of people making videos about this tool, something got broken. I reached out to them, and even though this will be closed, can I still send them this thread so they have deeper info and more proof of what I am dealing with?

[lllyasviel]

I am going to close this issue. feel free to let me know if real addtional bugs are reported and I will reopen it.

[lllyasviel]

Thank God it isn't just me, THAT is good news, so within the two days of people making videos about this tool, something got broken. I reached out to them, and even though this will be closed, can I still send them this thread so they have deeper info and more proof of what I am dealing with?

Again, any problem caused by other projects should be directly posted to that project, like https://github.com/TencentARC/T2I-Adapter/issues

[makeoo1]

so followed your settings and it seems like its working now for me. I will test it with more random settings and prompt and let you know how is it going

[makeoo1]

For me everything seems fixed. Thank you guys!