[Bug]: Got black image when trying to use the SD model 2.1

[ruradium]

Is there an existing issue for this?

• I have searched the existing issues and checked the recent builds/commits

What happened?

Got black image when trying to use the latest SD 2.1 model, even though I copy the v2-inference-v.yaml file and rename to the [model-name].yaml

Steps to reproduce the problem

as discribed above

What should have happened?

should generate image as prompted

Commit where the problem happens

44c46f0

What platforms do you use to access UI ?

Linux

What browsers do you use to access the UI ?

Google Chrome

Command Line Arguments

--api --listen --no-half-vae

Additional information, context and logs

No response

[ProGamerGov]

If you use --no-half it will work, but then it also requires a lot more VRAM to generate larger images.

[bsalberto77]

Same issue here, with Windows 10. :-(

[nousr]

--no-half --no-half-vae --api --listen works for me...but...

https://github.com/Stability-AI/stablediffusion/commit/c12d960d1ee4f9134c2516862ef991ec52d3f59e seems relevant. we may need to export some environment variable to enable fp16 for 2.1

[djdookie]

Use the v2-inference-v.yaml mentioned above. Use this file for the 768 model only, and the https://github.com/Stability-AI/stablediffusion/blob/main/configs/stable-diffusion/v2-inference.yaml (without -v) for the 512 model. Copy it besides your checkpoint file and give it the same name but with yaml extension.

[ProGamerGov]

Theoretically there shouldn't be an issue with using SD 2.1 if SD 2.0 already worked without --no-half , so I'm not sure why its broken.

[miguelgargallo]

Solution here #5506

[ProGamerGov]

@miguelgargallo Adding --no-half isn't really a PR worthy fix as it should work without that argument.

[ProGamerGov]

I did some more testing and I found another way to fix it!

If you enable xformers with --xformers, then you don't have to use --no-half!

[RainfoxAri]

You could try setting the following environment variable.

STABLE_DIFFUSION_COMMIT_HASH="8bde0cf64f3735bb33d93bdb8e28120be45c479b"

and additionally if you want to use half-precision

ATTN_PRECISION=fp16

So for example for the webui-user.bat

set STABLE_DIFFUSION_COMMIT_HASH="8bde0cf64f3735bb33d93bdb8e28120be45c479b"
set ATTN_PRECISION=fp16

This should checkout the stablediffusion-repository with the specified commit on the next launch. And "8bde0cf64f3735bb33d93bdb8e28120be45c479b" specifically is the commit that adds the ATTN_PRECISION environment variable (see https://github.com/Stability-AI/stablediffusion/commit/8bde0cf64f3735bb33d93bdb8e28120be45c479b).

Works for me, but my local fork is a bit diverged from the current master. So someone should retest this. :)

I can confirm black images only happen on 768 models for 2.1 and 2.0. 512 models doesn't produce black images except maybe for GTX 10xx models like before. I really don't have to use --no-half before and I probably can't since I only have 4GB ram. Well I can if I use --lowvram but yeah, I really don't have to before on pre 2.0 models.

we

You could try setting the following environment variable.

STABLE_DIFFUSION_COMMIT_HASH="8bde0cf64f3735bb33d93bdb8e28120be45c479b"

and additionally if you want to use half-precision

ATTN_PRECISION=fp16

So for example for the webui-user.bat

set STABLE_DIFFUSION_COMMIT_HASH="8bde0cf64f3735bb33d93bdb8e28120be45c479b"
set ATTN_PRECISION=fp16

This should checkout the stablediffusion-repository with the specified commit on the next launch. And "8bde0cf64f3735bb33d93bdb8e28120be45c479b" specifically is the commit that adds the ATTN_PRECISION environment variable (see Stability-AI/stablediffusion@8bde0cf).

Works for me, but my local fork is a bit diverged from the current master. So someone should retest this. :)

Where do we put this? STABLE_DIFFUSION_COMMIT_HASH="8bde0cf64f3735bb33d93bdb8e28120be45c479b"

[MegaScience]

I'm on a 1060 6GB, and the v2.1 512 model was returning images while the v2.1 768 model needed additional work to not end up blank. Turning xformers back on did allow the 768 model to properly generate an image for me. Considering almost all my VRAM is used while generating, --no-half probably isn't a viable solution without other flags which would slow the process for me.

Summary: xformers makes the 768 model function on my hardware.

[RainfoxAri]

Where do we put this? STABLE_DIFFUSION_COMMIT_HASH="8bde0cf64f3735bb33d93bdb8e28120be45c479b"

In whatever script you use to launch the webui.
So for windows most likely webui-user.bat, for linux most likely webui-user.sh.

So the webui-user.bat could look something like this (remember to set your COMMANDLINE_ARGS)

@echo off

set PYTHON=
set GIT=
set VENV_DIR=
set COMMANDLINE_ARGS=your command line options
set STABLE_DIFFUSION_COMMIT_HASH="c12d960d1ee4f9134c2516862ef991ec52d3f59e"
set ATTN_PRECISION=fp16

call webui.bat

Summary: xformers makes the 768 model function on my hardware.

Tried xformers with the 768 model before switchting the commit hash. Which worked fine for lower resolutions, but for unusually large pictures like 1920x1080 i kept consistently getting a black screen. I'm on a RTX 3090.

[curtwagner1984]

I did some more testing and I found another way to fix it!

If you enable xformers with --xformers, then you don't have to use --no-half!

Yes, Had the same issue and xformers fixed it.