Image resolution lower than 512x512 results in glitchy or black images

[singh-robbie]

Describe the bug

For even the most common prompt "a photo of an astronaut riding a horse on mars", any image resolution lower than 512x512 either outputs a glitchy image or a complete black (stating it's NSFW). Tried "iphone from 2050" which again generates similar result.
potentialBug_diffuser_imageResolution.zip

Reproduction

from diffusers import StableDiffusionPipeline
import torch

model_id = "runwayml/stable-diffusion-v1-5"
pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16)
pipe = pipe.to("cuda")

prompt = "a photo of an astronaut riding a horse on mars"
image = pipe(prompt, height=128, width=128).images[0]
#run a few times and it should classify most of the images as NSFW

### Logs

_No response_

### System Info

- `transformers` version: 4.26.0
- Platform: Linux-5.10.147+-x86_64-with-glibc2.29
- Python version: 3.8.10
- Huggingface_hub version: 0.12.0
- PyTorch version (GPU?): 1.13.1+cu116 (True)
- Tensorflow version (GPU?): 2.9.2 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

[patrickvonplaten]

Hey @singh-robbie,

In this case, we cannot do much and can only recommend to turn of the safety checker with:

from diffusers import StableDiffusionPipeline
import torch

model_id = "runwayml/stable-diffusion-v1-5"
pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16, safety_checker=None)
pipe = pipe.to("cuda")

prompt = "a photo of an astronaut riding a horse on mars"
image = pipe(prompt, height=128, width=128).images[0]
#run a few times and it should classify most of the images as NSFW

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.