SD3 - num_images_per_prompt no longer honoured (throws error)

[zagglez]

Describe the bug

With models prior to SD3, the parameter num_images_per_prompt is honoured, enabling generation of several images per prompt. With sd3-medium an error is generated.
RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 2 but got size 1 for tensor number 1 in the list.
Note: I have insufficient VRAM to run tests without clearing text_encoder_3 and tokenizer_3 and am not sure how to use the
sd3_medium_incl_clips_t5xxlfp8.safetensors variant in a normal diffusers workflow. It is always possible that clearing the T5-xxl has a side-effect of breaking num_images_per_prompt.

Reproduction

import torch
from diffusers import StableDiffusion3Pipeline

pipe = StableDiffusion3Pipeline.from_pretrained(
    "stabilityai/stable-diffusion-3-medium-diffusers",
    text_encoder_3=None,
    tokenizer_3=None,
    torch_dtype=torch.float16
)
pipe.to("cuda")

image = pipe(
    "A cat holding a sign that says hello world",
    negative_prompt="",
    num_inference_steps=28,
    num_images_per_prompt=2,
    guidance_scale=7.0,
).images[0]
image.save("sd3_hello_world-no-T5.png")

Logs

Traceback (most recent call last):
  File "/home/developer/src/hug_test_txt2img_sd3.py", line 12, in <module>
    image = pipe(
  File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/diffusers/pipelines/stable_diffusion_3/pipeline_stable_diffusion_3.py", line 778, in __call__
    ) = self.encode_prompt(
  File "/usr/local/lib/python3.10/dist-packages/diffusers/pipelines/stable_diffusion_3/pipeline_stable_diffusion_3.py", line 413, in encode_prompt
    prompt_embeds = torch.cat([clip_prompt_embeds, t5_prompt_embed], dim=-2)
RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 2 but got size 1 for tensor number 1 in the list.

System Info

• 🤗 Diffusers version: 0.29.0
• Platform: Linux-6.8.0-35-generic-x86_64-with-glibc2.35
• Running on a notebook?: No
• Running on Google Colab?: No
• Python version: 3.10.12
• PyTorch version (GPU?): 2.3.1+cu121 (True)
• Flax version (CPU?/GPU?/TPU?): not installed (NA)
• Jax version: not installed
• JaxLib version: not installed
• Huggingface_hub version: 0.23.4
• Transformers version: 4.41.2
• Accelerate version: 0.31.0
• PEFT version: 0.11.1
• Bitsandbytes version: not installed
• Safetensors version: 0.4.3
• xFormers version: 0.0.27+133d7f1.d20240619
• Accelerator: NVIDIA GeForce RTX 3060, 12288 MiB VRAM
• Using GPU in script?: yes
• Using distributed or parallel set-up in script?: no

Who can help?

No response

[haofanwang]

Cannot reproduce in 0.30.0.dev0

[sayakpaul]

Can you try uninstalling diffusers and try again with a source installation of the library?

pip uninstall diffusers -y && pip install git+https://github.com/huggingface/diffusers

[yiyixuxu]

it should be fixed in here #8558

[sayakpaul]

Closing this with #8558. Feel free to reopen if the issue persists.