Fixed incorrect width/height assignment in StableDiffusionDepth2ImgPi…

[antoche]

…peline when passing in tensor

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[patrickvonplaten]

@patil-suraj could you check this?

[patrickvonplaten]

@antoche can you give a bit more background why this change, I'm a bit surprised that we need this. Usually height and width are in the same order whether it's PIL or not

[antoche]

Hi Patrick, the image layout on torch tensors is BxCxHxW, so image[0].shape[-2:] is in the order height, width, not width, height. On the other hand, PIL.Image.size returns 2-tuple (width, height). So the previous code was incorrectly mixing up width and height (which would only be noticed when passing in non-square tensors as input).

This change fixes this bug and makes the code consistent with the rest of the diffusers, see for example

• diffusers/src/diffusers/pipelines/latent_diffusion/pipeline_latent_diffusion_superresolution.py

  Line 112 in 3231712

  height, width = image.shape[-2:]

• diffusers/src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_instruct_pix2pix.py

  Line 283 in e4c356d

  height, width = image.shape[-2:]

[patrickvonplaten]

Hey @antoche,

Thanks a lot for explaining further - you're 100% right!

For future reference, in diffusers we should always have the ... H x W ... layout for PyTorch and Numpy and always have ... W x H ... for PIL. This can also be seen by PIL automatically flipping the axis e.g.:

import numpy as np
import PIL.Image

height = 384
width = 512
image_np = np.ones(height, width, 3)

image = PIL.Image.fromarray(image_np)
image.sive
# gives (512, 384) => (width, height)

cc @yiyixuxu (this might also be important for the VAE refactor PR)

Thanks a lot for the PR @antoche !