[df-if II] add additional input checks to ensure the input is divisible by 8

[bghira]

What does this PR do?

Fixes #7842

Adds logic to check_inputs for the IF SuperResolution pipeline so that the user receives a clear error when attempting to run the pipeline with invalid image sizes for the input.

This is possible to hit when using the super-resolution model for upscaling evaluation images during training, if eg. the target 256 pixel resolution is aligned to 8px intervals and then divided by 4 to obtain the input image size. The stage II output resolution will be okay, but the input resolution would be wrong.

I suppose there's other ways to hit the problem, but it's always been a bit murky which input is causing the problems.

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline?
• Did you read our philosophy doc (important for complex PRs)?
• Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

• Pipelines: @sayakpaul @yiyixuxu @DN6

[yiyixuxu]

ohh thanks for looking into this!
What we usually do with our image inputs is that in the preprocessing step we resize it to default height and width that are divisible by 8

diffusers/src/diffusers/image_processor.py

Line 407 in 5823736

def get_default_height_width(

so I think instead of adding the checks, we should just resize it, we can either adding the resize step to the preprocess_image method for the IF pipeline, or we can just refactor the method with the VaeImageProcessor like what we do in rest of the pipelines

diffusers/src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_img2img.py

Line 292 in 5823736

self.image_processor = VaeImageProcessor(vae_scale_factor=self.vae_scale_factor)

what do you think?

[bghira]

i considered it. but because of the nature of this, i didn't really feel comfortable just squishing images on the users' behalf. with the small resolution of the inputs, it really can be noticeably distorted, whereas with SD and SDXL at 512/768/1024px it's far less destructive to adjust the size @yiyixuxu how do you feel about that mindset applied to a 64px model, where it might be somewhere around ~5-7% of the image size we end up adjusting by?

[yiyixuxu]

thanks!

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[bghira]

@yiyixuxu i notice the quality checks failed because of some unnecessary list comprehension. but when i look at it, it seems like the most reasonable way to do it? is there a better way? i would love to learn 😁

[yiyixuxu]

can you run make style again?

[bghira]

@yiyixuxu done

[yiyixuxu]

I think it is better to do this in a separate code block:
So we keep this section as it is to check image_batch_size

and then

if isinstance(image, list):
    check_image_size = image[0]
else:
    check_image_size = image

if isinstance(check_image_size, PIL.Image.Image):
    image_size = check_image_size.size
elif isinstance(check_image_size, torch.Tensor):
   image_size = check_image_size.shape[2:]
elif isinstanc(..., np.ndarray):
    image_size = check_image.shape[:1]

if image_size ....:
    raise ValueError(...)

The current code does not work for list of array or tensors

[yiyixuxu]

can we make sure this test works?

[bghira]

i can't run the test suite locally, i was waiting for it to run on the workflow here

[yiyixuxu]

oh so these are the tests that failed https://github.com/huggingface/diffusers/actions/runs/9044175339/job/24855064736#step:7:15620
I can trigger them again now but I think the results would be the same