Community pipeline img2img inpainting

[vvvm23]

Resolves #905

However, it would see the base implementation doesn't actually support Tensor input (only PIL image).

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[patrickvonplaten]

Hey @vvvm23 ,

Could you also add a section to the Examples README and a entry to the table: https://github.com/huggingface/diffusers/blob/main/examples/community/README.md#example-usages ?

This would be very nice :-)

[vvvm23]

Sure, I can do that tomorrow for you ~ I'll also try and fix the bug with tensor inputs, but that may come later.

[vvvm23]

Done, I can't fix the other issues today though, doing this PR from the airport 😅

[patrickvonplaten]

Super cool!

[socathie]

Hi @vvvm23 , thanks for the custom pipeline! It was very useful for my application - just wondering if the example usage script is accurate though? In particular, this line:

pipe = StableDiffusionInpaintPipeline.from_pretrained(
    "runwayml/stable-diffusion-inpainting",
    revision="fp16",
    torch_dtype=torch.float16,
)

seems like this just calls the regular inpaint pipeline and disregard the inner image argument?

I tried using the example but it seems that the entire mask gets inpainted and the inner image isn't pasted.

I would imagine it should be something more like (haven't tested yet):

pipe = DiffusionPipeline.from_pretrained(
    "runwayml/stable-diffusion-inpainting",
    custom_pipeline="img2img_inpainting",
)

[vvvm23]

Good catch, just goes to show you shouldn't write PRs in an airport 😅 I'll change that tomorrow + adds some example results. Likely in a separate PR that I will link to this one.

[vvvm23]

@socathie how does #1370 look to you?

[socathie]

@socathie how does #1370 look to you?

Looking good. Works on my end as well.