Image output tiling for seamless textures with Stable Diffusion

[torrinworx]

Is your feature request related to a problem? Please describe.
Currently there is no way to create seamless textures with Stable Diffusion, a crucial feature that is missing.

Describe the solution you'd like
Something similar to this pull on the sd-webui repo: Sygil-Dev/sygil-webui#911

A simple argument in the StableDiffusionPipeline that would enable seamless texture generation for 3D applications.

[anton-l]

Hi @torrinworx! As the Sygil-Dev/sygil-webui#911 PR suggests, you can make the Stable Diffusion models tile-able by patching the torch.nn.Conv2d before loading the pipeline:

# add global options to models
def patch_conv(**patch):
    cls = torch.nn.Conv2d
    init = cls.__init__
    def __init__(self, *args, **kwargs):
        return init(self, *args, **kwargs, **patch)
    cls.__init__ = __init__

patch_conv(padding_mode='circular')
print("patched for tiling")

Native support for tiling in diffusers is unlikely to come, as it would either be hacky, or complicate the model the model design with additional arguments :)

[shirayu]

FYI:
In my app, I toggle to be tile-able or not after pipe loading by change padding_mode in each layer.

1. First, save conv layers and their original padding modes
2. Change padding mode on each generation

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[patrickvonplaten]

Could we make this a community pipeline maybe? :-)

[camenduru]

is it possible for flax
maybe like this: https://flax.readthedocs.io/en/latest/api_reference/_autosummary/flax.linen.Conv.html?highlight=circular

update: I test it working 🎉

[uyo9ko]

Hi @torrinworx! As the sd-webui/stable-diffusion-webui#911 PR suggests, you can make the Stable Diffusion models tile-able by patching the torch.nn.Conv2d before loading the pipeline:

# add global options to models
def patch_conv(**patch):
    cls = torch.nn.Conv2d
    init = cls.__init__
    def __init__(self, *args, **kwargs):
        return init(self, *args, **kwargs, **patch)
    cls.__init__ = __init__

patch_conv(padding_mode='circular')
print("patched for tiling")

Native support for tiling in diffusers is unlikely to come, as it would either be hacky, or complicate the model the model design with additional arguments :)

hello, i follow your advice , add this snippet before loading the pipeline, but there is an error when loading pipeline:

pipe = StableDiffusionImg2ImgPipeline.from_pretrained(
    model_path,
    revision="fp16", 
    torch_dtype=torch.float16,
    use_auth_token=True
)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
[<ipython-input-29-796abcd4f547>](https://localhost:8080/#) in <module>
      6     revision="fp16",
      7     torch_dtype=torch.float16,
----> 8     use_auth_token=True
      9 )
     10 pipe = pipe.to(device)

8 frames
[<ipython-input-19-17c9e93293f0>](https://localhost:8080/#) in __init__(self, *args, **kwargs)
      4     init = cls.__init__
      5     def __init__(self, *args, **kwargs):
----> 6         return init(self, *args, **kwargs, **patch)
      7     cls.__init__ = __init__
      8 

TypeError: __init__() got multiple values for keyword argument 'padding_mode'

can you tell me why?