support TextDiffuser pipelines natively in Diffusers

[bghira]

Model/Pipeline/Scheduler description

TextDiffuser is an open-source framework and pretrained model series for generating unique depth maps that can be used to guide the model for typography.

It not only has open code and weights, but the dataset is also available, containing ~9.7 million typography samples 🤯

Open source status

• The model implementation is available.
• The model weights are available (Only relevant if addition is not a scheduler).

Provide useful links for the implementation

Original repository: https://github.com/microsoft/unilm/tree/master/textdiffuser
Follow-up repository: https://github.com/microsoft/unilm/tree/master/textdiffuser-2
Demo video (not mine): https://youtu.be/ApcJ1UyLQB8?t=62 (timestamp)

[bghira]

additionally they have a training script based on the Diffusers example t2i script

[sayakpaul]

If they have built it on top of diffusers, then it's already a good sign. I think it might be a good idea for users to use their work instead. But will see and decide based on community interest.

@yiyixuxu FYI.

[MnCSSJ4x]

@bghira From my understanding, we need to create a huggingface model/ a diffuser for this given model, right? If that's the case I would like to try this.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.