Text generation?

[fzyzcjy]

Hi thanks for this great library!

There seems to be some diffusion models that generate text, instead of images. (For example, these two surveys: https://arxiv.org/abs/2303.06574, https://www.semanticscholar.org/paper/Diffusion-models-in-text-generation%3A-a-survey-Yi-Chen/41941f072db18972b610de9979e755afba35f11e). Therefore, it would be great if Diffusers could support this.

[asomoza]

Hi, thanks for your suggestion.

We'll need an actual implementation and the model weights and second, we support models that are actually used by the community so we'll have to wait for that too.

Flux is a good model that's capable of generating text inside an image, if you want to give it a try.

[fzyzcjy]

@asomoza Thank you, but maybe I do not explain it clearly: I mean a diffusion model that generates text (instead of pixels). Usually autoregressive models generate text, while diffusion models generate image. However, as is shown in the surveys above, it seems that we can use diffusion to create text as well.

[asomoza]

oh, I'm just used to images and video that I didn't read that it was for actual text generation.

Sounds interesting, but the answer is still the same one, we need the implementation and weights to evaluate if we can add it.

[fzyzcjy]

I see. For a minimal toy one, I just looked at https://github.com/madaan/minimal-text-diffusion. For more advanced ones, I guess you can read the survey paper and check the open source code for the specific works that interests you.

[asomoza]

thanks a lot, the minimal looks cool as a PoC. I'll read them when I have time and look for an advanced one.

[fzyzcjy]

You are welcome, and looking forward to seeing it in diffusers!

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[fzyzcjy]

👀

[asomoza]

still haven't had the time to look into it, not stale though.

[fzyzcjy]

Take your time!

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.