Support for imagen-pytorch?

[krrishdholakia]

Hi,

Curious if I could use this alongside the imagen-pytorch opensource model - https://github.com/lucidrains/imagen-pytorch

Any help with this would be great!

[patrickvonplaten]

Hey @krrishdholakia,

Great question! We're still very much in the early phases of developing the library, but the goal is to have a library that can be used as a toolbox to quickly try out new architectures, sampling methods and so forth. So in the case of imagen-pytorch, the idea would be that you can soon use this library to try out different sampling methods as well as different model architectures

[patrickvonplaten]

Hey @krrishdholakia,

Imagen-PyTorch is actually conceptually quite simple as it's just a cascade of diffusion models and a T5 Text-Encoder. The difficulty is that no pre-trained weights are available yet.

If weights are available you could use any of the available schedulers that we provide:

• https://github.com/huggingface/diffusers/tree/main/src/diffusers/schedulers
  and then you could wrap this into a Pipeline with:
  https://github.com/huggingface/diffusers/tree/main/src/diffusers/pipelines

[patrickvonplaten]

Closing for now, feel free to open again if you have more questions. In general, we as Hugging Face would be more than happy to support efforts to replicate Imagen for the OS community