[Feature Request]: Explore potential StableDiffusion speed benefits from implementing kernl (Up to 12X faster GPU inference)

[0xdevalias]

Is there an existing issue for this?

• I have searched the existing issues and checked the recent builds/commits

What would your feature do ?

Explore whether implementing kernl would provide speedups for Stable Diffusion.

• https://github.com/ELS-RD/kernl

  • Kernl lets you run PyTorch transformer models several times faster on GPU with a single line of code, and is designed to be easily hackable.

• https://www.reddit.com/r/MachineLearning/comments/ydqmjp/p_up_to_12x_faster_gpu_inference_on_bert_t5_and/

  • Up to 12X faster GPU inference on Bert, T5 and other transformers with OpenAI Triton kernels

Proposed workflow

N/A

Additional information

• Crossposted on:
  • Explore potential speed benefits from implementing kernl (Up to 12X faster GPU inference) huggingface/diffusers#1094
  • Would it be possible to use kernl to speed up Stable Diffusion? ELS-RD/kernl#141
• A few other potential speedup/efficiency gain issues:
  • Stable Diffusion Meta AITemplate with >= 200% performance increase #1625
  • [Feature Request]: colossalai integration #4606
  • [Feature Request]: Explore NVIDIA/TransformerEngine for speed/efficiency #4721

[papuSpartan]

Pulled kernl in order to mess around but it seems that you need an Ampere architecture card(right now) ELS-RD/kernl#133. They are waiting on triton in order to have better GPU compatability.