How to add clip loss in Guidence part for stable diffusion latent ?

[marson666]

Hi, thanks for your great work.

I'm trying to add clip loss for stable diffusion latent in the Guidence part, but when i decode image from latent and put it into clip_model, when cal clip_loss with latent, it occurs an error, RuntimeError: One of the differentiated Tensors appears to not have been used in the graph. Set allow_unused=True if this is the desired behavior. How to fix this problem or is there any example to refer?

[johnowhitaker]

There is a community example showing CLIP guidance here that should help: https://github.com/huggingface/diffusers/blob/main/examples/community/clip_guided_stable_diffusion.py