Latent Consistency Models Support

[julien-blanchon]

Brief introduction to Latent Consistency Models

Latent Consistency Models (LCMs) are a novel approach in image synthesis that enhance pre-trained Latent Diffusion Models, such as Stable Diffusion 1.5 / SDX, by enabling high-resolution image generation with significantly fewer diffusion steps.
They achieve this by predicting the solution of the underlying ODE instead of predicting the score function and then propagate it.

Paper main page
Paper repo

Current state

The original version of LCMs could be used with two method:

• By applying a LoRA to a pretrain Latent Diffusion Model (LDM), checkpoints available here: Latent Consistency Models LoRAs
• By directly using the Standalone LCMs version, checkpoints available here: Latent Consistency Models Weights

Currently Standalone LCMs are already available in 🤗 Diffusers under LCM Pipeline and LCM Scheduler.

A pretrain version of LCM on SSD1B is available and achieve very good result, I would like to implement it here in Candle because this might enable a fast image generation directly in browser with WASM.

Are you guys interested in implementing LCM here ? The most direct way would be to implement Stable Diffusion LoRA, so it will also enable the usage of any custom LoRA aside of LCM