Caching

Execution Caching

Caching accelerates inference by storing and reusing intermediate outputs of different layers, such as attention and feedforward layers, instead of performing the entire computation at each inference step

It significantly improves generation speed at the expense of more memory and doesn’t require additional training

Tip

You may use caching on models that require larger step counts
But do not use caching on models that are tuned for fewer steps

Note

Which model supports which caching method depends on the model architecture: there is no universal approach, so experimentation is required

Caching methods:

• FasterCache
• FirstBlockCache
• LayerSkip
• MagCache
• PyramidAttentionBroadcast
• TaylorSeerCache
• TextKVCache

See https://huggingface.co/docs/diffusers/v0.40.0/en/optimization/cache for more information