Skip to content

Conversation

@saddam213
Copy link
Member

Cache tensors from text_encoders to speed up generation when the same prompt is used

Issue: #3

@saddam213
Copy link
Member Author

InputImage should also be cached to avoid running the vae_encoder for the same image

@saddam213
Copy link
Member Author

Good performance improvements for subsequent calls, quickly tested both the TextToImage and ImageToImage


TextToImage - No Cache

TextToImage: 2973.1986ms
TextToImage: 785.7539ms
TextToImage: 462.6996ms
TextToImage: 546.9946ms
TextToImage: 247.2857ms
TextToImage: 562.5287ms
TextToImage: 425.6608ms
TextToImage: 215.4142ms
TextToImage: 306.3986ms
TextToImage: 208.544ms

TextToImage - Cached

TextToImage: 2867.7748ms
TextToImage: 218.2006ms
TextToImage: 216.4643ms
TextToImage: 204.0963ms
TextToImage: 302.4757ms
TextToImage: 468.3393ms
TextToImage: 194.7951ms
TextToImage: 184.639ms
TextToImage: 191.3407ms
TextToImage: 195.7907ms

ImageToImage - No Cache

ImageToImage: 4096.9154ms
ImageToImage: 597.0858ms
ImageToImage: 732.4881ms
ImageToImage: 600.2822ms
ImageToImage: 301.5615ms
ImageToImage: 729.3553ms
ImageToImage: 336.8014ms
ImageToImage: 444.0248ms
ImageToImage: 330.6012ms
ImageToImage: 470.1376ms

ImageToImage - Cached

ImageToImage: 4044.009ms
ImageToImage: 320.9691ms
ImageToImage: 208.1214ms
ImageToImage: 186.4001ms
ImageToImage: 497.2423ms
ImageToImage: 183.8987ms
ImageToImage: 193.4917ms
ImageToImage: 204.4805ms
ImageToImage: 188.0385ms
ImageToImage: 196.0874ms

@saddam213 saddam213 marked this pull request as ready for review November 19, 2025 20:13
@saddam213 saddam213 merged commit bfcb3ad into master Nov 19, 2025
@saddam213 saddam213 deleted the PromptCache branch November 21, 2025 02:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants