Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request]: Add "Diffusion attentive attribution maps" to help understand how prompts affects image generation #4592

Closed
1 task done
ghost opened this issue Nov 11, 2022 · 7 comments
Labels
enhancement New feature or request

Comments

@ghost
Copy link

ghost commented Nov 11, 2022

Is there an existing issue for this?

  • I have searched the existing issues and checked the recent builds/commits

What would your feature do ?

https://github.com/castorini/daam would allow users to understand how different parts of prompt affect the generated image. Would require a custom implementation since of the interface the method in the repo doesn't seem to scale well to long prompts at all.

Example of what it can do:

image

Proposed workflow

Probably implemented as a custom extension that can hook up to txt2img (and img2img too) and add some kind of selector/dropdown to choose what parts of the prompt you want to see a heatmap for.

Additional information

No response

@some9000
Copy link

That looks really useful when trying to understand impact of promt keywords. If someone could make it work within the UI that would be amazing.

@eadnams22
Copy link

Seconded!

@Kameronski
Copy link

I believe this can be a very useful extension.

@ghost
Copy link

ghost commented Dec 1, 2022

@pomee4
Copy link

pomee4 commented Dec 1, 2022

Related: https://github.com/benkyoujouzu/stable-diffusion-webui-visualize-cross-attention-extension

This sounds good but i'd prefer the daam style heatmap instead of this one

@ghost
Copy link

ghost commented Dec 2, 2022

#5298

@mezotaken mezotaken added the enhancement New feature or request label Jan 12, 2023
@mezotaken
Copy link
Collaborator

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

5 participants