Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Question about Node: CLIP Postive-Negative w/Text #16

Open
Magenta-6 opened this issue Oct 30, 2023 · 6 comments
Open

Question about Node: CLIP Postive-Negative w/Text #16

Magenta-6 opened this issue Oct 30, 2023 · 6 comments

Comments

@Magenta-6
Copy link

@wallish77
I would like to better understand how to use the four fields in the Node CLIP +ve/-ve with Text.

Your Readme . . .
CLIP Postive-Negative w/Text | Same as the above, but with two output nodes to provide the positive and negative inputs to other nodes. Also available as an SDXL version
++++++++++++++++++++++++++++++++++++++++++++++++++++++++
My understanding is that
Field 1. Main Prompt, eg., "a cartoon drawing of a cat sleeping in a hat"
Field 2. Style Prompt, eg., "pastel tones with black outlines in the style of Will Eisener"
Field 3. Negative Prompt, "photograph, 3d render, dog, awake, ugly, bad fingers etc,
Field 4. Negative Style, "monochrome, smooth gradients, vivid colours"

Is that right or something else?
Thanks. I appreciate you making your workflows available.
regards - @magenta6

@wallish77
Copy link
Owner

Yes, that is correct. The SDXL CLIP processor can parse separate style prompts, so all four inputs are provided on that single node for convenience. It would be nice if the node backend for Comfy allowed labels above input boxes or 'placeholder' values that could be used to display the name but would not interfere with typing (i.e. you wouldn't have to delete them before writing your prompt)

@Magenta-6
Copy link
Author

Good point about labels for boxes on the node.
I did some more reading about the two prompt fields.
According to https://www.reddit.com/r/StableDiffusion/comments/15ggn9w/sdxl_mini_study_clip_g_vs_clip_l_best_prompting/

Field 1. +ve prompt using open CLIP Vit/G (Capable of understanding more naturalistic language - with far more parameters)
Field 2. +ve prompt using CLIP Vit/L (Based on tokenized language - and the same as what SD 1.5 was based on.)
Field 3. -ve prompt using open CLIP Vit/G
Field 4. -ve prompt using CLIP Vit/L

So my initial assumptions about separating Content from Style are unfounded.

Some commenters suggest:
a) that best results come from copy pasting +ve promt from field 1. into field 2, and
b) that -ve prompts are not particularly useful unless you need to boil off unwanted content.

The first part of the video below was quite helpful in describing this.
https://www.youtube.com/watch?v=Wk88rK9Zh1Y

Do you know if there's a way to link a Text Node to a .csv file?
This used to be a thing in A1111 which was super useful for collecting and organizing useful prompts.
At the moment I'm opening a second Comfy workspace to extract a prompt which I can c&paste into the first.
cheers
@Magenta-6

@wallish77
Copy link
Owner

That's interesting that the common suggestion was to just replicate the positive prompt(s). I had also read when SDXL first came out that the CLIP Vit/L could be used to provide broad stylistic input because of the fact that the tokenization is not natural language, but I think trends have shifted. I was planning on making a merged SD1.5/SDXL prompt/conditioning node but wondered about how to handle the two additional inputs (that would go unused with 1.5). Looks like I can just ignore then and duplicate in the background!

I believe the WAS Suite node pack allows you to load a line of text from a file, but I've not used the node before. How do you use such CSV loading?

@Magenta-6
Copy link
Author

Thanks for the link to the WAS suite node pack.
I think I already have it but there's so many now that I can't keep up.
Brief scan of the page suggests this might already be a thing - but have yet to understand it properly.

In old A1111 there used to be way to link a spreadsheet of prompt information to a drop down menu.
This meant you could quickly add prompt info to run renders using new models/loras for comparison purposes.
Also gave you a great set of templates to use when developing new prompts.
A1111/1.6 is more sophisticated (and therefore clunky) now

All it is are three columns A (Prompt Label) B (+ve prompt) and C (-ve prompt)
See screenshot of a couple of prompts below.
The flexibility of this is that you could work on your prompts in Excel before firing up SD.
I never imagined that Excel would ever become a tool for artists but such is the world we live in!
Screenshot csv
Cheers @Magenta-6

@wallish77
Copy link
Owner

wallish77 commented Oct 31, 2023

Oh, like the styles.csv file from Auto1111, I get you now! Looks like WAS also has a 'Prompt Styles Selector', which reads a list from what looks like a json file (as opposed to a csv) and outputs a positive/negative prompt from it. Looking at the code, there's reference to loading in the Auto1111 webui styles file, but I'm not sure how that's done.

In your screenshot, did you alter those headers yourself or are those the headers that Auto1111 uses now? I haven't used/updated Auto1111 in a while, so mine has name, prompt, and negative_prompt. I can try to make a node that reads a csv file with fixed headers from a fixed location and see if I can make it compatible with the auto1111 format, but no promises.

@Magenta-6
Copy link
Author

Yes exactly!
I'm still figuring out how to get WAS prompt style selector to work.

PS - Unthinkingly I changed the headers for consistency forgetting how that would stuff the code!!
Stick with name, prompt, and negative_prompt.
cheers

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants