[Bug]: Dynamic prompts/wildcards highly change the results vs firstpass, on the hi-res pass on new hi-res fix.

[Panchovix]

Hi there, really thanks for all your work! There's a new issue, after an update on the dev branch of the webui.

Since AUTOMATIC1111/stable-diffusion-webui@ff0e171 dev commit, if you use it at the same time with wildcards/dynamic prompts, it can use more than one value inside the {|} brackets.

The result is noticeable altered after the hi-res fix pass.

For example, if you use "{green|red|blue|black} eyes", and the choosen token was "green", the result can be a mix of all those eye colours, or a mix of green/red, green/blue, etc

This is more pronounced when using loras/tis with this syntax (for example, "{lora:a1:1|lora:a2:1|lora:a3:1|...}"

The issue get worse when using more than one dyanmic prompt at the same time (aka, random eyes, random hair colours, random hair style, etc)

The issue doesn't happen if the commit is reverted to AUTOMATIC1111/stable-diffusion-webui@3885f8a

And the issue doesn't happen when not using dynamic prompts/wildcards.

No visible error in the console

To reproduce the problem,

1. Generate any prompt with an extense wildcard/dynamic prompt (example if you want to test):
   {blue-gray eyes|orange eyes|light blue eyes|dark red eyes|blind eyes|silver eyes|light brown eyes|light green eyes|dark brown eyes|retro eyes|heart eyes|dark purple eyes|amber eyes|light pink eyes|hazel eyes|yellow eyes|blue eyes|dark green eyes|light red eyes|red eyes|diamond eyes|brown eyes|closed eyes|cat eyes|dark blue eyes|green-hazel eyes|dark pink eyes|purple eyes|white eyes|demon eyes|squinted eyes|gray eyes|pink eyes|light purple eyes|magenta eyes|sparkling eyes|gold eyes|black eyes|green eyes|heterochromia}
2. Enable/select hires fix.
3. Generate the image and see if the eye colour result is the desired one, and not a mix of the others colours.

The result should be only the selected by the wildcard/dynamic prompt, and not a mix of the possible values inside the {} brackets.

[Panchovix]

Okay, found the total culprit and not sure why it happens.

On modules/processing.py, line 1126 to 1135

    def parse_extra_network_prompts(self):
        res = super().parse_extra_network_prompts()

        if self.enable_hr:
            self.hr_prompts = self.all_hr_prompts[self.iteration * self.batch_size:(self.iteration + 1) * self.batch_size]
            self.hr_negative_prompts = self.all_hr_negative_prompts[self.iteration * self.batch_size:(self.iteration + 1) * self.batch_size]

            self.hr_prompts, self.hr_extra_network_data = extra_networks.parse_prompts(self.hr_prompts)

        return res

Seems that the first 2 lines after if self.enable_hr, or, specially the self.hr_prompts = self.all_hr_prompts[self.iteration * self.batch_size:(self.iteration + 1) * self.batch_size] lines makes the issue surface.

If you comment this line, or well all the function, it works as it was before AUTOMATIC1111/stable-diffusion-webui@ff0e171 commit, but then hi-res negative prompt doesn't work by itself. It needs to have any prompt on the positive hires prompt. Now, doing that, makes the issue come back (getting tokens outside the {} or wildcard values), but it is that code that's doing the issue.

I've made a fork with dynamic prompts/wildcards working as they should, but hr positive/negative prompt don't work. hr sampler works as it should.

https://github.com/Panchvzluck/stable-diffusion-webui/tree/dev

https://github.com/Panchvzluck/stable-diffusion-webui/blob/dev/modules/processing.py

[horsepowermini]

Same issue on my pc after last update of automatic1111. Hires prompt in generated image somehow look like {@var1|var2|var3} and generation look bad

[brimston3]

This should be fixed now by pr #483, which sets the hr_all_prompts/hr_all_negative_prompts parameters for upscaling. At least it fixed my problems with it.

[Panchovix]

@brimston3 just tested and it works TON better than before. It seems to be fixed.

But highres positive/negative prompt don't work. Do hr prompt work for you? Maybe after tinkering so much in my branch I broke something.

[brimston3]

hr prompt/hr negative work in auto1111 on main branch. #483 wrecks them both. I made a pr #495 to address hr_prompt/hr_negative a little, but it's not perfect and they are not processed using the dynamic prompt engine. I think this extension's maintainers are going to need to talk about what it should do about combinatorial generation and other extensions modifying all_prompts/negative/hr before a clear way forward on how to handle hr_prompt/hr_negative appears.

tbh, I think the language should be simplified for hr_prompt/negative and only allow cached results by keyword, though I haven't a clue how that should be handled.

[Panchovix]

closing @akx and @brimston3 , since it's fixed by PR #495 and PR #498.