correct a crash in img2img under particular circumstances #2088
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
When using the inpainting model, the following sequence of events would cause a predictable crash: 1. Use unified canvas to outcrop a portion of the image. 2. Accept outcropped image and import into img2img 3. Try any img2img operation This closes #1596. The crash was: ``` operands could not be broadcast together with shapes (320,512) (512,576) Traceback (most recent call last): File "/data/lstein/InvokeAI/backend/invoke_ai_web_server.py", line 1125, in generate_images self.generate.prompt2image( File "/data/lstein/InvokeAI/ldm/generate.py", line 492, in prompt2image results = generator.generate( File "/data/lstein/InvokeAI/ldm/invoke/generator/base.py", line 98, in generate image = make_image(x_T) File "/data/lstein/InvokeAI/ldm/invoke/generator/omnibus.py", line 138, in make_image return self.sample_to_image(samples) File "/data/lstein/InvokeAI/ldm/invoke/generator/omnibus.py", line 173, in sample_to_image corrected_result = super(Img2Img, self).repaste_and_color_correct(gen_result, self.pil_image, self.pil_mask, self.mask_blur_radius) File "/data/lstein/InvokeAI/ldm/invoke/generator/base.py", line 148, in repaste_and_color_correct mask_pixels = init_a_pixels * init_mask_pixels > 0 ValueError: operands could not be broadcast together with shapes (320,512) (512,576) ``` This error was caused by the image and its mask not being of identical size due to the outcropping operation. The ultimate cause of this error has something to do with different code paths being followed in the `inpaint` vs the `omnibus` modules. Since omnibus will be obsoleted by diffusers, I have chosen just to work around the problem rather than track it down to its source. The only ill effect is that color correction will not be applied to the first image created by `img2img` after applying the outcrop and immediately importing into the img2img canvas. Since the inpainting model has less of a color drift problem than the standard model, this is unlikely to be problematic.
hipsterusername
approved these changes
Dec 22, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When using the inpainting model, the following sequence of events would cause a predictable crash:
This closes #1596.
The crash was:
This error was caused by the image and its mask not being of identical size due to the outcropping operation. The ultimate cause of this error has something to do with different code paths being followed in the
inpaintvs theomnibusmodules.Since omnibus will be obsoleted by diffusers, I have chosen just to work around the problem rather than track it down to its source. The only ill effect is that color correction will not be applied to the first image created by
img2imgafter applying the outcrop and immediately importing into the img2img canvas. Since the inpainting model has less of a color drift problem than the standard model, this is unlikely to be problematic.