[bug]: outpainting and inpainting It gives a very bad result v4.0rc2

[wolfkingal2000]

Is there an existing issue for this problem?

• I have searched the existing issues

Operating system

Linux

GPU vendor

Nvidia (CUDA)

GPU model

No response

GPU VRAM

15

Version number

4.0rc2

Browser

Opera GX Browser

Python dependencies

[2024-03-15 20:54:08,268]::[uvicorn.access]::INFO --> 23.150.248.50:0 - "POST /api/v1/images/upload?image_category=general&is_intermediate=true HTTP/1.1" 201
[2024-03-15 20:54:09,284]::[uvicorn.access]::INFO --> 23.150.248.50:0 - "POST /api/v1/images/upload?image_category=mask&is_intermediate=true HTTP/1.1" 201
[2024-03-15 20:54:10,093]::[uvicorn.access]::INFO --> 23.150.248.50:0 - "POST /api/v1/queue/default/enqueue_batch HTTP/1.1" 200
[2024-03-15 20:54:10,725]::[uvicorn.access]::INFO --> 23.150.248.50:0 - "GET /api/v1/queue/default/status HTTP/1.1" 200
4% 1/26 [00:00<00:08, 2.86it/s][2024-03-15 20:54:11,502]::[uvicorn.access]::INFO --> 23.150.248.50:0 - "GET /api/v1/images/i/16fbb6b5-b922-4f97-80ab-1b175806c7da.png HTTP/1.1" 200
8% 2/26 [00:00<00:08, 2.86it/s][2024-03-15 20:54:12,124]::[uvicorn.access]::INFO --> 23.150.248.50:0 - "GET /api/v1/images/i/1e1c99bd-6529-4ff0-a59b-748f5ba15645.png HTTP/1.1" 200
23% 6/26 [00:02<00:07, 2.79it/s][2024-03-15 20:54:13,557]::[uvicorn.access]::INFO --> 23.150.248.50:0 - "GET /api/v1/images/i/baa295c7-cfe0-4f71-b5ce-26bbaf194bab.png HTTP/1.1" 200
100% 26/26 [00:09<00:00, 2.79it/s]
[2024-03-15 20:54:22,196]::[InvokeAI]::INFO --> Graph stats: a4850d08-3a3a-48df-ae2c-873a053e51d5
Node Calls Seconds VRAM Used
main_model_loader 1 0.001s 3.201G
clip_skip 1 0.000s 3.201G
compel 2 0.001s 3.201G
noise 1 0.001s 3.201G
img_resize 4 0.423s 3.201G
create_gradient_mask 1 0.028s 3.201G
vae_loader 1 0.000s 3.201G
i2l 1 0.582s 4.476G
denoise_latents 1 9.441s 3.462G
l2i 1 1.285s 5.217G
canvas_paste_back 1 0.280s 3.181G
TOTAL GRAPH EXECUTION TIME: 12.041s
TOTAL GRAPH WALL TIME: 12.054s
RAM used by InvokeAI process: 8.24G (+0.000G)
RAM used to load models: 1.76G
VRAM in use: 3.181G
RAM cache statistics:
Model cache hits: 4
Model cache misses: 0
Models cached: 6
Models cleared from cache: 0
Cache high water mark: 2.74/4.00G

[2024-03-15 20:54:23,498]::[uvicorn.access]::INFO --> 23.150.248.50:0 - "GET /api/v1/images/i/3f7e56a5-630d-4dda-a9bd-c2711cebd012.png HTTP/1.1" 200
[2024-03-15 20:54:23,508]::[uvicorn.access]::INFO --> 23.150.248.50:0 - "GET /api/v1/images/i/fbe9f47f-8e97-4119-a006-3746f9d0a090.png HTTP/1.1" 200
[2024-03-15 20:54:24,859]::[uvicorn.access]::INFO --> 23.150.248.50:0 - "GET /api/v1/images/i/fa109ee5-5dd0-4c5a-8e5a-61779c9bfa63.png HTTP/1.1" 200
[2024-03-15 20:54:27,965]::[uvicorn.access]::INFO --> 23.150.248.50:0 - "GET /api/v1/images/i/fbe9f47f-8e97-4119-a006-3746f9d0a090.png/full HTTP/1.1" 200
[2024-03-15 20:54:53,100]::[uvicorn.access]::INFO --> 23.150.248.50:0 - "PUT /api/v1/queue/default/cancel_by_batch_ids HTTP/1.1" 200
[2024-03-15 20:54:55,770]::[uvicorn.access]::INFO --> 23.150.248.50:0 - "GET /api/v1/queue/default/status HTTP/1.1" 200

What happened

outpainting

inpainting

and this my setting and abort

What you expected to happen

.

How to reproduce the problem

No response

Additional context

No response

Discord username

No response

[psychedelicious]

Hi, two issues:

1. We found an issue with torch 2.2.1 and inpainting models. We'll either fix or roll back the version.
2. You have mismatched torch versions. When you created this issue, torch was pinned to 2.1. How did you install the RC? Did you manually install anything?

[wolfkingal2000]

Hi, two issues:

1. We found an issue with torch 2.2.1 and inpainting models. We'll either fix or roll back the version.
2. You have mismatched torch versions. When you created this issue, torch was pinned to 2.1. How did you install the RC? Did you manually install anything?

I'm using Colab for installation so use git clone, and for Torch, this command fixes the issue:
!pip install --force-reinstall torch==2.1.2 --index-url https://download.pytorch.org/whl/cu121
!pip install -U typing-extensions
!pip install -U fsspec==2023.5.0
!pip3 install -U xformers --index-url https://download.pytorch.org/whl/cu121

Also, there's something else: "invokeai-model-install.py." I've checked the latest version, but it seems to have been removed. can restored back Since I use Colab, it was very useful.
Could you provide any guidance on using new [v4.0.0rc4] installation i installed but dont find invokeai.yaml file ? Thank you.

and for [v4.0.0rc4] when run invokeai-web.py give this error can tell me what is problem

Traceback (most recent call last): File "/usr/local/bin/invokeai-web", line 8, in sys.exit(run_app()) File "/content/InvokeAI/invokeai/app/run_app.py", line 10, in run_app from invokeai.app.api_app import invoke_api File "/content/InvokeAI/invokeai/app/api_app.py", line 19, in from torch.backends.mps import is_available as is_mps_available File "/usr/local/lib/python3.10/dist-packages/torch/init.py", line 237, in from torch._C import * # noqa: F403 ImportError: /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_cuda.so: undefined symbol: ncclCommRegister http (80)

[psychedelicious]

Ok, we can't do anything to help with colab. I just answered a similar question from your other issue.

Let's keep this issue focused to the inpainting issue.

[wolfkingal2000]

Ok, we can't do anything to help with colab. I just answered a similar question from your other issue.

Let's keep this issue focused to the inpainting issue.

It's ok, but I had a question, can you tell me all the prerequisites of the program, maybe I can fix it myself with the required versions, because the problem started with version 4.0.0rc4, 4.0.0rc2 ran without any problems and did not give this error.

[psychedelicious]

You can review https://github.com/invoke-ai/InvokeAI/blob/main/pyproject.toml to see the list of dependencies. Note you probably need the xformers group of optional dependencies too.

[wolfkingal2000]

You can review https://github.com/invoke-ai/InvokeAI/blob/main/pyproject.toml to see the list of dependencies. Note you probably need the xformers group of optional dependencies too.

i fix that problem thanks

but get this error for use in painting model, i use , in v4.0.0rc5

[2024-03-27 19:45:36,699]::[InvokeAI]::INFO --> Graph stats: 7a192474-3be8-41ae-bcb5-4832420f13d7
Node Calls Seconds VRAM Used
main_model_loader 1 0.001s 0.000G
clip_skip 1 0.000s 0.000G
compel 1 0.597s 0.000G
TOTAL GRAPH EXECUTION TIME: 0.598s
TOTAL GRAPH WALL TIME: 0.600s
RAM used by InvokeAI process: 1.28G (+0.030G)
RAM used to load models: 0.00G
RAM cache statistics:
Model cache hits: 0
Model cache misses: 0
Models cached: 0
Models cleared from cache: 0
Cache high water mark: 0.00/0.00G

[2024-03-27 19:45:38,510]::[uvicorn.access]::INFO --> 67.220.86.150:0 - "POST /api/v1/images/upload?image_category=general&is_intermediate=true HTTP/1.1" 201
[2024-03-27 19:45:39,119]::[uvicorn.access]::INFO --> 67.220.86.150:0 - "GET /api/v1/queue/default/status HTTP/1.1" 200
[2024-03-27 19:45:39,255]::[uvicorn.access]::INFO --> 67.220.86.150:0 - "POST /api/v1/images/upload?image_category=mask&is_intermediate=true HTTP/1.1" 201
[2024-03-27 19:45:41,029]::[uvicorn.access]::INFO --> 67.220.86.150:0 - "POST /api/v1/queue/default/enqueue_batch HTTP/1.1" 200
[2024-03-27 19:45:41,085]::[ModelLoadService]::INFO --> Converting /root/invokeai/models/sd-1/main/meinamix_v11-inpainting.safetensors to diffusers format
[2024-03-27 19:45:41,224]::[InvokeAI]::ERROR --> Error while invoking session b389a36a-3695-4c99-800b-9f53f78945a6, invocation 487f135b-e0ef-4fad-9e1f-64e27c9fe533 (compel):
Trying to set a tensor of shape torch.Size([320, 9, 3, 3]) in "weight" (which has shape torch.Size([320, 4, 3, 3])), this look incorrect.
[2024-03-27 19:45:41,224]::[InvokeAI]::ERROR --> Traceback (most recent call last):
File "/content/InvokeAI/invokeai/app/services/session_processor/session_processor_default.py", line 160, in _process
outputs = self._invocation.invoke_internal(
File "/content/InvokeAI/invokeai/app/invocations/baseinvocation.py", line 281, in invoke_internal
output = self.invoke(context)
File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/content/InvokeAI/invokeai/app/invocations/compel.py", line 57, in invoke
tokenizer_info = context.models.load(self.clip.tokenizer)
File "/content/InvokeAI/invokeai/app/services/shared/invocation_context.py", line 339, in load
return self._services.model_manager.load.load_model(model, _submodel_type, self._data)
File "/content/InvokeAI/invokeai/app/services/model_load/model_load_default.py", line 80, in load_model
).load_model(model_config, submodel_type)
File "/content/InvokeAI/invokeai/backend/model_manager/load/load_default.py", line 62, in load_model
model_path = self._convert_if_needed(model_config, model_path, submodel_type)
File "/content/InvokeAI/invokeai/backend/model_manager/load/load_default.py", line 82, in _convert_if_needed
return self._convert_model(config, model_path, cache_path)
File "/content/InvokeAI/invokeai/backend/model_manager/load/model_loaders/stable_diffusion.py", line 90, in _convert_model
convert_ckpt_to_diffusers(
File "/content/InvokeAI/invokeai/backend/model_manager/convert_ckpt_to_diffusers.py", line 46, in convert_ckpt_to_diffusers
pipe = download_from_original_stable_diffusion_ckpt(Path(checkpoint_path).as_posix(), **kwargs)
File "/usr/local/lib/python3.10/dist-packages/diffusers/pipelines/stable_diffusion/convert_from_ckpt.py", line 1472, in download_from_original_stable_diffusion_ckpt
set_module_tensor_to_device(unet, param_name, "cpu", value=param)
File "/usr/local/lib/python3.10/dist-packages/accelerate/utils/modeling.py", line 348, in set_module_tensor_to_device
raise ValueError(
ValueError: Trying to set a tensor of shape torch.Size([320, 9, 3, 3]) in "weight" (which has shape torch.Size([320, 4, 3, 3])), this look incorrect.

[psychedelicious]

ValueError: Trying to set a tensor of shape torch.Size([320, 9, 3, 3]) in "weight" (which has shape torch.Size([320, 4, 3, 3])), this look incorrect.

We just fixed that in #6064 , will be in the next release

[wolfkingal2000]

ValueError: Trying to set a tensor of shape torch.Size([320, 9, 3, 3]) in "weight" (which has shape torch.Size([320, 4, 3, 3])), this look incorrect.

We just fixed that in #6064 , will be in the next release

Thanks and add multi links in query in model manger use to add a lot models together

[psychedelicious]

Please create a separate issue for each feature request or bug report.

[wolfkingal2000]

Hi, two issues:

1. We found an issue with torch 2.2.1 and inpainting models. We'll either fix or roll back the version.
2. You have mismatched torch versions. When you created this issue, torch was pinned to 2.1. How did you install the RC? Did you manually install anything?

hi vr 4.0.0 rc6 out painting and in painting They are still bad i down vr torch but dont fixed that

this my setting
accelerate
0.28.0
compel
2.0.2
cuda
12.1
diffusers
0.27.2
numpy
1.26.4
opencv
4.9.0.80
onnx
1.15.0
pillow
9.4.0
python
3.10.12
torch
2.1.2+cu121
torchvision
0.16.2+cu121
transformers
4.39.1
xformers
Not Installed

[wolfkingal2000]

Hi, two issues:

1. We found an issue with torch 2.2.1 and inpainting models. We'll either fix or roll back the version.
2. You have mismatched torch versions. When you created this issue, torch was pinned to 2.1. How did you install the RC? Did you manually install anything?

I think found the problem in the conversion safetensors or ckpt to diffusers model Inpainting make bad result
and use diffusers like dreamshaper-8-inpainting from https://huggingface.co don't make bad result and for safetensors you can test https://civitai.com/api/download/models/120702 this