[bug]: ControlNet regression

[psychedelicious]

Is there an existing issue for this problem?

• I have searched the existing issues

Operating system

Linux

GPU vendor

Nvidia (CUDA)

GPU model

No response

GPU VRAM

No response

Version number

efea1a8

Browser

FF

Python dependencies

No response

What happened

Using this controlnet: https://huggingface.co/diffusers/controlnet-canny-sdxl-1.0

With this control image:

Which processed to this:

Generating gives me these outputs (all with the same seed as the initial image):

Resize Mode - Resize:

Resize Mode - Crop:

Resize Mode - Fill:

Resize Mode - Resize(Simple):

Example metadata:

{
  "generation_mode": "sdxl_txt2img",
  "positive_prompt": "supre cute tiger cub",
  "negative_prompt": "",
  "width": 1024,
  "height": 1024,
  "seed": 3559600503,
  "rand_device": "cpu",
  "cfg_scale": 7.5,
  "cfg_rescale_multiplier": 0,
  "steps": 42,
  "scheduler": "dpmpp_2m_sde_k",
  "model": {
    "key": "846fa0ba-3014-46b5-bbd8-c0dedf655bb8",
    "hash": "b8b07dda-a67f-47e9-84f9-1fe601154ba5",
    "name": "stable-diffusion-xl-base-1-0",
    "base": "sdxl",
    "type": "main"
  },
  "controlnets": [
    {
      "image": {
        "image_name": "606a6823-24ed-4a8d-94d4-e1ac7db4d9a2.png"
      },
      "control_model": {
        "key": "72cfca92-3445-4861-a98a-21243af7584f",
        "hash": "72a9b8da-01db-4ec7-add7-cf312f8feb4c",
        "name": "controlnet-canny-sdxl-1.0",
        "base": "sdxl",
        "type": "controlnet"
      },
      "control_weight": 1,
      "begin_step_percent": 0,
      "end_step_percent": 1,
      "control_mode": "balanced",
      "resize_mode": "just_resize"
    }
  ],
  "vae": {
    "key": "620a5bd7-b134-4013-9346-5b2efd1dce5d",
    "hash": "9c82ae8b-8fde-4168-a9aa-dbddbb938d23",
    "name": "sdxl-vae-fp16-fix",
    "base": "sdxl",
    "type": "vae"
  },
  "positive_style_prompt": "supre cute tiger cub",
  "negative_style_prompt": "",
  "app_version": "4.0.0rc1"
}

If I randomize the seed, I get some pretty far-out results:

What you expected to happen

ControlNets work

How to reproduce the problem

No response

Additional context

No response

Discord username

No response

[psychedelicious]

Looks like it's a general controlnet issue.

Here's with the sd1.5 canny controlnet, same control image:

And at 1024x1024:

[psychedelicious]

Ok, maybe more general than controlnet. Here's IP Adapter:

[psychedelicious]

I tried to figure out when the issue started occurring going back to prevent commits but I couldn't reproduce the issue. Switched back to main (efea1a8) the issue is no longer occurring. Mysterious...