[Bug]: API returns different results than web-ui

[Theagainmen]

Is there an existing issue for this?

• I have searched the existing issues and checked the recent builds/commits

What happened?

Using the API I retreive different results than over the web-ui. I have found multiple people saying the same, but all issues were closed due to them not having the exact same settings. This might be my case too, however I wouldn't know which setting is causing this then.

Steps to reproduce the problem

1. Run the API with the following payload to the endpoint "/sdapi/v1/txt2img"

        payload = {
           "seed": 180368056,
           "subseed": 0,
           "subseed_strength": 0,
           "seed_resize_from_h": 0,
           "seed_resize_from_w": 0,
           #"denoising_strength": 0.75,
           "prompt": "a cat",
           "steps": 10,
           "cfg_scale": 7.5,
           "sampler_index": "DDIM",
           "save_images": True,
           "alwayson_scripts": {
               "controlnet": {
                   "args": [
                       {
                        "input_image": inputImageBase64,
                        "module": "canny",
                        "model": CONTROLNET_CANNY_MODEL,
                        "weight": 1,
                        "lowvram": False,
                        "threshold_a": 100,
                        "threshold_b": 200,
                        "guidance_start": 0,
                        "guidance_end": 1,
                        "control_mode": "ControlNet is more important",
                        "resize_mode": "Crop and Resize"
                      },
                      {
                        "input_image": inputImageBase64,
                        "module": "reference_only",
                        "model": "None",
                        "weight": 1,
                        "lowvram": False,
                        "guidance_start": 0,
                        "guidance_end": 0.3,
                        "threshold_a": 0.5,                             # Style Fidelity
                        "control_mode": "ControlNet is more important",
                        "resize_mode": "Crop and Resize"
                        }
                    ]
                }
            }
        }
        r = requests.post(url, json=payload)

Make sure to send the input image using this method, otherwise you might upload an image that has slightly different colors, and that could cause different results

        # Read Image in RGB order
        img = cv2.imread('cat.png')[:, :, ::-1]

        # Encode into PNG and send to ControlNet
        retval, bytes = cv2.imencode('.png', img)
        inputImageBase64 = base64.b64encode(bytes).decode('utf-8')

2. Run an image in the web-ui with the same settings as the API:

• Upload the same input images to ControlNet as used by the API, and set the settings the same as shown in the API request. Leave all other settings to default in the web-ui, unless other specified by the API request.

3. Make sure that both images have the same seed, do not touch any default setting, other than defined in the API request.

What should have happened?

The API and the web-ui should have returned the exact same image. However they are different and when using PNG Info there is 1 difference I can find.

The API:

a cat
Steps: 10, Sampler: DDIM, CFG scale: 7.5, Seed: 180368056, Size: 512x512, Model hash: 8067368533, Model: moDi-v1-pruned, Denoising strength: 0, Version: v1.2.1, ControlNet 0: "preprocessor: canny, model: control_v11p_sd15_canny [d14c016b], weight: 1, starting/ending: (0, 1), resize mode: Crop and Resize, pixel perfect: False, control mode: ControlNet is more important, preprocessor params: (512, 100, 200)", ControlNet 1: "preprocessor: reference_only, model: None, weight: 1, starting/ending: (0, 0.3), resize mode: Crop and Resize, pixel perfect: False, control mode: ControlNet is more important, preprocessor params: (512, 0.5, 64)"

The web-ui

a cat
Steps: 10, Sampler: DDIM, CFG scale: 7.5, Seed: 180368056, Size: 512x512, Model hash: 8067368533, Model: moDi-v1-pruned, Version: v1.2.1, ControlNet 0: "preprocessor: canny, model: control_v11p_sd15_canny [d14c016b], weight: 1, starting/ending: (0, 1), resize mode: Crop and Resize, pixel perfect: False, control mode: ControlNet is more important, preprocessor params: (512, 100, 200)", ControlNet 1: "preprocessor: reference_only, model: None, weight: 1, starting/ending: (0, 0.3), resize mode: Crop and Resize, pixel perfect: False, control mode: ControlNet is more important, preprocessor params: (512, 0.5, 64)"

I will save you some time comparing, the only difference is the Denoising strength setting which is added from the API generated image. This should not affect the generated image since I am using txt2img and hires fix is disabled. I've looked into the source code and the default setting for this seems to be set at 0.75. I've tried to sent an API request with this value at 0, 1, 0.75 and 0.70 which all result in an image different than the one generated by the web-ui, which does not show this setting at all.

Commit where the problem happens

89f9faa

What Python version are you running on ?

Python 3.10.x

What platforms do you use to access the UI ?

Windows

What device are you running WebUI on?

Nvidia GPUs (GTX 16 below)

What browsers do you use to access the UI ?

Google Chrome

Command Line Arguments

--lowvram --opt-split-attention --theme dark --api

List of extensions

• ControlNet
• Ultimate Upscale

Console logs

Not relevant, see above for `PNG Info`.

Additional information

What could cause this difference? Is there anything I should change in my API request?

Thanks for any help!

[missionfloyd]

Your code is outputting a BGR image. Remove [:, :, ::-1].

Tested on a fresh installation.

Input

Your code

Your code with [:, :, ::-1] removed

Webui

[Theagainmen]

@missionfloyd thank you so much for taking your time and helping! This indeed fixed it. I specifically used this code because it was on ControlNets wiki, and it warned that if you didn't use it like this it would mess up your input image, but that doesn't seem to be the case.

Resolved 😄

[tomermesser]

Where are you guys seeing the BGR image - [:, :, ::-1] ??

I am having the same problem when sending this call with the img2img controlNet api.

export const IMG2IMG_CONTROLNET_OPTIONS = { init_images: null, denoising_strength: 0.3, image_cfg_scale: 0, mask_blur: 4, inpaint_full_res: false, inpainting_fill: 1, inpaint_full_res_padding: 32, mask: null, prompt: "", seed: -1, batch_size: 1, n_iter: 1, steps: 50, cfg_scale: 15, width: 1024, height: 1024, negative_prompt: "cgi, 3d render, under exposure", sampler_index: "Euler a", alwayson_scripts: { controlnet: { args: [ { input_image: "base64Img", module: ControlNetModule.softedge_hed, model: "control_sd15_hed [fef5e48e]", weight: 2, lowvram: false, pixel_perfect: false, guidance_start: 0, guidance_end: 1, control_mode: ControlNetMode.CONTROL, processor_res: 2048, is_using_inpainting_conditioning: false, resize_mode: "Crop and Resize", }, { input_image: "base64Img", module: ControlNetModule.depth_leres, model: "control_sd15_depth [fef5e48e]", weight: 2, lowvram: false, pixel_perfect: false, guidance_start: 0, guidance_end: 1, control_mode: ControlNetMode.CONTROL, processor_res: 2048, is_using_inpainting_conditioning: false, resize_mode: "Crop and Resize", }, ], }, }, };

For some reason I am getting weird results compared to the web-ui results.
They both has the same setup.