[Bug]: depth marigold is not working

[rltgjqmcpgjadyd]

Is there an existing issue for this?

• I have searched the existing issues and checked the recent builds/commits of both this extension and the webui

What happened?

Marigold preprocessor works fine, but ControlNet is not applied when creating images

Other depth preprocessors will correctly apply ControlNet when generating images

Steps to reproduce the problem

select depth_marigold
select any depth model(SDXL)
put any image
generate any image(SDXL)

What should have happened?

ControlNet must be working well

Commit where the problem happens

webui: dev ddb28b33a3561a360b429c76f28f7ff1ffe282a0
controlnet: a5c0da5
torch: 2.3.0+cu121 xformers: 0.0.26.post1

What browsers do you use to access the UI ?

Google Chrome

Command Line Arguments

--xformers --xformers-flash-attention --skip-python-version-check

List of enabled extensions

built-in extensions
sd-webui-controlnet
sd-webui-controlnet-marigold

Console logs

2024-05-20 08:39:47,281 - ControlNet - INFO - unit_separate = False, style_align = False 20/20 [00:04<00:00,  5.59it/s]
2024-05-20 08:39:47,281 - ControlNet - INFO - Loading model from cache: diffusers_xl_depth_mid [39c49e13]
2024-05-20 08:39:47,291 - ControlNet - INFO - Using preprocessor: depth_marigold
2024-05-20 08:39:47,291 - ControlNet - INFO - preprocessor resolution = 768
*** Error running process: D:\SDWebUI\extensions\sd-webui-controlnet\scripts\controlnet.py
    Traceback (most recent call last):
      File "D:\SDWebUI\modules\scripts.py", line 825, in process
        script.process(p, *script_args)
      File "D:\SDWebUI\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1222, in process
        self.controlnet_hack(p)
      File "D:\SDWebUI\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1207, in controlnet_hack
        self.controlnet_main_entry(p)
      File "D:\SDWebUI\extensions\sd-webui-controlnet\scripts\controlnet.py", line 941, in controlnet_main_entry
        controls, hr_controls, additional_maps = get_control(
                                                 ^^^^^^^^^^^^
      File "D:\SDWebUI\extensions\sd-webui-controlnet\scripts\controlnet.py", line 290, in get_control
        controls, hr_controls = list(zip(*[preprocess_input_image(img) for img in optional_tqdm(input_images)]))
                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "D:\SDWebUI\extensions\sd-webui-controlnet\scripts\controlnet.py", line 290, in <listcomp>
        controls, hr_controls = list(zip(*[preprocess_input_image(img) for img in optional_tqdm(input_images)]))
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "D:\SDWebUI\extensions\sd-webui-controlnet\scripts\controlnet.py", line 242, in preprocess_input_image
        result = preprocessor.cached_call(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^
      File "D:\SDWebUI\extensions\sd-webui-controlnet\scripts\supported_preprocessor.py", line 196, in cached_call
        result = self._cached_call(input_image, *args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "D:\SDWebUI\extensions\sd-webui-controlnet\scripts\utils.py", line 82, in decorated_func
        return cached_func(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "D:\SDWebUI\extensions\sd-webui-controlnet\scripts\utils.py", line 66, in cached_func
        return func(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^
      File "D:\SDWebUI\extensions\sd-webui-controlnet\scripts\supported_preprocessor.py", line 209, in _cached_call
        return self(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^
      File "D:\SDWebUI\extensions\sd-webui-controlnet-marigold\scripts\preprocessor_marigold.py", line 85, in __call__
        depth = pipeline(img, num_inference_steps=slider_1, show_pbar=False)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "D:\SDWebUI\venv\Lib\site-packages\torch\nn\modules\module.py", line 1532, in _wrapped_call_impl
        return self._call_impl(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "D:\SDWebUI\venv\Lib\site-packages\torch\nn\modules\module.py", line 1541, in _call_impl
        return forward_call(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "D:\SDWebUI\extensions\sd-webui-controlnet-marigold\marigold\model\marigold_pipeline.py", line 261, in forward
        depth_latent = self.noise_scheduler.step(
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "D:\SDWebUI\venv\Lib\site-packages\diffusers\schedulers\scheduling_ddim.py", line 406, in step
        alpha_prod_t_prev = self.alphas_cumprod[prev_timestep] if prev_timestep >= 0 else self.final_alpha_cumprod
                            ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
    RuntimeError: indices should be either on cpu or on the same device as the indexed tensor (cpu)

---

Additional information

No response

[Sarcasticest]

I'm having this same issue. I am unsure how to actually install it by the instruction given though, because of my experience. I feel like the issue is there, that ControlNet might not know what to do with the preprocessor because I didn't install it correctly. I added it as an extension and can generate a Preprocessor Preview, but cannot generate an image using the preprocessor.