[Bug]: CUDA error at ~60% each time. 4070 - 8gb vram - 16gb DDR5 ram. #268

Evilander · 2023-10-31T20:08:00Z

Is there an existing issue for this?

I have searched the existing issues and checked the recent builds/commits of both this extension and the webui

Have you read FAQ on README?

I have updated WebUI and this extension to the latest version

What happened?

RuntimeError: CUDA error: device-side assert triggered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.

This error happens each time I try and run animatediff. I have tried at 256x256 and 512x512. I have tried xformers and opt-sdp. I have tried turning prompt travel off and on. I have tried all of the motion models, including the newest. I am not sure what the problem is?

Steps to reproduce the problem

Go to .... animatediff tab - enable it
Set frames to 24, 8fps
Simple prompt and a prompt travel attempt
Cuda error as soon as the image frames start to show up

What should have happened?

Create a working gif or an mp4.

Commit where the problem happens

webui: SD1111 - newest version
extension: animatediff

What browsers do you use to access the UI ?

Google Chrome

Command Line Arguments

opt-sdp-attention, medvram-sdxl, no-half-vae, 

xformers, force-enable-xformers, medvram-sdxl, no-half-vae, opt-channelslast

I've also tried simpler command line args, without no half and channelslast.

Console logs

2023-10-31 15:01:48,019 - AnimateDiff - INFO - AnimateDiff process start.
2023-10-31 15:01:48,019 - AnimateDiff - INFO - Loading motion module mm_sd_v15.ckpt from C:\stable-diffusion-webui\extensions\sd-webui-animatediff\model\mm_sd_v15.ckpt
Calculating sha256 for C:\stable-diffusion-webui\extensions\sd-webui-animatediff\model\mm_sd_v15.ckpt: cf16ea656cb16124990c8e2c70a29c793f9841f3a2223073fac8bd89ebd9b69a
2023-10-31 15:01:49,767 - AnimateDiff - INFO - Guessed mm_sd_v15.ckpt architecture: MotionModuleType.AnimateDiffV1
2023-10-31 15:01:51,114 - AnimateDiff - WARNING - Missing keys <All keys matched successfully>
2023-10-31 15:01:51,616 - AnimateDiff - INFO - Hacking SD1.5 GroupNorm32 forward function.
2023-10-31 15:01:51,616 - AnimateDiff - INFO - Injecting motion module mm_sd_v15.ckpt into SD1.5 UNet input blocks.
2023-10-31 15:01:51,617 - AnimateDiff - INFO - Injecting motion module mm_sd_v15.ckpt into SD1.5 UNet output blocks.
2023-10-31 15:01:51,617 - AnimateDiff - INFO - Setting DDIM alpha.
2023-10-31 15:01:51,624 - AnimateDiff - INFO - Injection finished.
2023-10-31 15:01:51,624 - AnimateDiff - INFO - Hacking CFGDenoiser forward function.
2023-10-31 15:01:51,625 - AnimateDiff - INFO - Hacking ControlNet.
 65%|█████████████████████████████████████████████████████▎                            | 13/20 [00:08<00:04,  1.63it/s]C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [220,0,0], thread: [0,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [220,0,0], thread: [1,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [220Exception in thread MemMon:
Traceback (most recent call last):
  File "C:\ProgramData\anaconda3\lib\threading.py", line 1016, in _bootstrap_inner
,0,0], thread: [2,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [220,0,0], thread: [3,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [220    self.run()
  File "C:\stable-diffusion-webui\modules\memmon.py", line 53, in run
,0    free, total = self.cuda_mem_get_info()
  File "C:\stable-diffusion-webui\modules\memmon.py", line 34, in cuda_mem_get_info
    return torch.cuda.mem_get_info(index)
  File "C:\stable-diffusion-webui\venv\lib\site-packages\torch\cuda\memory.py", line 618, in mem_get_info
,0], thread: [4,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [220,0,0    return torch.cuda.cudart().cudaMemGetInfo(device)
RuntimeError: CUDA error: device-side assert triggered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

], thread: [5,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [220,0,0], thread: [6,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [220,0,0], thread: [7,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [220,0,0], thread: [8,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [220,0,0], thread: [9,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [220,0,0], thread: [10,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [220,0,0], thread: [11,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [220,0,0], thread: [12,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [220,0,0], thread: [13,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [220,0,0], thread: [14,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [220,0,0], thread: [15,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [220,0,0], thread: [16,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [220,0,0], thread: [17,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [220,0,0], thread: [18,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [220,0,0], thread: [19,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [220,0,0], thread: [20,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [220,0,0], thread: [21,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [220,0,0], thread: [22,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [220,0,0], thread: [23,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [220,0,0], thread: [24,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [220,0,0], thread: [25,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [220,0,0], thread: [26,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [220,0,0], thread: [27,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [220,0,0], thread: [28,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [220,0,0], thread: [29,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [220,0,0], thread: [30,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [220,0,0], thread: [31,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
 65%|█████████████████████████████████████████████████████▎                            | 13/20 [00:09<00:04,  1.43it/s]
*** Error completing request
*** Arguments: ('task(7d78lchb1y6u693)', 'a woman lifting up her breasts for a group of women', '', ['NEgative'], 20, 'DPM++ 2M Karras', 1, 1, 7, 256, 256, False, 0.7, 2, 'Latent', 0, 0, 0, 'Use same checkpoint', 'Use same sampler', '', '', [], <gradio.routes.Request object at 0x00000187AD40AA70>, 0, False, '', 0.8, -1, False, -1, 0, 0, 0, False, False, {'ad_model': 'face_yolov8n.pt', 'ad_prompt': '', 'ad_negative_prompt': '', 'ad_confidence': 0.3, 'ad_mask_k_largest': 0, 'ad_mask_min_ratio': 0, 'ad_mask_max_ratio': 1, 'ad_x_offset': 0, 'ad_y_offset': 0, 'ad_dilate_erode': 4, 'ad_mask_merge_invert': 'None', 'ad_mask_blur': 4, 'ad_denoising_strength': 0.4, 'ad_inpaint_only_masked': True, 'ad_inpaint_only_masked_padding': 32, 'ad_use_inpaint_width_height': False, 'ad_inpaint_width': 512, 'ad_inpaint_height': 512, 'ad_use_steps': False, 'ad_steps': 28, 'ad_use_cfg_scale': False, 'ad_cfg_scale': 7, 'ad_use_checkpoint': False, 'ad_checkpoint': 'Use same checkpoint', 'ad_use_vae': False, 'ad_vae': 'Use same VAE', 'ad_use_sampler': False, 'ad_sampler': 'DPM++ 2M Karras', 'ad_use_noise_multiplier': False, 'ad_noise_multiplier': 1, 'ad_use_clip_skip': False, 'ad_clip_skip': 1, 'ad_restore_face': False, 'ad_controlnet_model': 'None', 'ad_controlnet_module': 'None', 'ad_controlnet_weight': 1, 'ad_controlnet_guidance_start': 0, 'ad_controlnet_guidance_end': 1, 'is_api': ()}, {'ad_model': 'None', 'ad_prompt': '', 'ad_negative_prompt': '', 'ad_confidence': 0.3, 'ad_mask_k_largest': 0, 'ad_mask_min_ratio': 0, 'ad_mask_max_ratio': 1, 'ad_x_offset': 0, 'ad_y_offset': 0, 'ad_dilate_erode': 4, 'ad_mask_merge_invert': 'None', 'ad_mask_blur': 4, 'ad_denoising_strength': 0.4, 'ad_inpaint_only_masked': True, 'ad_inpaint_only_masked_padding': 32, 'ad_use_inpaint_width_height': False, 'ad_inpaint_width': 512, 'ad_inpaint_height': 512, 'ad_use_steps': False, 'ad_steps': 28, 'ad_use_cfg_scale': False, 'ad_cfg_scale': 7, 'ad_use_checkpoint': False, 'ad_checkpoint': 'Use same checkpoint', 'ad_use_vae': False, 'ad_vae': 'Use same VAE', 'ad_use_sampler': False, 'ad_sampler': 'DPM++ 2M Karras', 'ad_use_noise_multiplier': False, 'ad_noise_multiplier': 1, 'ad_use_clip_skip': False, 'ad_clip_skip': 1, 'ad_restore_face': False, 'ad_controlnet_model': 'None', 'ad_controlnet_module': 'None', 'ad_controlnet_weight': 1, 'ad_controlnet_guidance_start': 0, 'ad_controlnet_guidance_end': 1, 'is_api': ()}, False, 'Selected Style For All Images', 'sdxl_styles.json', 'base', <scripts.animatediff_ui.AnimateDiffProcess object at 0x00000187AD40B100>, <scripts.controlnet_ui.controlnet_ui_group.UiControlNetUnit object at 0x00000187AD40B6D0>, <scripts.controlnet_ui.controlnet_ui_group.UiControlNetUnit object at 0x00000187AD409780>, <scripts.controlnet_ui.controlnet_ui_group.UiControlNetUnit object at 0x0000018780C0F460>, None, False, '0', '0', 'inswapper_128.onnx', 'CodeFormer', 1, True, 'None', 1, 1, False, True, 1, 0, 0, False, 0.5, True, False, False, 'None', 20, False, False, 'positive', 'comma', 0, False, False, '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0, False, None, None, False, None, None, False, None, None, False, 50, 'linear (weight sum)', '10', 'C:\\stable-diffusion-webui\\extensions\\stable-diffusion-webui-prompt-travel\\img\\ref_ctrlnet', 'Lanczos', 2, 0, 0, 'mp4', 10.0, 0, '', True, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, 'linear', 'lerp', 'token', 'random', '30', 'fixed', 1, '8', None, 'Lanczos', 2, 0, 0, 'mp4', 10.0, 0, '', True, False, False) {}
    Traceback (most recent call last):
      File "C:\stable-diffusion-webui\modules\call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
      File "C:\stable-diffusion-webui\modules\call_queue.py", line 36, in f
        res = func(*args, **kwargs)
      File "C:\stable-diffusion-webui\modules\txt2img.py", line 55, in txt2img
        processed = processing.process_images(p)
      File "C:\stable-diffusion-webui\modules\processing.py", line 732, in process_images
        res = process_images_inner(p)
      File "C:\stable-diffusion-webui\extensions\sd-webui-animatediff\scripts\animatediff_cn.py", line 119, in hacked_processing_process_images_hijack
        return getattr(processing, '__controlnet_original_process_images_inner')(p, *args, **kwargs)
      File "C:\stable-diffusion-webui\modules\processing.py", line 867, in process_images_inner
        samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts)
      File "C:\stable-diffusion-webui\modules\processing.py", line 1140, in sample
        samples = self.sampler.sample(self, x, conditioning, unconditional_conditioning, image_conditioning=self.txt2img_image_conditioning(x))
      File "C:\stable-diffusion-webui\modules\sd_samplers_kdiffusion.py", line 235, in sample
        samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
      File "C:\stable-diffusion-webui\modules\sd_samplers_common.py", line 261, in launch_sampling
        return func()
      File "C:\stable-diffusion-webui\modules\sd_samplers_kdiffusion.py", line 235, in <lambda>
        samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
      File "C:\stable-diffusion-webui\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
        return func(*args, **kwargs)
      File "C:\stable-diffusion-webui\repositories\k-diffusion\k_diffusion\sampling.py", line 594, in sample_dpmpp_2m
        denoised = model(x, sigmas[i] * s_in, **extra_args)
      File "C:\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\stable-diffusion-webui\extensions\sd-webui-animatediff\scripts\animatediff_infv2v.py", line 277, in mm_cfg_forward
        x_out = mm_sd_forward(self, x_in, sigma_in, cond_in, image_cond_in, make_condition_dict) # hook
      File "C:\stable-diffusion-webui\extensions\sd-webui-animatediff\scripts\animatediff_infv2v.py", line 188, in mm_sd_forward
        x_in[_context], sigma_in[_context],
    RuntimeError: CUDA error: device-side assert triggered
    CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
    For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
    Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.


---
Traceback (most recent call last):
  File "C:\stable-diffusion-webui\venv\lib\site-packages\gradio\routes.py", line 488, in run_predict
    output = await app.get_blocks().process_api(
  File "C:\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1431, in process_api
    result = await self.call_function(
  File "C:\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1103, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "C:\stable-diffusion-webui\venv\lib\site-packages\anyio\to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "C:\stable-diffusion-webui\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "C:\stable-diffusion-webui\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "C:\stable-diffusion-webui\venv\lib\site-packages\gradio\utils.py", line 707, in wrapper
    response = f(*args, **kwargs)
  File "C:\stable-diffusion-webui\modules\call_queue.py", line 77, in f
    devices.torch_gc()
  File "C:\stable-diffusion-webui\modules\devices.py", line 51, in torch_gc
    torch.cuda.empty_cache()
  File "C:\stable-diffusion-webui\venv\lib\site-packages\torch\cuda\memory.py", line 133, in empty_cache
    torch._C._cuda_emptyCache()
RuntimeError: CUDA error: device-side assert triggered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

Additional information

No response

The text was updated successfully, but these errors were encountered:

clear326 · 2023-10-31T23:03:12Z

I have the literal exact same problem, 4070 and all. I have tried using xformers, not using xformers, disabling xformers, updating pip and torch, and so many other adjustments. Though I could never get a single gif to generate, it would just stop at around 60% and throw that same error.

continue-revolution · 2023-10-31T23:57:18Z

I have no idea what's going on on your side and I cannot reproduce your error. What's your pytorch version?

Try following:

revert AnimateDiff to v1.10.0 by running git checkout v1.10.0 via command line inside your AnimateDiff extension folder
reinstall PyTorch via [Bug]: 'NoneType' object has no attribute 'restore' #180 (comment)

if 1 and 2 do not work, try fresh re-install A1111, ControlNet and this extension, and see what's going on.

thezveroboy · 2023-11-01T10:53:41Z

i have this problem too. i try to use xformers and sdp but it doesn't affect. is there any other way to fix it instead of deleting whole a1111 and all dependencies?

continue-revolution · 2023-11-01T10:56:09Z

I have no idea why you meet this problem. Revert back to v1.10.0 is a good first step, to test if the problem is because of the update from v1.10.0 to v1.11.0

marberchen · 2023-11-01T15:07:34Z

I have the literal exact same problem, 4070 and all. I have tried using xformers, not using xformers, disabling xformers, updating pip and torch, and so many other adjustments. Though I could never get a single gif to generate, it would just stop at around 60% and throw that same error.

3060 12g have same problem

marberchen · 2023-11-01T15:13:21Z

I have the literal exact same problem, 4070 and all. I have tried using xformers, not using xformers, disabling xformers, updating pip and torch, and so many other adjustments. Though I could never get a single gif to generate, it would just stop at around 60% and throw that same error.

3060 12g have same problem

版本: v1.6.0 • python: 3.10.11 • torch: 2.0.1+cu118 • xformers: 0.0.21 • gradio: 3.41.2 • checkpoint: 4a5eb827f4
which part should i update？

continue-revolution · 2023-11-01T15:53:07Z

Who can try reverting AnimateDiff extension to v1.10.0? Please let me know whether or not v1.10.0 works for you. The problem actually pop up after v1.11.0 update.

Read my comments above for how to do it.

continue-revolution · 2023-11-01T15:55:50Z

There is no way for me to reproduce your problem. Your experiments are the only source for me to try addressing the problem.

continue-revolution · 2023-11-01T16:37:25Z

Another thing you can do - screenshot your webui to let me know ALL your configuration

continue-revolution · 2023-11-01T16:38:07Z

I don’t believe this is a VRAM problem. There must be somewhere that has a bug.

continue-revolution · 2023-11-01T16:39:54Z

https://github.com/continue-revolution/sd-webui-animatediff/blob/master/scripts/animatediff_infv2v.py#L187

Add print(_context) before this line. Indentation same as L187. Post here the terminal log.

continue-revolution · 2023-11-01T17:06:34Z

After some quick experiments, I believe that channelslast is one of the causes of the problem. Without channeles last, the vram consumption is only A: 5.33 GB, R: 8.16 GB, Sys: 12.0/23.6445 GB (50.6%), but with channels last, it becomes A: 11.25 GB, R: 13.29 GB, Sys: 17.1/23.6445 GB (72.2%). Please remove channels last and re-try,

GiusTex · 2023-11-01T17:08:32Z

Who can try reverting AnimateDiff extension to v1.10.0?

Info:

python: 3.10.6 • torch: 2.0.1+cu118 • opt-sdp-attention
COMMANDLINE_ARGS=--theme=dark --api --no-half-vae --deepdanbooru --opt-sdp-attention --opt-split-attention --opt-channelslast (I have also tried just with --theme=dark --api --deepdanbooru --opt-sdp-attention)
Webui options: "When you have --xformers in your command line args, you want AnimateDiff to:
Optimize attention layers with sdp (torch >= 2.0.0 required)"
I've also set in the webui-user:

set PYTORCH_CUDA_ALLOC_CONF=garbage_collection_threshold:0.6,max_split_size_mb:128
set CUDA_LAUNCH_BLOCKING=1
set TORCH_USE_CUDA_DSA=1

What I tried:

different torch versions (2.0.1+cu118, 2.1.0+cu118, 2.2.0.dev20231101+cu118 as suggested here except the fact that if I use cu117 I get torchvision version error);

Change torch version

2.0.1+cu118: I added to COMMANDLINE_ARGS: --reinstall-torch

2.1.0+cu118: I opened cmd and used stable-diffusion-webui\venv\Scripts\activate.bat (as suggested here), than: pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118

2.2.0.dev20231101+cu118: pip install numpy --pre torch torchvision --force-reinstall --index-url https://download.pytorch.org/whl/nightly/cu118 (as I said above, suggestion as suggested here)

last and v1.10.0 version of the extension;
yesterday I tried also with xformers, always without success.

Some times I succeded generating the images (not the gif) continuing to free the cuda cache (as suggested here), but now I can no more, I don't know why.

Free cuda cache

I opened cmd again and used stable-diffusion-webui\venv\Scripts\activate.bat, than I run the script:

import torch
import subprocess as sp
from threading import Timer

allowedGpu=10000 # gpu in mb
waitTime=4.0 # wait time in seconds

def get_gpu_memory():
    output_to_list = lambda x: x.decode('ascii').split('\n')[:-1]
    ACCEPTABLE_AVAILABLE_MEMORY = 1024
    COMMAND = "nvidia-smi --query-gpu=memory.used --format=csv"
    try:
        memory_use_info = output_to_list(sp.check_output(COMMAND.split(),stderr=sp.STDOUT))[1:]
    except sp.CalledProcessError as e:
        raise RuntimeError("command '{}' return with error (code {}): {}".format(e.cmd, e.returncode, e.output))
    memory_use_values = [int(x.split()[0]) for i, x in enumerate(memory_use_info)]
    return memory_use_values

def print_gpu_memory_every_5secs():
    gpu_usage = get_gpu_memory()

    if any(x >= allowedGpu for x in gpu_usage):
        print("Current gpu used:", gpu_usage)

        print("Clearing the cache...")
        torch.cuda.empty_cache()
        print("Cache cleared.")

        Timer(5.0, print_gpu_memory_every_5secs).start()
    else:
        print("Current gpu used:", gpu_usage)
        print("No action required.")    

print_gpu_memory_every_5secs()

When it worked I got:

File not found: mm_zoo.json
NoneType' object has no attribute 'save_infotext_img
NoneType' object has no attribute 'save_infotext_txt

and I didn't use motion lora, just mm_sd_v15_v2.safetensors.

Error log:

with context added here:

  mm_cn_select(_context)
  print(_context)
  out = self.inner_model(

Already up to date.
Torch version: 2.0.1+cu118
venv "C:\Users\wgius\Desktop\stable-diffusion-webui\venv\Scripts\Python.exe"
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Version: v1.6.0
Commit hash: 5ef669de080814067961f28357256e8fe27544f4
Launching Web UI with arguments: --theme=dark --api --deepdanbooru --opt-sdp-attention
no module 'xformers'. Processing without...
no module 'xformers'. Processing without...
No module 'xformers'. Proceeding without it.
[-] ADetailer initialized. version: 23.11.0, num models: 9
Using sqlite file: C:\Users\wgius\Desktop\stable-diffusion-webui\extensions\sd-webui-agent-scheduler\task_scheduler.sqlite3
2023-11-01 17:51:56,925 - ControlNet - INFO - ControlNet v1.1.411
ControlNet preprocessor location: C:\Users\wgius\Desktop\stable-diffusion-webui\extensions\sd-webui-controlnet\annotator\downloads
2023-11-01 17:51:57,044 - ControlNet - INFO - ControlNet v1.1.411
[ControlNet-Travel] extension Mikubill/sd-webui-controlnet found, ControlNet-Travel loaded :)
Loading weights [f243aabb88] from C:\Users\wgius\Desktop\stable-diffusion-webui\models\Stable-diffusion\cardosAnime_v20.safetensors
2023-11-01 17:51:59,109 - AnimateDiff - INFO - Hacking i2i-batch.
Creating model from config: C:\Users\wgius\Desktop\stable-diffusion-webui\configs\v1-inference.yaml
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 18.8s (prepare environment: 3.3s, import torch: 3.5s, import gradio: 0.9s, setup paths: 0.8s, initialize shared: 0.2s, other imports: 0.7s, setup codeformer: 0.1s, load scripts: 5.6s, create ui: 2.4s, gradio launch: 0.8s, app_started_callback: 0.3s).
Loading VAE weights specified in settings: C:\Users\wgius\Desktop\stable-diffusion-webui\models\VAE\orangemix.vae.pt
Applying attention optimization: sdp... done.
Model loaded in 6.8s (load weights from disk: 0.6s, create model: 0.7s, apply weights to model: 2.7s, load VAE: 1.6s, calculate empty prompt: 1.0s).
2023-11-01 17:52:18,004 - AnimateDiff - INFO - AnimateDiff process start.
2023-11-01 17:52:18,005 - AnimateDiff - INFO - Loading motion module mm_sd_v15_v2.safetensors from C:\Users\wgius\Desktop\stable-diffusion-webui\extensions\sd-webui-animatediff\model\mm_sd_v15_v2.safetensors
2023-11-01 17:52:18,072 - AnimateDiff - INFO - Guessed mm_sd_v15_v2.safetensors architecture: MotionModuleType.AnimateDiffV2
2023-11-01 17:52:20,495 - AnimateDiff - WARNING - Missing keys <All keys matched successfully>
2023-11-01 17:52:21,228 - AnimateDiff - INFO - Injecting motion module mm_sd_v15_v2.safetensors into SD1.5 UNet middle block.
2023-11-01 17:52:21,228 - AnimateDiff - INFO - Injecting motion module mm_sd_v15_v2.safetensors into SD1.5 UNet input blocks.
2023-11-01 17:52:21,228 - AnimateDiff - INFO - Injecting motion module mm_sd_v15_v2.safetensors into SD1.5 UNet output blocks.
2023-11-01 17:52:21,230 - AnimateDiff - INFO - Setting DDIM alpha.
2023-11-01 17:52:21,232 - AnimateDiff - INFO - Injection finished.
2023-11-01 17:52:21,233 - AnimateDiff - INFO - Hacking lora to support motion lora
2023-11-01 17:52:21,233 - AnimateDiff - INFO - Hacking CFGDenoiser forward function.
2023-11-01 17:52:21,233 - AnimateDiff - INFO - Hacking ControlNet.
  0%|                                                                                           | 0/22 [00:00<?, ?it/s][0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]
                                                                                                                       [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]s]
  5%|███▊                                                                               | 1/22 [00:24<08:29, 24.27s/it][0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]
                                                                                                                       [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]t]
  9%|███████▌                                                                           | 2/22 [00:45<07:30, 22.53s/it][0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]
                                                                                                                       [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]t]
 14%|███████████▎                                                                       | 3/22 [01:07<06:59, 22.10s/it][0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]
                                                                                                                       [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]t]
 18%|███████████████                                                                    | 4/22 [01:28<06:31, 21.78s/it][0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]
                                                                                                                       [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]t]
 23%|██████████████████▊                                                                | 5/22 [01:50<06:08, 21.70s/it][0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]
                                                                                                                       [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]t]
 27%|██████████████████████▋                                                            | 6/22 [02:12<05:51, 21.98s/it][0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]
                                                                                                                       [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]t]
 32%|██████████████████████████▍                                                        | 7/22 [02:34<05:28, 21.92s/it][0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]
                                                                                                                       [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]t]
 36%|██████████████████████████████▏                                                    | 8/22 [02:55<05:04, 21.72s/it][0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]
                                                                                                                       [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]t]
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [832,0,0], thread: [64,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [832,0,0], thread: [65,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [832,0,0], thread: [66,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [832,0,0], thread: [67,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [832,0,0], thread: [68,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [832,0,0], thread: [69,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [832,0,0], thread: [70,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [832,0,0], thread: [71,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [832,0,0], thread: [72,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [832,0,0], thread: [73,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [832,0,0], thread: [74,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [832,0,0], thread: [75,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [832,0,0], thread: [76,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [832,0,0], thread: [77,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [832,0,0], thread: [78,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [832,0,0], thread: [79,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [832,0,0], thread: [80,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [832,0,0], thread: [81,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [832,0,0], thread: [82,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [832,0,0], thread: [83,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [832,0,0], thread: [84,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [832,0,0], thread: [85,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [832,0,0], thread: [86,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [832,0,0], thread: [87,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [832,0,0], thread: [88,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [832,0,0], thread: [89,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [832,0,0], thread: [90,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [832,0,0], thread: [91,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [832,0,0], thread: [92,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [832,0,0], thread: [93,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [832,0,0], thread: [94,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [832,0,0], thread: [95,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
 36%|██████████████████████████████▏                                                    | 8/22 [03:06<05:26, 23.35s/it]Exception in thread MemMon:

Traceback (most recent call last):
  File "C:\Users\wgius\AppData\Local\Programs\Python\Python310\lib\threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "C:\Users\wgius\Desktop\stable-diffusion-webui\modules\memmon.py", line 53, in run
    free, total = self.cuda_mem_get_info()
  File "C:\Users\wgius\Desktop\stable-diffusion-webui\modules\memmon.py", line 34, in cuda_mem_get_info
    return torch.cuda.mem_get_info(index)
  File "C:\Users\wgius\Desktop\stable-diffusion-webui\venv\lib\site-packages\torch\cuda\memory.py", line 618, in mem_get_info
    return torch.cuda.cudart().cudaMemGetInfo(device)
RuntimeError: CUDA error: device-side assert triggered
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

*** Error completing request
*** Arguments: ('task(s9om7pppcf3m4yw)', 'my prompt <lora:eris_v6:0.8>', '(low quality, worst quality:1.3), easynegative, bad-hands-5', [], 22, 'DPM++ SDE Karras', 1, 1, 7, 672, 544, False, 0.25, 2, 'R-ESRGAN 4x+ Anime6B', 0, 0, 0, 'Use same checkpoint', 'Use same sampler', '', '', [], <gradio.routes.Request object at 0x0000023D73C004F0>, 0, False, '', 0.8, 3052799025, False, -1, 0, 0, 0, False, False, {'ad_model': 'face_yolov8n.pt', 'ad_prompt': '', 'ad_negative_prompt': '', 'ad_confidence': 0.3, 'ad_mask_k_largest': 0, 'ad_mask_min_ratio': 0, 'ad_mask_max_ratio': 1, 'ad_x_offset': 0, 'ad_y_offset': 0, 'ad_dilate_erode': 4, 'ad_mask_merge_invert': 'None', 'ad_mask_blur': 4, 'ad_denoising_strength': 0.4, 'ad_inpaint_only_masked': True, 'ad_inpaint_only_masked_padding': 32, 'ad_use_inpaint_width_height': False, 'ad_inpaint_width': 512, 'ad_inpaint_height': 512, 'ad_use_steps': False, 'ad_steps': 28, 'ad_use_cfg_scale': False, 'ad_cfg_scale': 7, 'ad_use_checkpoint': False, 'ad_checkpoint': 'Use same checkpoint', 'ad_use_vae': False, 'ad_vae': 'Use same VAE', 'ad_use_sampler': False, 'ad_sampler': 'DPM++ 2M Karras', 'ad_use_noise_multiplier': False, 'ad_noise_multiplier': 1, 'ad_use_clip_skip': False, 'ad_clip_skip': 1, 'ad_restore_face': False, 'ad_controlnet_model': 'None', 'ad_controlnet_module': 'None', 'ad_controlnet_weight': 1, 'ad_controlnet_guidance_start': 0, 'ad_controlnet_guidance_end': 1, 'is_api': ()}, {'ad_model': 'None', 'ad_prompt': '', 'ad_negative_prompt': '', 'ad_confidence': 0.3, 'ad_mask_k_largest': 0, 'ad_mask_min_ratio': 0, 'ad_mask_max_ratio': 1, 'ad_x_offset': 0, 'ad_y_offset': 0, 'ad_dilate_erode': 4, 'ad_mask_merge_invert': 'None', 'ad_mask_blur': 4, 'ad_denoising_strength': 0.4, 'ad_inpaint_only_masked': True, 'ad_inpaint_only_masked_padding': 32, 'ad_use_inpaint_width_height': False, 'ad_inpaint_width': 512, 'ad_inpaint_height': 512, 'ad_use_steps': False, 'ad_steps': 28, 'ad_use_cfg_scale': False, 'ad_cfg_scale': 7, 'ad_use_checkpoint': False, 'ad_checkpoint': 'Use same checkpoint', 'ad_use_vae': False, 'ad_vae': 'Use same VAE', 'ad_use_sampler': False, 'ad_sampler': 'DPM++ 2M Karras', 'ad_use_noise_multiplier': False, 'ad_noise_multiplier': 1, 'ad_use_clip_skip': False, 'ad_clip_skip': 1, 'ad_restore_face': False, 'ad_controlnet_model': 'None', 'ad_controlnet_module': 'None', 'ad_controlnet_weight': 1, 'ad_controlnet_guidance_start': 0, 'ad_controlnet_guidance_end': 1, 'is_api': ()}, False, 'keyword prompt', 'keyword1, keyword2', 'None', 'textual inversion first', 'None', <scripts.animatediff_ui.AnimateDiffProcess object at 0x0000023D73C009D0>, <scripts.controlnet_ui.controlnet_ui_group.UiControlNetUnit object at 0x0000023D6D19CB20>, <scripts.controlnet_ui.controlnet_ui_group.UiControlNetUnit object at 0x0000023D73BAB640>, <scripts.controlnet_ui.controlnet_ui_group.UiControlNetUnit object at 0x0000023D73C03040>, 1, '', 0, '', 0, '', True, False, False, False, 0, False, False, 'positive', 'comma', 0, False, False, '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0, False, None, None, False, None, None, False, None, None, False, 50, 'linear (weight sum)', '10', 'C:\\Users\\wgius\\Desktop\\stable-diffusion-webui\\extensions\\stable-diffusion-webui-prompt-travel\\img\\ref_ctrlnet', 'Lanczos', 2, 0, 0, 'mp4', 10.0, 0, '', True, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, 'linear', 'lerp', 'token', 'random', '30', 'fixed', 1, '8', None, 'Lanczos', 2, 0, 0, 'mp4', 10.0, 0, '', True, False, False) {}
    Traceback (most recent call last):
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\modules\call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\modules\call_queue.py", line 36, in f
        res = func(*args, **kwargs)
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\modules\txt2img.py", line 55, in txt2img
        processed = processing.process_images(p)
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\modules\processing.py", line 732, in process_images
        res = process_images_inner(p)
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\extensions\sd-webui-animatediff\scripts\animatediff_cn.py", line 119, in hacked_processing_process_images_hijack
        return getattr(processing, '__controlnet_original_process_images_inner')(p, *args, **kwargs)
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\modules\processing.py", line 867, in process_images_inner
        samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts)
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\modules\processing.py", line 1140, in sample
        samples = self.sampler.sample(self, x, conditioning, unconditional_conditioning, image_conditioning=self.txt2img_image_conditioning(x))
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\modules\sd_samplers_kdiffusion.py", line 235, in sample
        samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\modules\sd_samplers_common.py", line 261, in launch_sampling
        return func()
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\modules\sd_samplers_kdiffusion.py", line 235, in <lambda>
        samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
        return func(*args, **kwargs)
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\repositories\k-diffusion\k_diffusion\sampling.py", line 573, in sample_dpmpp_sde
        denoised_2 = model(x_2, sigma_fn(s) * s_in, **extra_args)
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\extensions\sd-webui-animatediff\scripts\animatediff_infv2v.py", line 278, in mm_cfg_forward
        x_out = mm_sd_forward(self, x_in, sigma_in, cond_in, image_cond_in, make_condition_dict) # hook
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\extensions\sd-webui-animatediff\scripts\animatediff_infv2v.py", line 189, in mm_sd_forward
        x_in[_context], sigma_in[_context],
    RuntimeError: CUDA error: device-side assert triggered
    Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.


---
Traceback (most recent call last):
  File "C:\Users\wgius\Desktop\stable-diffusion-webui\venv\lib\site-packages\gradio\routes.py", line 488, in run_predict
    output = await app.get_blocks().process_api(
  File "C:\Users\wgius\Desktop\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1431, in process_api
    result = await self.call_function(
  File "C:\Users\wgius\Desktop\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1103, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "C:\Users\wgius\Desktop\stable-diffusion-webui\venv\lib\site-packages\anyio\to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "C:\Users\wgius\Desktop\stable-diffusion-webui\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "C:\Users\wgius\Desktop\stable-diffusion-webui\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "C:\Users\wgius\Desktop\stable-diffusion-webui\venv\lib\site-packages\gradio\utils.py", line 707, in wrapper
    response = f(*args, **kwargs)
  File "C:\Users\wgius\Desktop\stable-diffusion-webui\modules\call_queue.py", line 77, in f
    devices.torch_gc()
  File "C:\Users\wgius\Desktop\stable-diffusion-webui\modules\devices.py", line 51, in torch_gc
    torch.cuda.empty_cache()
  File "C:\Users\wgius\Desktop\stable-diffusion-webui\venv\lib\site-packages\torch\cuda\memory.py", line 133, in empty_cache
    torch._C._cuda_emptyCache()
RuntimeError: CUDA error: device-side assert triggered
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

Update:

How do I remove channels last?

continue-revolution · 2023-11-01T17:14:59Z

@GiusTex I don't see any _context print information on your attached terminal log. Also the terminal log does not seem like v1.10.0. You did not successfully revert back.

GiusTex · 2023-11-01T17:16:09Z

I changed it back to latest version, and about print context there where many numbers under the generation, but when I got the error they where changed.

Update:

I'll revert back to v1.10.0 and copy the console before I get the error and after it

continue-revolution · 2023-11-01T17:22:25Z

no need to worry about extremely long output. I'm also renting a 3080. I have no idea what's going on and I cannot reproduce this error in whatever way in 4090.

do not add opt-channelslast to command line arguments

GiusTex · 2023-11-01T17:31:37Z

I understood: the context is removed when I turn the text into a code, damn. I'll leave it out then.

Already up to date.
Torch version: 2.0.1+cu118
venv "C:\Users\wgius\Desktop\stable-diffusion-webui\venv\Scripts\Python.exe"
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Version: v1.6.0
Commit hash: 5ef669de080814067961f28357256e8fe27544f4
Launching Web UI with arguments: --theme=dark --api --no-half-vae --deepdanbooru --opt-sdp-attention --opt-split-attention --opt-channelslast
no module 'xformers'. Processing without...
no module 'xformers'. Processing without...
No module 'xformers'. Proceeding without it.
[-] ADetailer initialized. version: 23.11.0, num models: 9
Using sqlite file: C:\Users\wgius\Desktop\stable-diffusion-webui\extensions\sd-webui-agent-scheduler\task_scheduler.sqlite3
2023-11-01 18:20:28,833 - ControlNet - INFO - ControlNet v1.1.411
ControlNet preprocessor location: C:\Users\wgius\Desktop\stable-diffusion-webui\extensions\sd-webui-controlnet\annotator\downloads
2023-11-01 18:20:29,015 - ControlNet - INFO - ControlNet v1.1.411
[ControlNet-Travel] extension Mikubill/sd-webui-controlnet found, ControlNet-Travel loaded :)
Loading weights [f243aabb88] from C:\Users\wgius\Desktop\stable-diffusion-webui\models\Stable-diffusion\cardosAnime_v20.safetensors
2023-11-01 18:20:31,024 - AnimateDiff - INFO - Hacking i2i-batch.
Creating model from config: C:\Users\wgius\Desktop\stable-diffusion-webui\configs\v1-inference.yaml
*** Failed reading extension data from Git repository (sd-webui-animatediff)
    Traceback (most recent call last):
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\modules\extensions.py", line 81, in do_read_info_from_repo
        if repo.active_branch:
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\venv\lib\site-packages\git\repo\base.py", line 896, in active_branch
        return self.head.reference
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\venv\lib\site-packages\git\refs\symbolic.py", line 309, in _get_reference
        raise TypeError("%s is a detached symbolic reference as it points to %r" % (self, sha))
    TypeError: HEAD is a detached symbolic reference as it points to 'a13370f0fe89243d67f71519fb3e397c76d8bd80'

---
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 19.4s (prepare environment: 3.4s, import torch: 3.6s, import gradio: 1.0s, setup paths: 0.8s, initialize shared: 0.2s, other imports: 0.7s, setup codeformer: 0.1s, load scripts: 5.9s, create ui: 2.5s, gradio launch: 0.8s, app_started_callback: 0.3s).
Loading VAE weights specified in settings: C:\Users\wgius\Desktop\stable-diffusion-webui\models\VAE\orangemix.vae.pt
Applying attention optimization: sdp... done.
Model loaded in 6.6s (load weights from disk: 0.7s, create model: 0.6s, apply weights to model: 2.6s, apply channels_last: 1.1s, load VAE: 0.5s, calculate empty prompt: 0.8s).
2023-11-01 18:20:57,972 - AnimateDiff - INFO - AnimateDiff process start.
2023-11-01 18:20:57,976 - AnimateDiff - INFO - You are using tested mm mm_sd_v15_v2.safetensors.
2023-11-01 18:20:57,976 - AnimateDiff - INFO - Loading motion module mm_sd_v15_v2.safetensors from C:\Users\wgius\Desktop\stable-diffusion-webui\extensions\sd-webui-animatediff\model\mm_sd_v15_v2.safetensors
2023-11-01 18:21:00,508 - AnimateDiff - WARNING - Missing keys <All keys matched successfully>
2023-11-01 18:21:01,277 - AnimateDiff - INFO - Injecting motion module mm_sd_v15_v2.safetensors into SD1.5 UNet middle block.
2023-11-01 18:21:01,277 - AnimateDiff - INFO - Injecting motion module mm_sd_v15_v2.safetensors into SD1.5 UNet input blocks.
2023-11-01 18:21:01,277 - AnimateDiff - INFO - Injecting motion module mm_sd_v15_v2.safetensors into SD1.5 UNet output blocks.
2023-11-01 18:21:01,277 - AnimateDiff - INFO - Setting DDIM alpha.
2023-11-01 18:21:01,281 - AnimateDiff - INFO - Injection finished.
2023-11-01 18:21:01,281 - AnimateDiff - INFO - Hacking lora to support motion lora
2023-11-01 18:21:01,281 - AnimateDiff - INFO - Hacking CFGDenoiser forward function.
2023-11-01 18:21:01,281 - AnimateDiff - INFO - Hacking ControlNet.

0%| | 0/22 [00:00<?, ?it/s][0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]s]
5%|███▊ | 1/22 [00:15<05:34, 15.95s/it][0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]t]
9%|███████▌ | 2/22 [00:30<05:00, 15.01s/it][0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]t]
14%|███████████▎ | 3/22 [00:45<04:44, 14.97s/it][0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]t]
18%|███████████████ | 4/22 [00:59<04:27, 14.85s/it][0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]t]
23%|██████████████████▊ | 5/22 [01:14<04:10, 14.72s/it][0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]t]
27%|██████████████████████▋ | 6/22 [01:29<03:55, 14.70s/it][0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]t]
32%|██████████████████████████▍ | 7/22 [01:43<03:39, 14.64s/it][0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]t]
36%|██████████████████████████████▏ | 8/22 [01:58<03:24, 14.61s/it][0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]t]

C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [717,0,0], thread: [32,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [717,0,0], thread: [33,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [717,0,0], thread: [34,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
...
...
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [717,0,0], thread: [62,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [717,0,0], thread: [63,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [792,0,0], thread: [0,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [792,0,0], thread: [1,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
...
...
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [793,0,0], thread: [94,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [793,0,0], thread: [95,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
Exception in thread MemMon:
Traceback (most recent call last):
  File "C:\Users\wgius\AppData\Local\Programs\Python\Python310\lib\threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "C:\Users\wgius\Desktop\stable-diffusion-webui\modules\memmon.py", line 53, in run
    free, total = self.cuda_mem_get_info()
  File "C:\Users\wgius\Desktop\stable-diffusion-webui\modules\memmon.py", line 34, in cuda_mem_get_info
    return torch.cuda.mem_get_info(index)
  File "C:\Users\wgius\Desktop\stable-diffusion-webui\venv\lib\site-packages\torch\cuda\memory.py", line 618, in mem_get_info
    return torch.cuda.cudart().cudaMemGetInfo(device)
RuntimeError: CUDA error: device-side assert triggered
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [805,0,0], thread: [64,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [805,0,0], thread: [65,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
...
...
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [805,0,0], thread: [94,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [805,0,0], thread: [95,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
 36%|██████████████████████████████▏                                                    | 8/22 [02:06<03:40, 15.76s/it]
*** Error completing request
*** Arguments: ('task(u1pov667v0gqz53)', 'my prompt <lora:eris_v6:0.8>', '(low quality, worst quality:1.3), easynegative, bad-hands-5', [], 22, 'DPM++ SDE Karras', 1, 1, 7, 672, 544, False, 0.25, 2, 'R-ESRGAN 4x+ Anime6B', 0, 0, 0, 'Use same checkpoint', 'Use same sampler', '', '', [], <gradio.routes.Request object at 0x0000017473BD6C50>, 0, False, '', 0.8, 3052799025, False, -1, 0, 0, 0, False, False, {'ad_model': 'face_yolov8n.pt', 'ad_prompt': '', 'ad_negative_prompt': '', 'ad_confidence': 0.3, 'ad_mask_k_largest': 0, 'ad_mask_min_ratio': 0, 'ad_mask_max_ratio': 1, 'ad_x_offset': 0, 'ad_y_offset': 0, 'ad_dilate_erode': 4, 'ad_mask_merge_invert': 'None', 'ad_mask_blur': 4, 'ad_denoising_strength': 0.4, 'ad_inpaint_only_masked': True, 'ad_inpaint_only_masked_padding': 32, 'ad_use_inpaint_width_height': False, 'ad_inpaint_width': 512, 'ad_inpaint_height': 512, 'ad_use_steps': False, 'ad_steps': 28, 'ad_use_cfg_scale': False, 'ad_cfg_scale': 7, 'ad_use_checkpoint': False, 'ad_checkpoint': 'Use same checkpoint', 'ad_use_vae': False, 'ad_vae': 'Use same VAE', 'ad_use_sampler': False, 'ad_sampler': 'DPM++ 2M Karras', 'ad_use_noise_multiplier': False, 'ad_noise_multiplier': 1, 'ad_use_clip_skip': False, 'ad_clip_skip': 1, 'ad_restore_face': False, 'ad_controlnet_model': 'None', 'ad_controlnet_module': 'None', 'ad_controlnet_weight': 1, 'ad_controlnet_guidance_start': 0, 'ad_controlnet_guidance_end': 1, 'is_api': ()}, {'ad_model': 'None', 'ad_prompt': '', 'ad_negative_prompt': '', 'ad_confidence': 0.3, 'ad_mask_k_largest': 0, 'ad_mask_min_ratio': 0, 'ad_mask_max_ratio': 1, 'ad_x_offset': 0, 'ad_y_offset': 0, 'ad_dilate_erode': 4, 'ad_mask_merge_invert': 'None', 'ad_mask_blur': 4, 'ad_denoising_strength': 0.4, 'ad_inpaint_only_masked': True, 'ad_inpaint_only_masked_padding': 32, 'ad_use_inpaint_width_height': False, 'ad_inpaint_width': 512, 'ad_inpaint_height': 512, 'ad_use_steps': False, 'ad_steps': 28, 'ad_use_cfg_scale': False, 'ad_cfg_scale': 7, 'ad_use_checkpoint': False, 'ad_checkpoint': 'Use same checkpoint', 'ad_use_vae': False, 'ad_vae': 'Use same VAE', 'ad_use_sampler': False, 'ad_sampler': 'DPM++ 2M Karras', 'ad_use_noise_multiplier': False, 'ad_noise_multiplier': 1, 'ad_use_clip_skip': False, 'ad_clip_skip': 1, 'ad_restore_face': False, 'ad_controlnet_model': 'None', 'ad_controlnet_module': 'None', 'ad_controlnet_weight': 1, 'ad_controlnet_guidance_start': 0, 'ad_controlnet_guidance_end': 1, 'is_api': ()}, False, 'keyword prompt', 'keyword1, keyword2', 'None', 'textual inversion first', 'None', <scripts.animatediff_ui.AnimateDiffProcess object at 0x0000017473B630D0>, <scripts.controlnet_ui.controlnet_ui_group.UiControlNetUnit object at 0x0000017473FE5EA0>, <scripts.controlnet_ui.controlnet_ui_group.UiControlNetUnit object at 0x0000017473FE66E0>, <scripts.controlnet_ui.controlnet_ui_group.UiControlNetUnit object at 0x0000017473BD5090>, 1, '', 0, '', 0, '', True, False, False, False, 0, False, False, 'positive', 'comma', 0, False, False, '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0, False, None, None, False, None, None, False, None, None, False, 50, 'linear (weight sum)', '10', 'C:\\Users\\wgius\\Desktop\\stable-diffusion-webui\\extensions\\stable-diffusion-webui-prompt-travel\\img\\ref_ctrlnet', 'Lanczos', 2, 0, 0, 'mp4', 10.0, 0, '', True, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, 'linear', 'lerp', 'token', 'random', '30', 'fixed', 1, '8', None, 'Lanczos', 2, 0, 0, 'mp4', 10.0, 0, '', True, False, False) {}
    Traceback (most recent call last):
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\modules\call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\modules\call_queue.py", line 36, in f
        res = func(*args, **kwargs)
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\modules\txt2img.py", line 55, in txt2img
        processed = processing.process_images(p)
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\modules\processing.py", line 732, in process_images
        res = process_images_inner(p)
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\extensions\sd-webui-animatediff\scripts\animatediff_cn.py", line 119, in hacked_processing_process_images_hijack
        return getattr(processing, '__controlnet_original_process_images_inner')(p, *args, **kwargs)
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\modules\processing.py", line 867, in process_images_inner
        samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts)
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\modules\processing.py", line 1140, in sample
        samples = self.sampler.sample(self, x, conditioning, unconditional_conditioning, image_conditioning=self.txt2img_image_conditioning(x))
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\modules\sd_samplers_kdiffusion.py", line 235, in sample
        samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\modules\sd_samplers_common.py", line 261, in launch_sampling
        return func()
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\modules\sd_samplers_kdiffusion.py", line 235, in <lambda>
        samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
        return func(*args, **kwargs)
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\repositories\k-diffusion\k_diffusion\sampling.py", line 573, in sample_dpmpp_sde
        denoised_2 = model(x_2, sigma_fn(s) * s_in, **extra_args)
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\extensions\sd-webui-animatediff\scripts\animatediff_infv2v.py", line 275, in mm_cfg_forward
        x_out = mm_sd_forward(self, x_in, sigma_in, cond_in, image_cond_in, make_condition_dict) # hook
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\extensions\sd-webui-animatediff\scripts\animatediff_infv2v.py", line 189, in mm_sd_forward
        out = self.inner_model(x_in[_context], sigma_in[_context], cond=make_condition_dict(cond_in[_context], image_cond_in[_context]))
    RuntimeError: CUDA error: device-side assert triggered
    Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.


---
Traceback (most recent call last):
  File "C:\Users\wgius\Desktop\stable-diffusion-webui\venv\lib\site-packages\gradio\routes.py", line 488, in run_predict
    output = await app.get_blocks().process_api(
  File "C:\Users\wgius\Desktop\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1431, in process_api
    result = await self.call_function(
  File "C:\Users\wgius\Desktop\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1103, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "C:\Users\wgius\Desktop\stable-diffusion-webui\venv\lib\site-packages\anyio\to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "C:\Users\wgius\Desktop\stable-diffusion-webui\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "C:\Users\wgius\Desktop\stable-diffusion-webui\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "C:\Users\wgius\Desktop\stable-diffusion-webui\venv\lib\site-packages\gradio\utils.py", line 707, in wrapper
    response = f(*args, **kwargs)
  File "C:\Users\wgius\Desktop\stable-diffusion-webui\modules\call_queue.py", line 77, in f
    devices.torch_gc()
  File "C:\Users\wgius\Desktop\stable-diffusion-webui\modules\devices.py", line 51, in torch_gc
    torch.cuda.empty_cache()
  File "C:\Users\wgius\Desktop\stable-diffusion-webui\venv\lib\site-packages\torch\cuda\memory.py", line 133, in empty_cache
    torch._C._cuda_emptyCache()
RuntimeError: CUDA error: device-side assert triggered
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

I'm testing without channelslast as you said.
Same error.

continue-revolution · 2023-11-01T17:40:07Z

I see. Seems like not an update problem. I'm still investigating. I have no idea what's going on.

continue-revolution · 2023-11-01T17:41:37Z

Why do people start to pop up this issue so late? I don't understand. Are all people here new users?

GiusTex · 2023-11-01T17:47:00Z

User-bat:

python: 3.10.6 • torch: 2.0.1+cu118 • opt-sdp-attention
COMMANDLINE_ARGS=--theme=dark --api --no-half-vae --deepdanbooru --opt-sdp-attention

set PYTORCH_CUDA_ALLOC_CONF=garbage_collection_threshold:0.6,max_split_size_mb:128
set CUDA_LAUNCH_BLOCKING=1
set TORCH_USE_CUDA_DSA=1

Webui options:

"When you have --xformers in your command line args, you want AnimateDiff to:
Optimize attention layers with sdp (torch >= 2.0.0 required)"
AnimateDiff v.1.10.0

In the message above there is the error log with channelslast, down below without them:

Already up to date.
Torch version: 2.0.1+cu118
venv "C:\Users\wgius\Desktop\stable-diffusion-webui\venv\Scripts\Python.exe"
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Version: v1.6.0
Commit hash: 5ef669de080814067961f28357256e8fe27544f4
Launching Web UI with arguments: --theme=dark --api --no-half-vae --deepdanbooru --opt-sdp-attention --opt-split-attention
no module 'xformers'. Processing without...
no module 'xformers'. Processing without...
No module 'xformers'. Proceeding without it.
[-] ADetailer initialized. version: 23.11.0, num models: 9
Using sqlite file: C:\Users\wgius\Desktop\stable-diffusion-webui\extensions\sd-webui-agent-scheduler\task_scheduler.sqlite3
2023-11-01 18:32:38,054 - ControlNet - INFO - ControlNet v1.1.411
ControlNet preprocessor location: C:\Users\wgius\Desktop\stable-diffusion-webui\extensions\sd-webui-controlnet\annotator\downloads
2023-11-01 18:32:38,179 - ControlNet - INFO - ControlNet v1.1.411
[ControlNet-Travel] extension Mikubill/sd-webui-controlnet found, ControlNet-Travel loaded :)
Loading weights [f243aabb88] from C:\Users\wgius\Desktop\stable-diffusion-webui\models\Stable-diffusion\cardosAnime_v20.safetensors
2023-11-01 18:32:40,142 - AnimateDiff - INFO - Hacking i2i-batch.
Creating model from config: C:\Users\wgius\Desktop\stable-diffusion-webui\configs\v1-inference.yaml
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 17.9s (prepare environment: 3.4s, import torch: 3.6s, import gradio: 0.9s, setup paths: 0.8s, initialize shared: 0.2s, other imports: 0.7s, setup codeformer: 0.1s, load scripts: 5.0s, create ui: 2.5s, gradio launch: 0.4s, app_started_callback: 0.3s).
Loading VAE weights specified in settings: C:\Users\wgius\Desktop\stable-diffusion-webui\models\VAE\orangemix.vae.pt
Applying attention optimization: sdp... done.
Model loaded in 6.3s (load weights from disk: 0.7s, create model: 0.6s, apply weights to model: 2.3s, apply dtype to VAE: 1.0s, load VAE: 0.6s, calculate empty prompt: 0.9s).
2023-11-01 18:32:54,881 - AnimateDiff - INFO - AnimateDiff process start.
2023-11-01 18:32:54,883 - AnimateDiff - INFO - You are using tested mm mm_sd_v15_v2.safetensors.
2023-11-01 18:32:54,883 - AnimateDiff - INFO - Loading motion module mm_sd_v15_v2.safetensors from C:\Users\wgius\Desktop\stable-diffusion-webui\extensions\sd-webui-animatediff\model\mm_sd_v15_v2.safetensors
2023-11-01 18:32:57,583 - AnimateDiff - WARNING - Missing keys <All keys matched successfully>
2023-11-01 18:32:58,430 - AnimateDiff - INFO - Injecting motion module mm_sd_v15_v2.safetensors into SD1.5 UNet middle block.
2023-11-01 18:32:58,431 - AnimateDiff - INFO - Injecting motion module mm_sd_v15_v2.safetensors into SD1.5 UNet input blocks.
2023-11-01 18:32:58,431 - AnimateDiff - INFO - Injecting motion module mm_sd_v15_v2.safetensors into SD1.5 UNet output blocks.
2023-11-01 18:32:58,431 - AnimateDiff - INFO - Setting DDIM alpha.
2023-11-01 18:32:58,434 - AnimateDiff - INFO - Injection finished.
2023-11-01 18:32:58,434 - AnimateDiff - INFO - Hacking lora to support motion lora
2023-11-01 18:32:58,435 - AnimateDiff - INFO - Hacking CFGDenoiser forward function.
2023-11-01 18:32:58,435 - AnimateDiff - INFO - Hacking ControlNet.

0%| | 0/22 [00:00<?, ?it/s][0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]s]
5%|███▊ | 1/22 [00:27<09:28, 27.09s/it][0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]t]
9%|███████▌ | 2/22 [00:54<09:02, 27.15s/it][0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]t]
14%|███████████▎ | 3/22 [01:20<08:31, 26.92s/it][0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]t]
18%|███████████████ | 4/22 [01:48<08:07, 27.09s/it][0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]t]
23%|██████████████████▊ | 5/22 [02:14<07:34, 26.74s/it][0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]t]
27%|██████████████████████▋ | 6/22 [02:41<07:09, 26.83s/it][0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]t]
32%|██████████████████████████▍ | 7/22 [03:07<06:39, 26.64s/it][0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]t]
36%|██████████████████████████████▏ | 8/22 [03:34<06:11, 26.56s/it][0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]t]

C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [769,0,0], thread: [0,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [769,0,0], thread: [1,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
...
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [769,0,0], thread: [31,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
 36%|██████████████████████████████▏                                                    | 8/22 [03:47<06:38, 28.45s/it]
Exception in thread MemMon:
Traceback (most recent call last):
  File "C:\Users\wgius\AppData\Local\Programs\Python\Python310\lib\threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "C:\Users\wgius\Desktop\stable-diffusion-webui\modules\memmon.py", line 53, in run
    free, total = self.cuda_mem_get_info()
  File "C:\Users\wgius\Desktop\stable-diffusion-webui\modules\memmon.py", line 34, in cuda_mem_get_info
    return torch.cuda.mem_get_info(index)
  File "C:\Users\wgius\Desktop\stable-diffusion-webui\venv\lib\site-packages\torch\cuda\memory.py", line 618, in mem_get_info
    return torch.cuda.cudart().cudaMemGetInfo(device)
RuntimeError: CUDA error: device-side assert triggered
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

*** Error completing request
*** Arguments: ('task(9r0f3s4bdig489q)', 'my prompt <lora:eris_v6:0.8>', '(low quality, worst quality:1.3), easynegative, bad-hands-5', [], 22, 'DPM++ SDE Karras', 1, 1, 7, 672, 544, False, 0.25, 2, 'R-ESRGAN 4x+ Anime6B', 0, 0, 0, 'Use same checkpoint', 'Use same sampler', '', '', [], <gradio.routes.Request object at 0x000002E0F6A3FB50>, 0, False, '', 0.8, 3052799025, False, -1, 0, 0, 0, False, False, {'ad_model': 'face_yolov8n.pt', 'ad_prompt': '', 'ad_negative_prompt': '', 'ad_confidence': 0.3, 'ad_mask_k_largest': 0, 'ad_mask_min_ratio': 0, 'ad_mask_max_ratio': 1, 'ad_x_offset': 0, 'ad_y_offset': 0, 'ad_dilate_erode': 4, 'ad_mask_merge_invert': 'None', 'ad_mask_blur': 4, 'ad_denoising_strength': 0.4, 'ad_inpaint_only_masked': True, 'ad_inpaint_only_masked_padding': 32, 'ad_use_inpaint_width_height': False, 'ad_inpaint_width': 512, 'ad_inpaint_height': 512, 'ad_use_steps': False, 'ad_steps': 28, 'ad_use_cfg_scale': False, 'ad_cfg_scale': 7, 'ad_use_checkpoint': False, 'ad_checkpoint': 'Use same checkpoint', 'ad_use_vae': False, 'ad_vae': 'Use same VAE', 'ad_use_sampler': False, 'ad_sampler': 'DPM++ 2M Karras', 'ad_use_noise_multiplier': False, 'ad_noise_multiplier': 1, 'ad_use_clip_skip': False, 'ad_clip_skip': 1, 'ad_restore_face': False, 'ad_controlnet_model': 'None', 'ad_controlnet_module': 'None', 'ad_controlnet_weight': 1, 'ad_controlnet_guidance_start': 0, 'ad_controlnet_guidance_end': 1, 'is_api': ()}, {'ad_model': 'None', 'ad_prompt': '', 'ad_negative_prompt': '', 'ad_confidence': 0.3, 'ad_mask_k_largest': 0, 'ad_mask_min_ratio': 0, 'ad_mask_max_ratio': 1, 'ad_x_offset': 0, 'ad_y_offset': 0, 'ad_dilate_erode': 4, 'ad_mask_merge_invert': 'None', 'ad_mask_blur': 4, 'ad_denoising_strength': 0.4, 'ad_inpaint_only_masked': True, 'ad_inpaint_only_masked_padding': 32, 'ad_use_inpaint_width_height': False, 'ad_inpaint_width': 512, 'ad_inpaint_height': 512, 'ad_use_steps': False, 'ad_steps': 28, 'ad_use_cfg_scale': False, 'ad_cfg_scale': 7, 'ad_use_checkpoint': False, 'ad_checkpoint': 'Use same checkpoint', 'ad_use_vae': False, 'ad_vae': 'Use same VAE', 'ad_use_sampler': False, 'ad_sampler': 'DPM++ 2M Karras', 'ad_use_noise_multiplier': False, 'ad_noise_multiplier': 1, 'ad_use_clip_skip': False, 'ad_clip_skip': 1, 'ad_restore_face': False, 'ad_controlnet_model': 'None', 'ad_controlnet_module': 'None', 'ad_controlnet_weight': 1, 'ad_controlnet_guidance_start': 0, 'ad_controlnet_guidance_end': 1, 'is_api': ()}, False, 'keyword prompt', 'keyword1, keyword2', 'None', 'textual inversion first', 'None', <scripts.animatediff_ui.AnimateDiffProcess object at 0x000002E0F71DB490>, <scripts.controlnet_ui.controlnet_ui_group.UiControlNetUnit object at 0x000002E0F71D9000>, <scripts.controlnet_ui.controlnet_ui_group.UiControlNetUnit object at 0x000002E0F71DAE60>, <scripts.controlnet_ui.controlnet_ui_group.UiControlNetUnit object at 0x000002E0ED84C400>, 1, '', 0, '', 0, '', True, False, False, False, 0, False, False, 'positive', 'comma', 0, False, False, '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0, False, None, None, False, None, None, False, None, None, False, 50, 'linear (weight sum)', '10', 'C:\\Users\\wgius\\Desktop\\stable-diffusion-webui\\extensions\\stable-diffusion-webui-prompt-travel\\img\\ref_ctrlnet', 'Lanczos', 2, 0, 0, 'mp4', 10.0, 0, '', True, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, 'linear', 'lerp', 'token', 'random', '30', 'fixed', 1, '8', None, 'Lanczos', 2, 0, 0, 'mp4', 10.0, 0, '', True, False, False) {}
    Traceback (most recent call last):
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\modules\call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\modules\call_queue.py", line 36, in f
        res = func(*args, **kwargs)
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\modules\txt2img.py", line 55, in txt2img
        processed = processing.process_images(p)
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\modules\processing.py", line 732, in process_images
        res = process_images_inner(p)
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\extensions\sd-webui-animatediff\scripts\animatediff_cn.py", line 119, in hacked_processing_process_images_hijack
        return getattr(processing, '__controlnet_original_process_images_inner')(p, *args, **kwargs)
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\modules\processing.py", line 867, in process_images_inner
        samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts)
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\modules\processing.py", line 1140, in sample
        samples = self.sampler.sample(self, x, conditioning, unconditional_conditioning, image_conditioning=self.txt2img_image_conditioning(x))
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\modules\sd_samplers_kdiffusion.py", line 235, in sample
        samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\modules\sd_samplers_common.py", line 261, in launch_sampling
        return func()
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\modules\sd_samplers_kdiffusion.py", line 235, in <lambda>
        samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
        return func(*args, **kwargs)
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\repositories\k-diffusion\k_diffusion\sampling.py", line 573, in sample_dpmpp_sde
        denoised_2 = model(x_2, sigma_fn(s) * s_in, **extra_args)
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\extensions\sd-webui-animatediff\scripts\animatediff_infv2v.py", line 275, in mm_cfg_forward
        x_out = mm_sd_forward(self, x_in, sigma_in, cond_in, image_cond_in, make_condition_dict) # hook
      File "C:\Users\wgius\Desktop\stable-diffusion-webui\extensions\sd-webui-animatediff\scripts\animatediff_infv2v.py", line 189, in mm_sd_forward
        out = self.inner_model(x_in[_context], sigma_in[_context], cond=make_condition_dict(cond_in[_context], image_cond_in[_context]))
    RuntimeError: CUDA error: device-side assert triggered
    Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.


---
Traceback (most recent call last):
  File "C:\Users\wgius\Desktop\stable-diffusion-webui\venv\lib\site-packages\gradio\routes.py", line 488, in run_predict
    output = await app.get_blocks().process_api(
  File "C:\Users\wgius\Desktop\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1431, in process_api
    result = await self.call_function(
  File "C:\Users\wgius\Desktop\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1103, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "C:\Users\wgius\Desktop\stable-diffusion-webui\venv\lib\site-packages\anyio\to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "C:\Users\wgius\Desktop\stable-diffusion-webui\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "C:\Users\wgius\Desktop\stable-diffusion-webui\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "C:\Users\wgius\Desktop\stable-diffusion-webui\venv\lib\site-packages\gradio\utils.py", line 707, in wrapper
    response = f(*args, **kwargs)
  File "C:\Users\wgius\Desktop\stable-diffusion-webui\modules\call_queue.py", line 77, in f
    devices.torch_gc()
  File "C:\Users\wgius\Desktop\stable-diffusion-webui\modules\devices.py", line 51, in torch_gc
    torch.cuda.empty_cache()
  File "C:\Users\wgius\Desktop\stable-diffusion-webui\venv\lib\site-packages\torch\cuda\memory.py", line 133, in empty_cache
    torch._C._cuda_emptyCache()
RuntimeError: CUDA error: device-side assert triggered
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

And yes I'm a new user... sorry. I searched everywhere yesterday and before asking here though.

continue-revolution · 2023-11-01T17:49:19Z

I understand your situation, but I don't understand the problem. Wait patiently while I'm addressing it.

GiusTex · 2023-11-01T17:55:24Z

I posted the new log just for completeness, in case something changed before and after the channeslast. Anyway, what torch version are you using? This comment here:

change torch version 2.0.0+cu118 to 2.1.0.dev20230501+cu117 works for me, but I have no idea what the reason is. pip3 install numpy --pre torch torchvision torchaudio --force-reinstall --index-url https://download.pytorch.org/whl/nightly/cu117

is making me crazy because I can't replicate it with cu117 (only with cu118 that doesn't work).
And there is also the free cache thing, that sometimes it worked, maybe it was with a specific torch version..

continue-revolution · 2023-11-01T17:56:16Z

@GiusTex Do you mean that it works for cu117? I'm using cu117, so maybe it's really a problem of pytorch.

GiusTex · 2023-11-01T17:58:29Z

Yeah for him it worked, but I can't find a way to download dependencies with cu117

GiusTex · 2023-11-01T17:59:35Z

If I use /cu117 it says it can't find a torchvision version from none, and online I can't find what torchvision version to use with cu117.
I think I tried also this: pip install pytorch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 pytorch-cuda=11.7 -c pytorch -c nvidia from here and it didn't work, I'll try again after this torch version finishes downloading

continue-revolution · 2023-11-01T18:01:17Z

https://download.pytorch.org/whl/nightly/cu117

ask pytorch website and/or gpt what you should do. Maybe you can find a way to re-install torchvision from this link. activate your venv before doing that.

continue-revolution · 2023-11-01T18:01:47Z

This is an extremely unexpecting problem, and I think it's most likely a problem of pytorch.

GiusTex · 2023-11-01T18:02:32Z

I think it's most likely a problem of pytorch

I think so too.

That or the cuda version. I'm trying 2.0.1+cu117: (venv) C:\Desktop\stable-diffusion-webui>conda install pytorch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 pytorch-cuda=11.7 --force-reinstall -c pytorch -c nvidia -y

Update:

Verifying transaction: failed

RemoveError: 'requests' is a dependency of conda and cannot be removed from
conda's operating environment.
RemoveError: 'tqdm' is a dependency of conda and cannot be removed from
conda's operating environment.

Damn, I have to use pip (same command with pip didn't work, obviously...)

Update:

2.0.1+cu117 install:
Ok let's try this: pip install torch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2

continue-revolution · 2023-11-01T18:20:10Z

version: v1.6.0 • python: 3.11.4 • torch: 2.1.0+cu121 • xformers: N/A • gradio: 3.41.2 • checkpoint: a1535d0a42

This setting on 3080 does not produce any problem, even with prompt travel. Please DO NOT USE torch 2.0.1+cu118.

A: 5.32 GB, R: 6.25 GB, Sys: 6.5/9.77539 GB (66.9%)

Evilander · 2023-11-01T23:47:45Z

Damn. Forgot my charger at work, so I can't test this yet, but after looking at the responses - I'll just say I'm confident after setting those parameters to zero, I'll be in business.

3dcinetv · 2023-11-02T03:50:59Z

What solves this for me (after having reinstalled the webui from scratch which solved the issue only to have it resurface after changing some of the Optimization settings) was to set all of the following settings to 0:
Negative Guidance minimum sigma 
Token merging ratio
Token merging ratio for img2img 
Token merging ratio for high-res pass 
I booted up the old install and set those to zero and then the issue went away there as well.

This does not solve the issue of SEED drifting (AnimateDiff uses a different seed when generating images, not the once provided by the user). ISSUE NOT SOLVED.

continue-revolution · 2023-11-02T03:56:54Z

@3dcinetv your issue is a different one. Not related to this thread.

LIQUIDMIND111 · 2023-11-02T13:25:10Z

What solves this for me (after having reinstalled the webui from scratch which solved the issue only to have it resurface after changing some of the Optimization settings) was to set all of the following settings to 0:
Negative Guidance minimum sigma 
Token merging ratio
Token merging ratio for img2img 
Token merging ratio for high-res pass 
I booted up the old install and set those to zero and then the issue went away there as well.

Where do i find these settings or section to change to 0?

FreeeFry · 2023-11-02T14:02:07Z

What solves this for me (after having reinstalled the webui from scratch which solved the issue only to have it resurface after changing some of the Optimization settings) was to set all of the following settings to 0:
Negative Guidance minimum sigma 
Token merging ratio
Token merging ratio for img2img 
Token merging ratio for high-res pass 
I booted up the old install and set those to zero and then the issue went away there as well.
Where do i find these settings or section to change to 0?

It's in the Stable Diffusion Webui under the Settings tab -> Optimizations.
@LIQUIDMIND111

LIQUIDMIND111 · 2023-11-02T14:05:24Z

What solves this for me (after having reinstalled the webui from scratch which solved the issue only to have it resurface after changing some of the Optimization settings) was to set all of the following settings to 0:
Negative Guidance minimum sigma 
Token merging ratio
Token merging ratio for img2img 
Token merging ratio for high-res pass 
I booted up the old install and set those to zero and then the issue went away there as well.
Where do i find these settings or section to change to 0?
It's in the Stable Diffusion Webui under the Settings tab -> Optimizations. @LIQUIDMIND111

thanks mate

edwinzeng2005 · 2023-11-03T06:05:52Z

I checked my settings for optimisations, but those four sliders were already at zero.
I am still getting these errors.

RuntimeError: CUDA error: invalid configuration argument CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.

pipa0979 · 2023-11-03T17:21:58Z

The problem persists where the image generator stops functioning midway. Additionally, the specified width and height parameters are not adhered to, and revert to 512. This causes a discrepancy in the appearance of the GIF when compared to the picture generated with a resolution of 512 x 768; it defaults to a resolution of 512 x 512 instead.

pipa0979 · 2023-11-03T17:51:41Z

Deleted venv
Latest driver for nvidia
4090
Same issue. Crashes at 50% ish

Already up to date.
venv "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\venv\Scripts\Python.exe"
Python 3.10.11 (tags/v3.10.11:7d4cc5a, Apr  5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)]
Version: v1.6.0-2-g4afaaf8a
Commit hash: 4afaaf8a020c1df457bcf7250cb1c7f609699fa7
Launching Web UI with arguments: --xformers --gradio-img2img-tool color-sketch
[-] ADetailer initialized. version: 23.11.0, num models: 9
2023-11-03 10:49:08,294 - ControlNet - INFO - ControlNet v1.1.415
ControlNet preprocessor location: C:\Users\User\stable-diffusion-2\stable-diffusion-webui\extensions\sd-webui-controlnet\annotator\downloads
2023-11-03 10:49:08,456 - ControlNet - INFO - ControlNet v1.1.415
[ControlNet-Travel] extension Mikubill/sd-webui-controlnet found, ControlNet-Travel loaded :)
Loading weights [b2851ad0cc] from C:\Users\User\stable-diffusion-2\stable-diffusion-webui\models\Stable-diffusion\qwdhvx.ckpt
2023-11-03 10:49:08,943 - AnimateDiff - INFO - Hacking i2i-batch.
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 11.1s (prepare environment: 2.9s, import torch: 2.4s, import gradio: 1.1s, setup paths: 1.2s, initialize shared: 0.1s, other imports: 0.9s, load scripts: 1.8s, create ui: 0.4s, gradio launch: 0.2s).
Creating model from config: C:\Users\User\stable-diffusion-2\stable-diffusion-webui\configs\v1-inference.yaml
Loading VAE weights specified in settings: C:\Users\User\stable-diffusion-2\stable-diffusion-webui\models\VAE\vae-ft-mse-840000-ema-pruned.ckpt
Applying attention optimization: xformers... done.
Model loaded in 2.0s (load weights from disk: 0.8s, create model: 0.6s, apply weights to model: 0.3s, load VAE: 0.1s).

To create a public link, set `share=True` in `launch()`.
Startup time: 11.1s (prepare environment: 2.9s, import torch: 2.4s, import gradio: 1.1s, setup paths: 1.2s, initialize shared: 0.1s, other imports: 0.9s, load scripts: 1.8s, create ui: 0.4s, gradio launch: 0.2s).
Creating model from config: C:\Users\User\stable-diffusion-2\stable-diffusion-webui\configs\v1-inference.yaml
Loading VAE weights specified in settings: C:\Users\User\stable-diffusion-2\stable-diffusion-webui\models\VAE\vae-ft-mse-840000-ema-pruned.ckpt
Applying attention optimization: xformers... done.
Model loaded in 2.0s (load weights from disk: 0.8s, create model: 0.6s, apply weights to model: 0.3s, load VAE: 0.1s).
2023-11-03 10:50:17,392 - AnimateDiff - INFO - AnimateDiff process start.
2023-11-03 10:50:17,392 - AnimateDiff - INFO - Loading motion module mm_sd_v15_v2.ckpt from C:\Users\User\stable-diffusion-2\stable-diffusion-webui\extensions\sd-webui-animatediff\model\mm_sd_v15_v2.ckpt
2023-11-03 10:50:17,674 - AnimateDiff - INFO - Guessed mm_sd_v15_v2.ckpt architecture: MotionModuleType.AnimateDiffV2
2023-11-03 10:50:19,133 - AnimateDiff - WARNING - Missing keys <All keys matched successfully>
2023-11-03 10:50:19,662 - AnimateDiff - INFO - Injecting motion module mm_sd_v15_v2.ckpt into SD1.5 UNet middle block.
2023-11-03 10:50:19,662 - AnimateDiff - INFO - Injecting motion module mm_sd_v15_v2.ckpt into SD1.5 UNet input blocks.
2023-11-03 10:50:19,662 - AnimateDiff - INFO - Injecting motion module mm_sd_v15_v2.ckpt into SD1.5 UNet output blocks.
2023-11-03 10:50:19,663 - AnimateDiff - INFO - Setting DDIM alpha.
2023-11-03 10:50:19,666 - AnimateDiff - INFO - Injection finished.
2023-11-03 10:50:19,666 - AnimateDiff - INFO - Hacking lora to support motion lora
2023-11-03 10:50:19,666 - AnimateDiff - INFO - Hacking CFGDenoiser forward function.
2023-11-03 10:50:19,666 - AnimateDiff - INFO - Hacking ControlNet.
100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:53<00:00,  2.68s/it]
  0%|                                                                                           | 0/20 [00:00<?, ?it/s]
*** Error completing request
*** Arguments: ('task(3ydivz7zim4raw5)', 'portrait of qwdhvx woman, blazer, sophisticated, LinkedIn Image, futuristic, 8k uhd, high quality, film grain, Fujifilm XT3', '(deformed iris, deformed pupils, semi-realistic, cgi, 3d, render, sketch, cartoon, drawing, anime, mutated hands and fingers:1.4), (deformed, distorted, disfigured:1.3), poorly drawn, bad anatomy, wrong anatomy, extra limb, missing limb, floating limbs, disconnected limbs, mutation, mutated, ugly, disgusting, amputation, UnrealisticDream, bush, plant, tree, colorful, crowd, pedestrian, saturated, bright, vivid, color, orange', [], 20, 'DPM++ SDE Karras', 1, 1, 7, 768, 512, True, 0.35, 2, 'None', 0, 0, 0, 'Use same checkpoint', 'Use same sampler', '', '', [], <gradio.routes.Request object at 0x00000220A45CACB0>, 0, False, '', 0.8, 834590493, False, -1, 0, 0, 0, False, False, {'ad_model': 'face_yolov8n.pt', 'ad_prompt': 'qwdhvx woman, 8k uhd, high quality, film grain, Fujifilm XT3', 'ad_negative_prompt': '(deformed iris, deformed pupils, semi-realistic, cgi, 3d, render, sketch, cartoon, drawing, anime, mutated hands and fingers:1.4), (deformed, distorted, disfigured:1.3), poorly drawn, bad anatomy, wrong anatomy, extra limb, missing limb, floating limbs, disconnected limbs, mutation, mutated, ugly, disgusting, amputation, UnrealisticDream', 'ad_confidence': 0.3, 'ad_mask_k_largest': 0, 'ad_mask_min_ratio': 0, 'ad_mask_max_ratio': 1, 'ad_x_offset': 0, 'ad_y_offset': 0, 'ad_dilate_erode': 4, 'ad_mask_merge_invert': 'None', 'ad_mask_blur': 4, 'ad_denoising_strength': 0.3, 'ad_inpaint_only_masked': True, 'ad_inpaint_only_masked_padding': 32, 'ad_use_inpaint_width_height': False, 'ad_inpaint_width': 512, 'ad_inpaint_height': 512, 'ad_use_steps': False, 'ad_steps': 28, 'ad_use_cfg_scale': False, 'ad_cfg_scale': 7, 'ad_use_checkpoint': False, 'ad_checkpoint': 'Use same checkpoint', 'ad_use_vae': False, 'ad_vae': 'Use same VAE', 'ad_use_sampler': False, 'ad_sampler': 'DPM++ 2M Karras', 'ad_use_noise_multiplier': False, 'ad_noise_multiplier': 1, 'ad_use_clip_skip': False, 'ad_clip_skip': 1, 'ad_restore_face': False, 'ad_controlnet_model': 'None', 'ad_controlnet_module': 'None', 'ad_controlnet_weight': 1, 'ad_controlnet_guidance_start': 0, 'ad_controlnet_guidance_end': 1, 'is_api': ()}, {'ad_model': 'hand_yolov8n.pt', 'ad_prompt': 'hand', 'ad_negative_prompt': 'badhandv4, (deformed hand, deformed fingers, incorrect fingers), (negative_hand-neg:1.5), extra fingers, missing fingers', 'ad_confidence': 0.3, 'ad_mask_k_largest': 0, 'ad_mask_min_ratio': 0, 'ad_mask_max_ratio': 1, 'ad_x_offset': 0, 'ad_y_offset': 0, 'ad_dilate_erode': 4, 'ad_mask_merge_invert': 'None', 'ad_mask_blur': 4, 'ad_denoising_strength': 0.4, 'ad_inpaint_only_masked': True, 'ad_inpaint_only_masked_padding': 32, 'ad_use_inpaint_width_height': False, 'ad_inpaint_width': 512, 'ad_inpaint_height': 512, 'ad_use_steps': False, 'ad_steps': 28, 'ad_use_cfg_scale': False, 'ad_cfg_scale': 7, 'ad_use_checkpoint': False, 'ad_checkpoint': 'Use same checkpoint', 'ad_use_vae': False, 'ad_vae': 'Use same VAE', 'ad_use_sampler': False, 'ad_sampler': 'DPM++ 2M Karras', 'ad_use_noise_multiplier': False, 'ad_noise_multiplier': 1, 'ad_use_clip_skip': False, 'ad_clip_skip': 1, 'ad_restore_face': False, 'ad_controlnet_model': 'None', 'ad_controlnet_module': 'None', 'ad_controlnet_weight': 1, 'ad_controlnet_guidance_start': 0, 'ad_controlnet_guidance_end': 1, 'is_api': ()}, <scripts.animatediff_ui.AnimateDiffProcess object at 0x00000220A488EEF0>, <scripts.controlnet_ui.controlnet_ui_group.UiControlNetUnit object at 0x00000220A45C1660>, <scripts.controlnet_ui.controlnet_ui_group.UiControlNetUnit object at 0x00000220A45C2890>, <scripts.controlnet_ui.controlnet_ui_group.UiControlNetUnit object at 0x00000220A45C0820>, False, False, 'positive', 'comma', 0, False, False, '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0, False, None, None, False, None, None, False, None, None, False, 50, 'linear (weight sum)', '10', 'C:\\Users\\User\\stable-diffusion-2\\stable-diffusion-webui\\extensions\\stable-diffusion-webui-prompt-travel\\img\\ref_ctrlnet', 'Lanczos', 2, 0, 0, 'mp4', 10.0, 0, '', True, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, 'linear', 'lerp', 'token', 'random', '30', 'fixed', 1, '8', None, 'Lanczos', 2, 0, 0, 'mp4', 10.0, 0, '', True, False, False) {}
    Traceback (most recent call last):
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\modules\call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\modules\call_queue.py", line 36, in f
        res = func(*args, **kwargs)
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\modules\txt2img.py", line 55, in txt2img
        processed = processing.process_images(p)
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\modules\processing.py", line 732, in process_images
        res = process_images_inner(p)
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\extensions\sd-webui-animatediff\scripts\animatediff_cn.py", line 119, in hacked_processing_process_images_hijack
        return getattr(processing, '__controlnet_original_process_images_inner')(p, *args, **kwargs)
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\modules\processing.py", line 867, in process_images_inner
        samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts)
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\modules\processing.py", line 1156, in sample
        return self.sample_hr_pass(samples, decoded_samples, seeds, subseeds, subseed_strength, prompts)
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\modules\processing.py", line 1242, in sample_hr_pass
        samples = self.sampler.sample_img2img(self, samples, noise, self.hr_c, self.hr_uc, steps=self.hr_second_pass_steps or self.steps, image_conditioning=image_conditioning)
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\modules\sd_samplers_kdiffusion.py", line 188, in sample_img2img
        samples = self.launch_sampling(t_enc + 1, lambda: self.func(self.model_wrap_cfg, xi, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\modules\sd_samplers_common.py", line 261, in launch_sampling
        return func()
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\modules\sd_samplers_kdiffusion.py", line 188, in <lambda>
        samples = self.launch_sampling(t_enc + 1, lambda: self.func(self.model_wrap_cfg, xi, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
        return func(*args, **kwargs)
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\repositories\k-diffusion\k_diffusion\sampling.py", line 553, in sample_dpmpp_sde
        denoised = model(x, sigmas[i] * s_in, **extra_args)
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\extensions\sd-webui-animatediff\scripts\animatediff_infv2v.py", line 296, in mm_cfg_forward
        x_out[a:b] = self.inner_model(x_in[a:b], sigma_in[a:b], cond=make_condition_dict(c_crossattn, image_cond_in[a:b]))
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\repositories\k-diffusion\k_diffusion\external.py", line 112, in forward
        eps = self.get_eps(input * c_in, self.sigma_to_t(sigma), **kwargs)
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\repositories\k-diffusion\k_diffusion\external.py", line 138, in get_eps
        return self.inner_model.apply_model(*args, **kwargs)
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\modules\sd_hijack_utils.py", line 17, in <lambda>
        setattr(resolved_obj, func_path[-1], lambda *args, **kwargs: self(*args, **kwargs))
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\modules\sd_hijack_utils.py", line 28, in __call__
        return self.__orig_func(*args, **kwargs)
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\models\diffusion\ddpm.py", line 858, in apply_model
        x_recon = self.model(x_noisy, t, **cond)
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\models\diffusion\ddpm.py", line 1335, in forward
        out = self.diffusion_model(x, t, context=cc)
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\modules\sd_unet.py", line 91, in UNetModel_forward
        return ldm.modules.diffusionmodules.openaimodel.copy_of_UNetModel_forward_for_webui(self, x, timesteps, context, *args, **kwargs)
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\openaimodel.py", line 797, in forward
        h = module(h, emb, context)
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\openaimodel.py", line 86, in forward
        x = layer(x)
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\extensions\sd-webui-animatediff\motion_module.py", line 107, in forward
        return self.temporal_transformer(input_tensor, encoder_hidden_states, attention_mask)
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\extensions\sd-webui-animatediff\motion_module.py", line 173, in forward
        hidden_states = block(hidden_states, encoder_hidden_states=encoder_hidden_states, video_length=video_length)
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\extensions\sd-webui-animatediff\motion_module.py", line 237, in forward
        hidden_states = attention_block(
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\extensions\sd-webui-animatediff\motion_module.py", line 596, in forward
        hidden_states = self._memory_efficient_attention(query, key, value, attention_mask, optimizer_name)
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\extensions\sd-webui-animatediff\motion_module.py", line 494, in _memory_efficient_attention
        hidden_states = xformers.ops.memory_efficient_attention(
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\venv\lib\site-packages\xformers\ops\fmha\__init__.py", line 192, in memory_efficient_attention
        return _memory_efficient_attention(
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\venv\lib\site-packages\xformers\ops\fmha\__init__.py", line 290, in _memory_efficient_attention
        return _memory_efficient_attention_forward(
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\venv\lib\site-packages\xformers\ops\fmha\__init__.py", line 310, in _memory_efficient_attention_forward
        out, *_ = op.apply(inp, needs_gradient=False)
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\venv\lib\site-packages\xformers\ops\fmha\cutlass.py", line 175, in apply
        out, lse, rng_seed, rng_offset = cls.OPERATOR(
      File "C:\Users\User\stable-diffusion-2\stable-diffusion-webui\venv\lib\site-packages\torch\_ops.py", line 502, in __call__
        return self._op(*args, **kwargs or {})
    RuntimeError: CUDA error: invalid configuration argument
    CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
    For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
    Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.


---

continue-revolution · 2023-11-03T17:54:07Z

@pipa0979 This error is unrelated to this issue. It is xformer's bug. Go to Settings/AnimateDiff and you'll know what you should do.

continue-revolution · 2023-11-03T17:58:19Z

For people coming here. This issue is related to you ONLY IF you switch from xformers to --opt-sdp-attention and still see CUDA error TORCH_USE_CUDA_DSA. The majority versions of xformers are buggy, and AnimateDiff is very unfortunately affected. xformers==0.0.20 does not have that bug.

pipa0979 · 2023-11-03T17:59:26Z

This is the exact same issue. I think you are talking about these settings right?

Optimize attention layers with xformers
Optimize attention layers with sdp (torch >= 2.0.0 required)
Do not optimize attention layers

My Cmd

set COMMANDLINE_ARGS=--xformers --gradio-img2img-tool color-sketch

I am currently on Optimize attention layers with xformers and still see this issue.

continue-revolution · 2023-11-03T18:00:38Z

No you should not use the first option. you should NOT use xformers

pipa0979 · 2023-11-03T18:10:14Z

Yes that worked @continue-revolution . For the error above, did deleting the venb dir from SD home and retrying running the webui-user.bat work? @Evilander

FrankHGS · 2023-11-09T00:19:11Z

也有相同問題
我的方法是:

a. When you have --xformers in your command line args, you want AnimateDiff to > Optimize attention layers with sdp

或 (但出圖很慢!)
b. CMD完全沒加任何參數

Optimizations 歸零 > Cross attention optimization 自動
然後就正常使用出圖了!

版本:
version: v1.6.0-2 • python: 3.10.6 • torch: 2.0.1+cu118 • xformers: N/A • gradio: 3.41.2

wywywywy · 2023-11-13T12:23:43Z

What solves this for me (after having reinstalled the webui from scratch which solved the issue only to have it resurface after changing some of the Optimization settings) was to set all of the following settings to 0:
Negative Guidance minimum sigma 
Token merging ratio
Token merging ratio for img2img 
Token merging ratio for high-res pass 
I booted up the old install and set those to zero and then the issue went away there as well.

From my testing, it looks like only Negative Guidance minimum sigma needs to be zero. Token merging still works correctly.

lcllge · 2023-11-20T01:52:25Z

是的, 我昨天也遇到了一样的问题, 在我的3060 12g上表现为AnimateDiff生成视频时百分之31就导致webui崩溃, 具体报错为:

RuntimeError: CUDA error: invalid configuration argument
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

经过我测试, 把版本回退到1.5.2然后升级到1.6.0正常使用AnimateDiff就没有问题, 但是我因为觉得我的设备生成速度不够快, 然后改动了配置

Negative Guidance minimum sigma 
Token merging ratio

(另外我看到我仅仅改动上述两项配置时, 但保存的修改项足足有四十多个, 抱歉我这里没有截图, 如果可以复现请反馈)
后来我将这两个配置又设置变成0, 可他还是会重复出现, 目前我没有再下载一个1.5.2的环境升级到1.6.0再测试, 希望这个示例可以给你带来帮助, 然后就又复现了, 因此我觉得不一定是xforms的问题, 有可能是webui在第一次保存配置时会将一些对运行animatediff不利的配置保存起来,然后再次运行animatediff时就出现那个显存不足以至于直接崩溃的程序问题.

问题复现步骤:
1.5.2 -> 1.6.0 (不改动配置正常可以使用animatediff)
在1.6.0基础上修配置并保存, 百分之百触发 "RuntimeError: CUDA error: invalid configuration argument" 这个错误, 并导致webui, 崩溃

解决方案:
暂无, 我会回去试试调整上面几位爱好者说的那几个选项, 看它们是否有效

lcllge · 2023-11-20T06:35:13Z

是的，我昨天也遇到了同样的问题，在我的 3060 12g 上接收 AnimateDiff 生成视频时，有百分之 31 就导致 webui 崩溃，具体报错为：
RuntimeError: CUDA error: invalid configuration argument
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.
经过我测试，把版本退到1.5.2然后升级到1.6.0正常使用AnimateDiff就没有问题，但是我觉得因为我的设备生成速度不够快，然后就了配置
Negative Guidance minimum sigma 
Token merging ratio
（另外我看到我大致上述两个配置时，但保存的修改项足有四十多个，抱歉我这里没有截图，如果可以现复请反馈）后来我将这两个配置又设置了变成0，可他还是会重复出现，目前我没有再下载一个1.5.2的环境升级到1.6.0再测试，希望这个例子可以给你带来帮助，然后就又复现了，因此我觉得不一定是xforms的问题，有可能是webui在第一次保存配置时把一些对运行animatediff不利的配置保存起来，然后再次运行animatediff时就出现那个明显存在不足以致直接崩溃的程序问题。

问题复现步骤： 1.5.2 -> 1.6.0（不配置配置正常可以使用animatediff）在1.6.0基础上修改配置并保存，百分之百触发“RuntimeError：CUDA错误：无效的配置参数”这个错误，并导致WebUI，崩溃

解决方案：暂无，我会重新尝试调整上面几位爱好者说的那几个选项，看看它们是否有效

或者webui是否可以提供恢复默认配置的方法, 以便于将config.json文件重置为未修改时的值

continue-revolution · 2023-11-20T06:41:05Z

删除config.json就能恢复到默认

lcllge · 2023-11-20T06:43:32Z

删除config.json即可恢复到默认

哦, 我的朋友, 你说的这个方法没准真的可以, 如果它解决了这个问题, 我决定授予你大佬的称号

lcllge · 2023-11-21T01:20:20Z

删除config.json即可恢复到默认

昨天晚上回去找到原因了, 在秋叶的高级设置里面找到Pytorch配置, 不要选带有Cuda选项的配置, 直接选Pytorch就可以了, 但是现在又遇到一个问题, 生成速度很慢这个不晓得咋整哦, 按LCM采样器来看, 开启优化选项可以达到10s内出1024x1536图片, 但是我画16帧率图片, 需要十分钟, 按理来讲应该是3分钟的事情, 不知道为什么变得这么慢, 但是现在不爆显存了, 还是值得庆贺的, 希望可以优化下在LCM采样器下的生成速度

tofof · 2023-11-25T03:21:12Z

What solves this for me (after having reinstalled the webui from scratch which solved the issue only to have it resurface after changing some of the Optimization settings) was to set all of the following settings to 0:
Negative Guidance minimum sigma 
Token merging ratio
Token merging ratio for img2img 
Token merging ratio for high-res pass 
From my testing, it looks like only Negative Guidance minimum sigma needs to be zero. Token merging still works correctly.

I agree.
torch 2.0.1+cu118, sdp enabled under animatediff's "when --xformers.." setting and as the cross-attention setting for base SD, with and without token merging enabled works just fine.
torch 2.1.0+cu121, same settings work both ways.
As soon as I enable negative guidance minimum sigma to any value > 0, with either version of torch, it errors midway through the generation (presumably at the point that the minimum sigma threshold is actually crossed at which point that optimization tries to remove the negative prompt).

s4130 · 2023-11-27T06:39:41Z

我也有这个问题,虽然我只是4g的1650ti,
这是我的命令行参数:
--lowvram --theme light --xformers --precision full --no-half --no-half-vae --listen --enable-insecure-extension-access --no-gradio-queue
这是我的环境:
version: [v1.6.0-419-g3d341ebc7](https://github.com/AUTOMATIC1111/stable-diffusion-webui/commit/3d341ebc7dcb44df3b4c013b3805c08d8a35e24a) • python: 3.10.11 • torch: 2.1.0+cu121 • xformers: 0.0.22.post7 • gradio: 3.41.2
下面是我控制台的报错:

C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [518,0,0], thread: [32,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [518,0,0], thread: [33,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [518,0,0], thread: [34,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [518,0,0], thread: [35,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [518,0,0], thread: [36,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [518,0,0], thread: [37,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [518,0,0], thread: [38,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [518,0,0], thread: [39,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [518,0,0], thread: [40,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [518,0,0], thread: [41,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [518,0,0], thread: [42,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [518,0,0], thread: [43,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [518,0,0], thread: [44,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [518,0,0], thread: [45,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [518,0,0], thread: [46,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [518,0,0], thread: [47,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [518,0,0], thread: [48,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [518,0,0], thread: [49,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [518,0,0], thread: [50,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [518,0,0], thread: [51,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [518,0,0], thread: [52,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [518,0,0], thread: [53,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [518,0,0], thread: [54,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [518,0,0], thread: [55,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [518,0,0], thread: [56,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [518,0,0], thread: [57,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [518,0,0], thread: [58,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [518,0,0], thread: [59,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [518,0,0], thread: [60,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [518,0,0], thread: [61,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [518,0,0], thread: [62,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [518,0,0], thread: [63,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [580,0,0], thread: [32,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [580,0,0], thread: [33,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [580,0,0], thread: [34,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [580,0,0], thread: [35,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [580,0,0], thread: [36,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [580,0,0], thread: [37,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [580,0,0], thread: [38,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [580,0,0], thread: [39,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [580,0,0], thread: [40,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [580,0,0], thread: [41,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [580,0,0], thread: [42,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [580,0,0], thread: [43,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [580,0,0], thread: [44,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [580,0,0], thread: [45,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [580,0,0], thread: [46,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [580,0,0], thread: [47,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [580,0,0], thread: [48,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [580,0,0], thread: [49,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [580,0,0], thread: [50,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [580,0,0], thread: [51,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [580,0,0], thread: [52,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [580,0,0], thread: [53,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [580,0,0], thread: [54,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [580,0,0], thread: [55,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [580,0,0], thread: [56,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [580,0,0], thread: [57,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [580,0,0], thread: [58,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [580,0,0], thread: [59,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [580,0,0], thread: [60,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [580,0,0], thread: [61,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [580,0,0], thread: [62,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [580,0,0], thread: [63,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
Exception in thread MemMon:
提示：Python 运行时抛出了一个异常。请检查疑难解答页面。
Traceback (most recent call last):
  File "D:\sd-webui-aki-v4.2\python\lib\threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "<enhanced_experience vendors.sentry_sdk.integrations.threading>", line 72, in run
  File "<enhanced_experience vendors.sentry_sdk._compat>", line 115, in reraise
  File "<enhanced_experience vendors.sentry_sdk.integrations.threading>", line 70, in run
  File "D:\sd-webui-aki-v4.2\modules\memmon.py", line 53, in run
    free, total = self.cuda_mem_get_info()
  File "D:\sd-webui-aki-v4.2\modules\memmon.py", line 34, in cuda_mem_get_info
    return torch.cuda.mem_get_info(index)
  File "D:\sd-webui-aki-v4.2\python\lib\site-packages\torch\cuda\memory.py", line 663, in mem_get_info
    return torch.cuda.cudart().cudaMemGetInfo(device)
RuntimeError: CUDA error: device-side assert triggered
提示：Python 运行时抛出了一个异常。请检查疑难解答页面。
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

*** Error completing request
*** Arguments: ('task(fo26lybrkqh4x65)', '<lora:pytorch_lora_weights>,nsfw,anime screencap,\nmagic century,1girl,moyou,witch,witch hat,(large breasts:1.1),black hair,asian,setting up stalls in medieval markets,stand behind the counter,sky,outdoors,open air,a tattered stall,(the tattered wooden shelves are adorned with colorful potions:1.1),(clamp a bottle of potion with your chest:1.4),<lora:1.5_perfect hands:1>,Perfect Hands', 'EasyNegative,negative_hand-neg,(dedicated_to_artificial_humans_1.2)', [], 10, 'Euler a', 1, 1, 1, 512, 512, False, 0.7, 2, 'Latent', 0, 0, 0, 'Use same checkpoint', 'Use same sampler', '', '', [], <gradio.routes.Request object at 0x000002475D484B50>, 0, False, '', 0.8, -1, False, -1, 0, 0, 0, '', False, False, {'ad_model': 'face_yolov8s.pt', 'ad_prompt': '', 'ad_negative_prompt': '', 'ad_confidence': 0.3, 'ad_mask_k_largest': 0, 'ad_mask_min_ratio': 0, 'ad_mask_max_ratio': 1, 'ad_x_offset': 0, 'ad_y_offset': 0, 'ad_dilate_erode': 4, 'ad_mask_merge_invert': 'None', 'ad_mask_blur': 4, 'ad_denoising_strength': 0.4, 'ad_inpaint_only_masked': True, 'ad_inpaint_only_masked_padding': 32, 'ad_use_inpaint_width_height': False, 'ad_inpaint_width': 512, 'ad_inpaint_height': 512, 'ad_use_steps': False, 'ad_steps': 28, 'ad_use_cfg_scale': False, 'ad_cfg_scale': 7, 'ad_use_checkpoint': False, 'ad_checkpoint': 'Use same checkpoint', 'ad_use_vae': False, 'ad_vae': 'Use same VAE', 'ad_use_sampler': False, 'ad_sampler': 'Euler a', 'ad_use_noise_multiplier': False, 'ad_noise_multiplier': 1, 'ad_use_clip_skip': False, 'ad_clip_skip': 1, 'ad_restore_face': False, 'ad_controlnet_model': 'None', 'ad_controlnet_module': 'None', 'ad_controlnet_weight': 1, 'ad_controlnet_guidance_start': 0, 'ad_controlnet_guidance_end': 1, 'is_api': ()}, {'ad_model': 'None', 'ad_prompt': '', 'ad_negative_prompt': '', 'ad_confidence': 0.3, 'ad_mask_k_largest': 0, 'ad_mask_min_ratio': 0, 'ad_mask_max_ratio': 1, 'ad_x_offset': 0, 'ad_y_offset': 0, 'ad_dilate_erode': 4, 'ad_mask_merge_invert': 'None', 'ad_mask_blur': 4, 'ad_denoising_strength': 0.4, 'ad_inpaint_only_masked': True, 'ad_inpaint_only_masked_padding': 32, 'ad_use_inpaint_width_height': False, 'ad_inpaint_width': 512, 'ad_inpaint_height': 512, 'ad_use_steps': False, 'ad_steps': 28, 'ad_use_cfg_scale': False, 'ad_cfg_scale': 7, 'ad_use_checkpoint': False, 'ad_checkpoint': 'Use same checkpoint', 'ad_use_vae': False, 'ad_vae': 'Use same VAE', 'ad_use_sampler': False, 'ad_sampler': 'Euler a', 'ad_use_noise_multiplier': False, 'ad_noise_multiplier': 1, 'ad_use_clip_skip': False, 'ad_clip_skip': 1, 'ad_restore_face': False, 'ad_controlnet_model': 'None', 'ad_controlnet_module': 'None', 'ad_controlnet_weight': 1, 'ad_controlnet_guidance_start': 0, 'ad_controlnet_guidance_end': 1, 'is_api': ()}, {'ad_model': 'None', 'ad_prompt': '', 'ad_negative_prompt': '', 'ad_confidence': 0.3, 'ad_mask_k_largest': 0, 'ad_mask_min_ratio': 0, 'ad_mask_max_ratio': 1, 'ad_x_offset': 0, 'ad_y_offset': 0, 'ad_dilate_erode': 4, 'ad_mask_merge_invert': 'None', 'ad_mask_blur': 4, 'ad_denoising_strength': 0.4, 'ad_inpaint_only_masked': True, 'ad_inpaint_only_masked_padding': 32, 'ad_use_inpaint_width_height': False, 'ad_inpaint_width': 512, 'ad_inpaint_height': 512, 'ad_use_steps': False, 'ad_steps': 28, 'ad_use_cfg_scale': False, 'ad_cfg_scale': 7, 'ad_use_checkpoint': False, 'ad_checkpoint': 'Use same checkpoint', 'ad_use_vae': False, 'ad_vae': 'Use same VAE', 'ad_use_sampler': False, 'ad_sampler': 'Euler a', 'ad_use_noise_multiplier': False, 'ad_noise_multiplier': 1, 'ad_use_clip_skip': False, 'ad_clip_skip': 1, 'ad_restore_face': False, 'ad_controlnet_model': 'None', 'ad_controlnet_module': 'None', 'ad_controlnet_weight': 1, 'ad_controlnet_guidance_start': 0, 'ad_controlnet_guidance_end': 1, 'is_api': ()}, False, 'MultiDiffusion', False, True, 1024, 1024, 96, 96, 48, 1, 'None', 2, False, 10, 1, 1, 64, False, False, False, False, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 960, 64, True, True, True, False, False, 7, 100, 'Constant', 0, 'Constant', 0, 4, True, 'MEAN', 'AD', 1, <scripts.animatediff_ui.AnimateDiffProcess object at 0x0000024758A506D0>, UiControlNetUnit(enabled=False, module='none', model='None', weight=1, image=None, resize_mode='Crop and Resize', low_vram=False, processor_res=64, threshold_a=64, threshold_b=64, guidance_start=0, guidance_end=1, pixel_perfect=False, control_mode='Balanced', save_detected_map=True), UiControlNetUnit(enabled=False, module='none', model='None', weight=1, image=None, resize_mode='Crop and Resize', low_vram=False, processor_res=64, threshold_a=64, threshold_b=64, guidance_start=0, guidance_end=1, pixel_perfect=False, control_mode='Balanced', save_detected_map=True), UiControlNetUnit(enabled=False, module='none', model='None', weight=1, image=None, resize_mode='Crop and Resize', low_vram=False, processor_res=64, threshold_a=64, threshold_b=64, guidance_start=0, guidance_end=1, pixel_perfect=False, control_mode='Balanced', save_detected_map=True), False, '', 0.5, True, False, '', 'Lerp', False, False, False, 'Matrix', 'Columns', 'Mask', 'Prompt', '1,1', '0.2', False, False, False, 'Attention', [False], '0', '0', '0.4', None, '0', '0', False, False, False, 'positive', 'comma', 0, False, False, 'start', '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0, False, None, None, False, None, None, False, None, None, False, 50, [], 30, '', 4, [], 1, '', '', '', '') {}
    Traceback (most recent call last):
      File "D:\sd-webui-aki-v4.2\modules\call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
      File "D:\sd-webui-aki-v4.2\modules\call_queue.py", line 36, in f
        res = func(*args, **kwargs)
      File "D:\sd-webui-aki-v4.2\modules\txt2img.py", line 55, in txt2img
        processed = processing.process_images(p)
      File "D:\sd-webui-aki-v4.2\modules\processing.py", line 734, in process_images
        res = process_images_inner(p)
      File "D:\sd-webui-aki-v4.2\extensions\sd-webui-animatediff\scripts\animatediff_cn.py", line 118, in hacked_processing_process_images_hijack
        return getattr(processing, '__controlnet_original_process_images_inner')(p, *args, **kwargs)
      File "D:\sd-webui-aki-v4.2\modules\processing.py", line 868, in process_images_inner
        samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts)
      File "D:\sd-webui-aki-v4.2\modules\processing.py", line 1143, in sample
        samples = self.sampler.sample(self, x, conditioning, unconditional_conditioning, image_conditioning=self.txt2img_image_conditioning(x))
      File "D:\sd-webui-aki-v4.2\modules\sd_samplers_kdiffusion.py", line 235, in sample
        samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
      File "D:\sd-webui-aki-v4.2\modules\sd_samplers_common.py", line 261, in launch_sampling
        return func()
      File "D:\sd-webui-aki-v4.2\modules\sd_samplers_kdiffusion.py", line 235, in <lambda>
        samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
      File "D:\sd-webui-aki-v4.2\python\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
        return func(*args, **kwargs)
      File "D:\sd-webui-aki-v4.2\repositories\k-diffusion\k_diffusion\sampling.py", line 145, in sample_euler_ancestral
        denoised = model(x, sigmas[i] * s_in, **extra_args)
      File "D:\sd-webui-aki-v4.2\python\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
        return self._call_impl(*args, **kwargs)
      File "D:\sd-webui-aki-v4.2\python\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
        return forward_call(*args, **kwargs)
      File "D:\sd-webui-aki-v4.2\extensions\sd-webui-animatediff\scripts\animatediff_infv2v.py", line 250, in mm_cfg_forward
        x_out = mm_sd_forward(self, x_in, sigma_in, cond_in, image_cond_in, make_condition_dict) # hook
      File "D:\sd-webui-aki-v4.2\extensions\sd-webui-animatediff\scripts\animatediff_infv2v.py", line 161, in mm_sd_forward
        x_in[_context], sigma_in[_context],
    RuntimeError: CUDA error: device-side assert triggered
提示：Python 运行时抛出了一个异常。请检查疑难解答页面。
    CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
    For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
    Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

---
Traceback (most recent call last):
  File "D:\sd-webui-aki-v4.2\python\lib\site-packages\gradio\routes.py", line 488, in run_predict
    output = await app.get_blocks().process_api(
  File "D:\sd-webui-aki-v4.2\python\lib\site-packages\gradio\blocks.py", line 1431, in process_api
    result = await self.call_function(
  File "D:\sd-webui-aki-v4.2\python\lib\site-packages\gradio\blocks.py", line 1103, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "D:\sd-webui-aki-v4.2\python\lib\site-packages\anyio\to_thread.py", line 31, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "D:\sd-webui-aki-v4.2\python\lib\site-packages\anyio\_backends\_asyncio.py", line 937, in run_sync_in_worker_thread
    return await future
  File "D:\sd-webui-aki-v4.2\python\lib\site-packages\anyio\_backends\_asyncio.py", line 867, in run
    result = context.run(func, *args)
  File "D:\sd-webui-aki-v4.2\python\lib\site-packages\gradio\utils.py", line 707, in wrapper
    response = f(*args, **kwargs)
  File "D:\sd-webui-aki-v4.2\modules\call_queue.py", line 77, in f
    devices.torch_gc()
  File "D:\sd-webui-aki-v4.2\modules\devices.py", line 68, in torch_gc
    torch.cuda.empty_cache()
  File "D:\sd-webui-aki-v4.2\python\lib\site-packages\torch\cuda\memory.py", line 159, in empty_cache
    torch._C._cuda_emptyCache()
RuntimeError: CUDA error: device-side assert triggered
提示：Python 运行时抛出了一个异常。请检查疑难解答页面。
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

grechkaaaaa · 2023-12-07T04:07:41Z

Что решило это для меня (после переустановки webui с нуля, которая решила проблему только для того, чтобы она снова появилась после изменения некоторых настроек), так это установила все следующие настройки на 0:
Negative Guidance minimum sigma 
Token merging ratio
Token merging ratio for img2img 
Token merging ratio for high-res pass 
Я загрузил старые установки и установил их на ноль, после чего проблема тоже исчезла.

it works, thank bro @FreeeFry

zhang-sir029 · 2023-12-12T13:08:37Z

我也有遇到同样的问题，请求的时候控制台报设备错误，
报错信息：Traceback (most recent call last):
File "/opt/conda/envs/LCM2/lib/python3.10/site-packages/gradio/queueing.py", line 407, in call_prediction
output = await route_utils.call_process_api(
File "/opt/conda/envs/LCM2/lib/python3.10/site-packages/gradio/route_utils.py", line 226, in call_process_api
output = await app.get_blocks().process_api(
File "/opt/conda/envs/LCM2/lib/python3.10/site-packages/gradio/blocks.py", line 1550, in process_api
result = await self.call_function(
File "/opt/conda/envs/LCM2/lib/python3.10/site-packages/gradio/blocks.py", line 1185, in call_function
prediction = await anyio.to_thread.run_sync(
File "/opt/conda/envs/LCM2/lib/python3.10/site-packages/anyio/to_thread.py", line 33, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "/opt/conda/envs/LCM2/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
return await future
File "/opt/conda/envs/LCM2/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 807, in run
result = context.run(func, *args)
File "/opt/conda/envs/LCM2/lib/python3.10/site-packages/gradio/utils.py", line 661, in wrapper
response = f(*args, **kwargs)
File "/opt/conda/envs/LCM2/lib/python3.10/site-packages/gradio/utils.py", line 661, in wrapper
response = f(*args, **kwargs)
File "/home/ec2-user/LCM/latent-consistency-model/local_gradio/app.py", line 103, in generate
result = pipe(
File "/opt/conda/envs/LCM2/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/opt/conda/envs/LCM2/lib/python3.10/site-packages/diffusers/pipelines/latent_consistency_models/pipeline_latent_consistency_text2img.py", line 769, in call
latents, denoised = self.scheduler.step(model_pred, t, latents, **extra_step_kwargs, return_dict=False)
File "/opt/conda/envs/LCM2/lib/python3.10/site-packages/diffusers/schedulers/scheduling_lcm.py", line 508, in step
self._init_step_index(timestep)
File "/opt/conda/envs/LCM2/lib/python3.10/site-packages/diffusers/schedulers/scheduling_lcm.py", line 259, in _init_step_index
index_candidates = (self.timesteps == timestep).nonzero()
RuntimeError: CUDA error: an illegal memory access was encountered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.

+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 0 N/A N/A 4315 C python 5488MiB |
+-----------------------------------------------------------------------------+

zhang-sir029 · 2023-12-12T13:19:13Z

我也有遇到同样的问题，请求的时候控制台报设备错误，报错信息：Traceback (most recent call last): File "/opt/conda/envs/LCM2/lib/python3.10/site-packages/gradio/queueing.py", line 407, in call_prediction output = await route_utils.call_process_api( File "/opt/conda/envs/LCM2/lib/python3.10/site-packages/gradio/route_utils.py", line 226, in call_process_api output = await app.get_blocks().process_api( File "/opt/conda/envs/LCM2/lib/python3.10/site-packages/gradio/blocks.py", line 1550, in process_api result = await self.call_function( File "/opt/conda/envs/LCM2/lib/python3.10/site-packages/gradio/blocks.py", line 1185, in call_function prediction = await anyio.to_thread.run_sync( File "/opt/conda/envs/LCM2/lib/python3.10/site-packages/anyio/to_thread.py", line 33, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "/opt/conda/envs/LCM2/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread return await future File "/opt/conda/envs/LCM2/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 807, in run result = context.run(func, *args) File "/opt/conda/envs/LCM2/lib/python3.10/site-packages/gradio/utils.py", line 661, in wrapper response = f(*args, **kwargs) File "/opt/conda/envs/LCM2/lib/python3.10/site-packages/gradio/utils.py", line 661, in wrapper response = f(*args, **kwargs) File "/home/ec2-user/LCM/latent-consistency-model/local_gradio/app.py", line 103, in generate result = pipe( File "/opt/conda/envs/LCM2/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/opt/conda/envs/LCM2/lib/python3.10/site-packages/diffusers/pipelines/latent_consistency_models/pipeline_latent_consistency_text2img.py", line 769, in call latents, denoised = self.scheduler.step(model_pred, t, latents, **extra_step_kwargs, return_dict=False) File "/opt/conda/envs/LCM2/lib/python3.10/site-packages/diffusers/schedulers/scheduling_lcm.py", line 508, in step self._init_step_index(timestep) File "/opt/conda/envs/LCM2/lib/python3.10/site-packages/diffusers/schedulers/scheduling_lcm.py", line 259, in _init_step_index index_candidates = (self.timesteps == timestep).nonzero() RuntimeError: CUDA error: an illegal memory access was encountered CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.

GPU信息： (base) [ec2-user@ip-172-31-1-151 ~]$ nvidia-smi Tue Dec 12 11:54:25 2023 +-----------------------------------------------------------------------------+ | NVIDIA-SMI 525.85.12 Driver Version: 525.85.12 CUDA Version: 12.0 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 Tesla T4 On | 00000000:00:1E.0 Off | 0 | | N/A 26C P0 30W / 70W | 5493MiB / 15360MiB | 0% Default | | | | N/A | +-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=============================================================================| | 0 N/A N/A 4315 C python 5488MiB | +-----------------------------------------------------------------------------+

模型：https://huggingface.co/SimianLuo/LCM_Dreamshaper_v7/tree/main
工程：https://github.com/luosiallen/latent-consistency-model
python: 3.10.6 • • gradio: 3.50.0

gitmylo · 2024-04-19T10:07:11Z

I'm experiencing this same error, but instantly when starting the generation.

I'm on stable diffusion webui forge 29be1da7cf2b5dccfc70fbdd33eb35c56a31ffb7 (Currently the latest version, I also experienced this issue on older versions)

C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [78,0,0], thread: [61,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.

Steps to reproduce:

Enable animatediff
Select mm_sd15_v3_adapter.safetensors as the model (from The download link from the readme)
Run animatediff
The error is thrown in the console almost immedately after clicking "Generate"

Torch versions:
torch==2.1.2+cu121
torchvision==0.16.2+cu121

FreeeFry · 2024-04-19T10:11:23Z

I'm experiencing this same error, but instantly when starting the generation.

I'm on stable diffusion webui forge 29be1da7cf2b5dccfc70fbdd33eb35c56a31ffb7 (Currently the latest version, I also experienced this issue on older versions)
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\IndexKernel.cu:92: block: [78,0,0], thread: [61,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.
Steps to reproduce:
1. Enable animatediff

2. Select `mm_sd15_v3_adapter.safetensors` as the model (from [The download link from the readme](https://huggingface.co/conrevo/AnimateDiff-A1111/resolve/main/lora_v2/mm_sd15_v3_adapter.safetensors?download=true))

3. Run animatediff

4. The error is thrown in the console almost immedately after clicking "Generate"
Torch versions: torch==2.1.2+cu121 torchvision==0.16.2+cu121

Sounds like you've selected the mm_sd15_v3_adapter (LORA) as Motion module?

gitmylo · 2024-04-19T10:14:28Z

@FreeeFry
Bruh... You're right. I wasn't aware it was a LoRA. I was thinking like "Why is it in the lora_v2 folder?"

continue-revolution closed this as completed Nov 1, 2023

zappityzap mentioned this issue Nov 2, 2023

[Bug]: AnimateDiff goes away from the seed number #270

Closed

2 tasks

[Bug]: CUDA error at ~60% each time. 4070 - 8gb vram - 16gb DDR5 ram. #268

[Bug]: CUDA error at ~60% each time. 4070 - 8gb vram - 16gb DDR5 ram. #268

Comments

Evilander commented Oct 31, 2023

Is there an existing issue for this?

Have you read FAQ on README?

What happened?

Steps to reproduce the problem

What should have happened?

Commit where the problem happens

What browsers do you use to access the UI ?

Command Line Arguments

Console logs

Additional information

clear326 commented Oct 31, 2023

continue-revolution commented Oct 31, 2023

thezveroboy commented Nov 1, 2023

continue-revolution commented Nov 1, 2023

marberchen commented Nov 1, 2023

marberchen commented Nov 1, 2023

continue-revolution commented Nov 1, 2023 • edited Loading

continue-revolution commented Nov 1, 2023

continue-revolution commented Nov 1, 2023

continue-revolution commented Nov 1, 2023

continue-revolution commented Nov 1, 2023 • edited Loading

continue-revolution commented Nov 1, 2023 • edited Loading

GiusTex commented Nov 1, 2023 • edited Loading

Info:

What I tried:

Error log:

Update:

continue-revolution commented Nov 1, 2023

GiusTex commented Nov 1, 2023 • edited Loading

Update:

continue-revolution commented Nov 1, 2023

GiusTex commented Nov 1, 2023 • edited Loading

continue-revolution commented Nov 1, 2023

continue-revolution commented Nov 1, 2023

GiusTex commented Nov 1, 2023

User-bat:

Webui options:

continue-revolution commented Nov 1, 2023

GiusTex commented Nov 1, 2023

continue-revolution commented Nov 1, 2023

GiusTex commented Nov 1, 2023

GiusTex commented Nov 1, 2023 • edited Loading

continue-revolution commented Nov 1, 2023

continue-revolution commented Nov 1, 2023

GiusTex commented Nov 1, 2023 • edited Loading

Update:

Update:

continue-revolution commented Nov 1, 2023 • edited Loading

Evilander commented Nov 1, 2023

3dcinetv commented Nov 2, 2023

continue-revolution commented Nov 2, 2023

LIQUIDMIND111 commented Nov 2, 2023

FreeeFry commented Nov 2, 2023

LIQUIDMIND111 commented Nov 2, 2023

edwinzeng2005 commented Nov 3, 2023

pipa0979 commented Nov 3, 2023

pipa0979 commented Nov 3, 2023

continue-revolution commented Nov 3, 2023

continue-revolution commented Nov 3, 2023 • edited Loading

pipa0979 commented Nov 3, 2023 • edited Loading

continue-revolution commented Nov 3, 2023

pipa0979 commented Nov 3, 2023

FrankHGS commented Nov 9, 2023 • edited Loading

wywywywy commented Nov 13, 2023

lcllge commented Nov 20, 2023 • edited Loading

lcllge commented Nov 20, 2023

continue-revolution commented Nov 20, 2023

lcllge commented Nov 20, 2023

lcllge commented Nov 21, 2023

tofof commented Nov 25, 2023 • edited Loading

s4130 commented Nov 27, 2023 • edited Loading

grechkaaaaa commented Dec 7, 2023

zhang-sir029 commented Dec 12, 2023

zhang-sir029 commented Dec 12, 2023

gitmylo commented Apr 19, 2024

FreeeFry commented Apr 19, 2024

continue-revolution commented Nov 1, 2023 •

edited

Loading

continue-revolution commented Nov 1, 2023 •

edited

Loading

continue-revolution commented Nov 1, 2023 •

edited

Loading

GiusTex commented Nov 1, 2023 •

edited

Loading

GiusTex commented Nov 1, 2023 •

edited

Loading

GiusTex commented Nov 1, 2023 •

edited

Loading

GiusTex commented Nov 1, 2023 •

edited

Loading

GiusTex commented Nov 1, 2023 •

edited

Loading

continue-revolution commented Nov 1, 2023 •

edited

Loading

continue-revolution commented Nov 3, 2023 •

edited

Loading

pipa0979 commented Nov 3, 2023 •

edited

Loading

FrankHGS commented Nov 9, 2023 •

edited

Loading

lcllge commented Nov 20, 2023 •

edited

Loading

tofof commented Nov 25, 2023 •

edited

Loading

s4130 commented Nov 27, 2023 •

edited

Loading