VBAR allocation failed

[seattled6-max]

Custom Node Testing

• I have tried disabling custom nodes and the issue persists (see how to disable custom nodes if you need help)

Expected Behavior

videos to generate

Actual Behavior

after around 20 videos, i got error listed in title and comfyui stopped generating videos. i have not noticed a particular way to replicate the issue, i have a 5090, 128gb RAM

Steps to Reproduce

i am new to comfyui so i am not totally sure what ti include in the bug report :(

Debug Logs

got prompt
WARNING: PlaySound.IS_CHANGED() missing 1 required positional argument: 'self'
Requested to load WanTEModel
aimdo: src/model-vbar.c:198:ERROR:Could not reseve Virtual Address space for VBAR
!!! Exception during processing !!! VBAR allocation failed
Traceback (most recent call last):
  File "D:\rbwu4v\ComfyUI_apprentice_portable\ComfyUI\execution.py", line 524, in execute
    output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
                                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\rbwu4v\ComfyUI_apprentice_portable\ComfyUI\execution.py", line 333, in get_output_data
    return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\rbwu4v\ComfyUI_apprentice_portable\ComfyUI\custom_nodes\ComfyUI-Lora-Manager\py\metadata_collector\metadata_hook.py", line 168, in async_map_node_over_list_with_metadata
    results = await original_map_node_over_list(
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\rbwu4v\ComfyUI_apprentice_portable\ComfyUI\execution.py", line 307, in _async_map_node_over_list
    await process_inputs(input_dict, i)
  File "D:\rbwu4v\ComfyUI_apprentice_portable\ComfyUI\execution.py", line 295, in process_inputs
    result = f(**inputs)
             ^^^^^^^^^^^
  File "D:\rbwu4v\ComfyUI_apprentice_portable\ComfyUI\nodes.py", line 80, in encode
    return (clip.encode_from_tokens_scheduled(tokens), )
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\rbwu4v\ComfyUI_apprentice_portable\ComfyUI\comfy\sd.py", line 313, in encode_from_tokens_scheduled
    pooled_dict = self.encode_from_tokens(tokens, return_pooled=return_pooled, return_dict=True)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\rbwu4v\ComfyUI_apprentice_portable\ComfyUI\comfy\sd.py", line 375, in encode_from_tokens
    self.load_model(tokens)
  File "D:\rbwu4v\ComfyUI_apprentice_portable\ComfyUI\comfy\sd.py", line 422, in load_model
    model_management.load_models_gpu([self.patcher], memory_required=memory_used)
  File "D:\rbwu4v\ComfyUI_apprentice_portable\ComfyUI\comfy\model_management.py", line 755, in load_models_gpu
    loaded_model.model_load(lowvram_model_memory, force_patch_weights=force_patch_weights)
  File "D:\rbwu4v\ComfyUI_apprentice_portable\ComfyUI\comfy\model_management.py", line 538, in model_load
    self.model_use_more_vram(use_more_vram, force_patch_weights=force_patch_weights)
  File "D:\rbwu4v\ComfyUI_apprentice_portable\ComfyUI\comfy\model_management.py", line 568, in model_use_more_vram
    return self.model.partially_load(self.device, extra_memory, force_patch_weights=force_patch_weights)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\rbwu4v\ComfyUI_apprentice_portable\ComfyUI\comfy\model_patcher.py", line 1666, in partially_load
    raise e
  File "D:\rbwu4v\ComfyUI_apprentice_portable\ComfyUI\comfy\model_patcher.py", line 1663, in partially_load
    self.load(device_to, dirty=dirty)
  File "D:\rbwu4v\ComfyUI_apprentice_portable\ComfyUI\comfy\model_patcher.py", line 1507, in load
    vbar = self._vbar_get(create=True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\rbwu4v\ComfyUI_apprentice_portable\ComfyUI\comfy\model_patcher.py", line 1455, in _vbar_get
    vbar = comfy_aimdo.model_vbar.ModelVBAR(self.model_size() * 10, self.load_device.index)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\rbwu4v\ComfyUI_apprentice_portable\python_embeded\Lib\site-packages\comfy_aimdo\model_vbar.py", line 40, in __init__
    raise MemoryError("VBAR allocation failed")
MemoryError: VBAR allocation failed

Prompt executed in 0.11 seconds

Other

No response

[rattus128]

What models / templates are you using and do you have any special startup flags? Can you paste your startup log from when comfy starts up to "To see the GUI ..."

[rattus128]

You have a custom node ComfyUI-Lora-Manager, can you describe how it sits in your workflow?

[rattus128]

Thanks a lot for this bug report! We found and fixed an issue and it is now likely resolved for you if you update comfy and refresh your requirements.txt. If you still have an issue feel free to repaste with fresh logs and workflow etc.

[PowerSix]

I got hit by this bug yesterday, figured I'll deal with it in the morning.
Looking into it now, and not only I see it reported, but already fixed!

This is crazy. You guys are awesome! ❤️

[seattled6-max]

Thanks a lot for this bug report! We found and fixed an issue and it is now likely resolved for you if you update comfy and refresh your requirements.txt. If you still have an issue feel free to repaste with fresh logs and workflow etc.

As far as I can tell, this has fixed the issue, thanks!

[Kallamamran]

Well what can I say? 🤔

got prompt
Model Flux2TEModel_ prepared for dynamic VRAM loading. 15622MB Staged. 0 patches attached. Force pre-loaded 145 weights: 602 KB.
0 models unloaded.
Model Flux2 prepared for dynamic VRAM loading. 17316MB Staged. 0 patches attached.
  0%|                                                                                                                        | 0/20 [00:00<?, ?it/s,   Model Initializing ...  ]aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 519 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 520 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 521 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 522 pin_count=4
  5%|█████▎                                                                                                   | 1/20 [00:11<03:31, 11.13s/it,  Model Initialization complete!  ]aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 519 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 520 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 521 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 522 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 519 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 520 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 521 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 522 pin_count=4
 15%|█████████████████████                                                                                                                       | 3/20 [00:02<00:09,  1.74it/s]aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 519 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 520 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 521 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 522 pin_count=4
 20%|████████████████████████████                                                                                                                | 4/20 [00:03<00:22,  1.38s/it]aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 519 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 520 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 521 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 522 pin_count=4
 25%|███████████████████████████████████                                                                                                         | 5/20 [00:04<00:12,  1.24it/s]aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 519 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 520 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 521 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 522 pin_count=4
 30%|██████████████████████████████████████████                                                                                                  | 6/20 [00:05<00:19,  1.40s/it]aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 519 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 520 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 521 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 522 pin_count=4
 35%|█████████████████████████████████████████████████                                                                                           | 7/20 [00:06<00:10,  1.20it/s]aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 519 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 520 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 521 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 522 pin_count=4
 40%|████████████████████████████████████████████████████████                                                                                    | 8/20 [00:08<00:16,  1.38s/it]aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 519 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 520 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 521 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 522 pin_count=4
 45%|███████████████████████████████████████████████████████████████                                                                             | 9/20 [00:08<00:08,  1.24it/s]aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 519 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 520 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 521 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 522 pin_count=4
 50%|█████████████████████████████████████████████████████████████████████▌                                                                     | 10/20 [00:09<00:10,  1.07s/it]aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 519 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 520 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 521 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 522 pin_count=4
 55%|████████████████████████████████████████████████████████████████████████████▍                                                              | 11/20 [00:11<00:09,  1.08s/it]aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 519 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 520 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 521 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 522 pin_count=4
 60%|███████████████████████████████████████████████████████████████████████████████████▍                                                       | 12/20 [00:12<00:08,  1.06s/it]aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 519 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 520 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 521 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 522 pin_count=4
 65%|██████████████████████████████████████████████████████████████████████████████████████████▎                                                | 13/20 [00:13<00:07,  1.07s/it]aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 519 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 520 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 521 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 522 pin_count=4
 70%|█████████████████████████████████████████████████████████████████████████████████████████████████▎                                         | 14/20 [00:14<00:06,  1.07s/it]aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 519 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 520 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 521 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 522 pin_count=4
 75%|████████████████████████████████████████████████████████████████████████████████████████████████████████▎                                  | 15/20 [00:15<00:05,  1.08s/it]aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 519 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 520 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 521 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 522 pin_count=4
 80%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████▏                           | 16/20 [00:16<00:04,  1.07s/it]aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 519 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 520 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 521 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 522 pin_count=4
 85%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏                    | 17/20 [00:17<00:03,  1.06s/it]aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 519 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 520 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 521 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 522 pin_count=4
 90%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████              | 18/20 [00:18<00:02,  1.07s/it]aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 519 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 520 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 521 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 522 pin_count=4
 95%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████       | 19/20 [00:19<00:01,  1.08s/it]aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 519 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 520 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 521 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 522 pin_count=4
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:20<00:00,  1.03s/it]
0 models unloaded.
Model AutoencoderKL prepared for dynamic VRAM loading. 160MB Staged. 0 patches attached.
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 519 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 520 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 521 pin_count=4
aimdo: src/model-vbar.c:74:WARNING:VBAR 0000024A04168030: Page 522 pin_count=4
Prompt executed in 33.53 seconds