hostbuf_file_reader_read failed

[RuneGjerde]

Custom Node Testing

• I have tried disabling custom nodes and the issue persists (see how to disable custom nodes if you need help)

Expected Behavior

Tried disable all custom nodes. And issue persists

Model/workflow used: Ideogram
Latest updated ComfyUI with aimdo 0.4.8

RTX 3090

(unfortunately i could not test --disable-dynamic-vram since that made python crash, perhaps due to model size or something)

Not sure if this topic is related but just in case the reference is useful #14239

Just in case the progress flow is useful :

Actual Behavior

Steps to Reproduce

Ideogram workflow

Debug Logs

[INFO] got prompt
[INFO] Using xformers attention in VAE
[INFO] Using xformers attention in VAE
[INFO] VAE load device: cuda:0, offload device: cpu, dtype: torch.bfloat16
[INFO] Found quantization metadata version 1
[INFO] Detected mixed precision quantization
[INFO] Using mixed precision operations
[INFO] Native ops: int8_tensorwise , emulated ops: mxfp8, nvfp4, float8_e4m3fn, float8_e5m2
[INFO] model weight dtype torch.bfloat16, manual cast: torch.bfloat16
[INFO] model_type FLOW
[INFO] Found quantization metadata version 1
[MultiGPU Core Patching] text_encoder_device_patched returning device: cuda:0 (current_text_encoder_device=cuda:0)
[INFO] Using MixedPrecisionOps for text encoder
[INFO] CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cpu, dtype: torch.float16
[INFO] Requested to load Ideogram4TEModel_
[INFO] Model Ideogram4TEModel_ prepared for dynamic VRAM loading. 7811MB Staged. 0 patches attached. Force pre-loaded 144 weights: 594 KB.

[ERROR] !!! Exception during processing !!! hostbuf_file_reader_read failed
[ERROR] Traceback (most recent call last):

Other

No response

[coderabbitai]

⚠️ Possible Duplicate Issue(s)

• RuntimeError: hostbuf_file_reader_read failed #14239

🔗 Related PRs

#13801 - fix: Fix bug when mask not on same device (CORE-181) [merged]

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

[RuneGjerde]

Not sure if its needed, but in case it helps, the full error log:

    output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
                                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\AI\ComfyUI\execution.py", line 336, in get_output_data
    return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\AI\ComfyUI\execution.py", line 310, in _async_map_node_over_list
    await process_inputs(input_dict, i)
  File "E:\AI\ComfyUI\execution.py", line 298, in process_inputs
    result = f(**inputs)
  File "E:\AI\ComfyUI\nodes.py", line 79, in encode
    return (clip.encode_from_tokens_scheduled(tokens), )
            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^
  File "E:\AI\ComfyUI\comfy\sd.py", line 326, in encode_from_tokens_scheduled
    pooled_dict = self.encode_from_tokens(tokens, return_pooled=return_pooled, return_dict=True)
  File "E:\AI\ComfyUI\comfy\sd.py", line 395, in encode_from_tokens
    o = self.cond_stage_model.encode_token_weights(tokens)
  File "E:\AI\ComfyUI\comfy\text_encoders\ideogram4.py", line 62, in encode_token_weights
    out, pooled, extra = super().encode_token_weights(token_weight_pairs)
                         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "E:\AI\ComfyUI\comfy\sd1_clip.py", line 737, in encode_token_weights
    out = getattr(self, self.clip).encode_token_weights(token_weight_pairs)
  File "E:\AI\ComfyUI\comfy\sd1_clip.py", line 45, in encode_token_weights
    o = self.encode(to_encode)
  File "E:\AI\ComfyUI\comfy\sd1_clip.py", line 306, in encode
    return self(tokens)
  File "E:\AI\ComfyUI\.venv-pt210\Lib\site-packages\torch\nn\modules\module.py", line 1776, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "E:\AI\ComfyUI\.venv-pt210\Lib\site-packages\torch\nn\modules\module.py", line 1787, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\AI\ComfyUI\comfy\sd1_clip.py", line 279, in forward
    outputs = self.transformer(None, attention_mask_model, embeds=embeds, num_tokens=num_tokens, intermediate_output=intermediate_output, final_layer_norm_intermediate=self.layer_norm_hidden_state, dtype=torch.float32, embeds_info=embeds_info)
  File "E:\AI\ComfyUI\.venv-pt210\Lib\site-packages\torch\nn\modules\module.py", line 1776, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "E:\AI\ComfyUI\.venv-pt210\Lib\site-packages\torch\nn\modules\module.py", line 1787, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\AI\ComfyUI\comfy\text_encoders\llama.py", line 834, in forward
    return self.model(input_ids, *args, **kwargs)
           ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\AI\ComfyUI\.venv-pt210\Lib\site-packages\torch\nn\modules\module.py", line 1776, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "E:\AI\ComfyUI\.venv-pt210\Lib\site-packages\torch\nn\modules\module.py", line 1787, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\AI\ComfyUI\comfy\text_encoders\llama.py", line 759, in forward
    x, current_kv = layer(
                    ~~~~~^
        x=x,
        ^^^^
    ...<3 lines>...
        past_key_value=past_kv,
        ^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "E:\AI\ComfyUI\.venv-pt210\Lib\site-packages\torch\nn\modules\module.py", line 1776, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "E:\AI\ComfyUI\.venv-pt210\Lib\site-packages\torch\nn\modules\module.py", line 1787, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\AI\ComfyUI\comfy\text_encoders\llama.py", line 580, in forward
    x, present_key_value = self.self_attn(
                           ~~~~~~~~~~~~~~^
        hidden_states=x,
        ^^^^^^^^^^^^^^^^
    ...<3 lines>...
        past_key_value=past_key_value,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "E:\AI\ComfyUI\.venv-pt210\Lib\site-packages\torch\nn\modules\module.py", line 1776, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "E:\AI\ComfyUI\.venv-pt210\Lib\site-packages\torch\nn\modules\module.py", line 1787, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\AI\ComfyUI\comfy\text_encoders\llama.py", line 501, in forward
    xq = self.q_proj(hidden_states)
  File "E:\AI\ComfyUI\.venv-pt210\Lib\site-packages\torch\nn\modules\module.py", line 1776, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "E:\AI\ComfyUI\.venv-pt210\Lib\site-packages\torch\nn\modules\module.py", line 1787, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\AI\ComfyUI\comfy\ops.py", line 1244, in forward
    output = self.forward_comfy_cast_weights(input, compute_dtype, want_requant=isinstance(input, QuantizedTensor))
  File "E:\AI\ComfyUI\comfy\ops.py", line 1187, in forward_comfy_cast_weights
    weight, bias, offload_stream = cast_bias_weight(self, input, offloadable=True, compute_dtype=compute_dtype, want_requant=want_requant)
                                   ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\AI\ComfyUI\comfy\ops.py", line 321, in cast_bias_weight
    offload_stream = cast_modules_with_vbar([s], dtype, device, bias_dtype, non_blocking)
  File "E:\AI\ComfyUI\comfy\ops.py", line 188, in cast_modules_with_vbar
    handle_pin(s, pin, xfer_source, xfer_dest, size=dest_size)
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\AI\ComfyUI\comfy\ops.py", line 186, in handle_pin
    cast_maybe_lowvram_patch(source, pin, offload_stream, xfer_dest2=dest)
    ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\AI\ComfyUI\comfy\ops.py", line 177, in cast_maybe_lowvram_patch
    comfy.model_management.cast_to_gathered(xfer_source, xfer_dest, non_blocking=non_blocking, stream=stream, r2=xfer_dest2)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\AI\ComfyUI\comfy\model_management.py", line 1439, in cast_to_gathered
    if comfy.memory_management.read_tensor_file_slice_into(tensor, dest_view, stream=stream, destination2=dest2_view):
       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\AI\ComfyUI\comfy\memory_management.py", line 21, in read_tensor_file_slice_into
    if not read_tensor_file_slice_into(tensor._qdata,
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
                                       destination._qdata if destination is not None else None, stream=stream,
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                       destination2=(destination2._qdata if destination2 is not None else None)):
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\AI\ComfyUI\comfy\memory_management.py", line 59, in read_tensor_file_slice_into
    comfy_aimdo.host_buffer.read_file_to_device(file_obj, info.offset, info.size,
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                                stream_ptr, destination2.data_ptr(),
                                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                                destination2.device.index,
                                                ^^^^^^^^^^^^^^^^^^^^^^^^^^
                                                mark_cold=False)
                                                ^^^^^^^^^^^^^^^^
  File "E:\AI\ComfyUI\.venv-pt210\Lib\site-packages\comfy_aimdo\host_buffer.py", line 71, in read_file_to_device
    raise RuntimeError("hostbuf_file_reader_read failed")
RuntimeError: hostbuf_file_reader_read failed```

[rattus128]

Can I get your startup log?

[RuneGjerde]

===================================
Updating "ComfyUI"
===================================
Already up to date.
===================================
Updating "ComfyUI-Manager"
===================================
Already up to date.
[INFO] setup plugin alembic.autogenerate.schemas
[INFO] setup plugin alembic.autogenerate.tables
[INFO] setup plugin alembic.autogenerate.types
[INFO] setup plugin alembic.autogenerate.constraints
[INFO] setup plugin alembic.autogenerate.defaults
[INFO] setup plugin alembic.autogenerate.comments
[INFO] Adding extra search path checkpoints I:\checkpoint-safetensors
[INFO] Adding extra search path configs F:\AI\Automatic1111\stable-diffusion-webui\models\Stable-diffusion
[INFO] Adding extra search path vae F:\AI\Automatic1111\stable-diffusion-webui\models\VAE
[INFO] Adding extra search path loras I:\lora-models
[INFO] Adding extra search path loras F:\AI\Automatic1111\stable-diffusion-webui\models\LyCORIS
[INFO] Adding extra search path upscale_models F:\AI\Automatic1111\stable-diffusion-webui\models\ESRGAN
[INFO] Adding extra search path upscale_models F:\AI\Automatic1111\stable-diffusion-webui\models\RealESRGAN
[INFO] Adding extra search path upscale_models F:\AI\Automatic1111\stable-diffusion-webui\models\SwinIR
[INFO] Adding extra search path embeddings F:\AI\Automatic1111\stable-diffusion-webui\embeddings
[INFO] Adding extra search path hypernetworks F:\AI\Automatic1111\stable-diffusion-webui\models\hypernetworks
[INFO] Adding extra search path controlnet F:\AI\Automatic1111\stable-diffusion-webui\models\ControlNet
E:\AI\ComfyUI\.venv-pt210\Lib\site-packages\torch\cuda\__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
  import pynvml  # type: ignore[import]
[INFO] Found comfy_kitchen backend cuda: {'available': True, 'disabled': False, 'unavailable_reason': None, 'capabilities': ['apply_rope', 'apply_rope1', 'apply_rope_split_half', 'apply_rope_split_half1', 'dequantize_nvfp4', 'dequantize_per_tensor_fp8', 'gemv_awq_w4a16', 'quantize_mxfp8', 'quantize_nvfp4', 'quantize_per_tensor_fp8', 'quantize_svdquant_w4a4', 'scaled_mm_nvfp4', 'scaled_mm_svdquant_w4a4', 'stochastic_rounding_fp8']}
[INFO] Found comfy_kitchen backend triton: {'available': True, 'disabled': True, 'unavailable_reason': None, 'capabilities': ['apply_rope', 'apply_rope1', 'apply_rope_split_half', 'apply_rope_split_half1', 'dequantize_nvfp4', 'dequantize_per_tensor_fp8', 'quantize_mxfp8', 'quantize_nvfp4', 'quantize_per_tensor_fp8']}
[INFO] Found comfy_kitchen backend eager: {'available': True, 'disabled': False, 'unavailable_reason': None, 'capabilities': ['apply_rope', 'apply_rope1', 'apply_rope_split_half', 'apply_rope_split_half1', 'dequantize_mxfp8', 'dequantize_nvfp4', 'dequantize_per_tensor_fp8', 'gemv_awq_w4a16', 'quantize_mxfp8', 'quantize_nvfp4', 'quantize_per_tensor_fp8', 'quantize_svdquant_w4a4', 'scaled_mm_mxfp8', 'scaled_mm_nvfp4', 'scaled_mm_svdquant_w4a4', 'stochastic_rounding_fp8']}
[INFO] Checkpoint files will always be loaded safely.
[INFO] Total VRAM 24576 MB, total RAM 65349 MB
[INFO] pytorch version: 2.10.0+cu130
[INFO] xformers version: 0.0.35
[INFO] Set vram state to: NORMAL_VRAM
[INFO] Device: cuda:0 NVIDIA GeForce RTX 3090 : cudaMallocAsync
[INFO] Using async weight offloading with 2 streams
[INFO] Enabled pinned memory 26139.0
[INFO] working around nvidia conv3d memory bug.
[INFO] Using xformers attention
E:\AI\ComfyUI\.venv-pt210\Lib\site-packages\requests\__init__.py:113: RequestsDependencyWarning: urllib3 (2.19.907) or chardet (7.2.0)/charset_normalizer (3.4.6) doesn't match a supported version!
  warnings.warn(
aimdo: src-win/cuda-detour.c:38:INFO:aimdo_setup_hooks: installing 6 hooks
aimdo: src-win/cuda-detour.c:28:DEBUG:install_hook_entries: hooks successfully installed
aimdo: src-win/shmem-detect.c:80:INFO:comfy-aimdo WDDM adapter match: NVIDIA GeForce RTX 3090 runtime_luid=00000000:00012204 dxgi_luid=00000000:00012204
aimdo: src/control.c:241:INFO:comfy-aimdo inited for GPU: NVIDIA GeForce RTX 3090 (VRAM: 24575 MB)
[INFO] DynamicVRAM support detected and enabled
[INFO] Python version: 3.13.5 (tags/v3.13.5:6cb20a2, Jun 11 2025, 16:15:46) [MSC v.1943 64 bit (AMD64)]
[INFO] ComfyUI version: 0.24.0
[INFO] comfy-aimdo version: 0.4.8
[INFO] comfy-kitchen version: 0.2.10
[INFO] comfyui-frontend-package version: 1.44.19
[INFO] comfyui-workflow-templates version: 0.9.94
[INFO] comfyui-embedded-docs version: 0.5.2
[INFO] comfy-kitchen version: 0.2.10
[INFO] comfy-aimdo version: 0.4.8
[INFO] [Prompt Server] web root: E:\AI\ComfyUI\.venv-pt210\Lib\site-packages\comfyui_frontend_package\static
[INFO] Asset seeder disabled
[INFO] Skipping loading of custom nodes
[INFO] Context impl SQLiteImpl.
[INFO] Will assume non-transactional DDL.
[INFO] Using RAM pressure cache.
[INFO] Starting server

[INFO] To see the GUI go to: http://127.0.0.1:8188

[RuneGjerde]

hopefully not related to the custom file path for models or Automatic1111 import ;-) that would be my fault if so

(but i guess its still supported in comfy)

[rattus128]

You have a line from ComfyUI-MultiGPU, but you only probe one GPU. Can you try with Multi-GPU custom node pack disabled or --disable-all-custom-nodes?

Note that its not always enough to just not use the nodes, as some custom node packs do core code edits on startup.

[RuneGjerde]

You have a line from ComfyUI-MultiGPU, but you only probe one GPU. Can you try with Multi-GPU custom node pack disabled or --disable-all-custom-nodes?

Note that its not always enough to just not use the nodes, as some custom node packs do core code edits on startup.

Already did the flag, but will try remove that node, its not something i use (was back in the days trying to fake some "virtual vram" before comfy now does all the magic.. hehe)

[RuneGjerde]

ComfyUI-MultiGPU removed, --disable-all-custom-nodes

But same error

======================
Already up to date.
[INFO] setup plugin alembic.autogenerate.schemas
[INFO] setup plugin alembic.autogenerate.tables
[INFO] setup plugin alembic.autogenerate.types
[INFO] setup plugin alembic.autogenerate.constraints
[INFO] setup plugin alembic.autogenerate.defaults
[INFO] setup plugin alembic.autogenerate.comments
[INFO] Adding extra search path checkpoints I:\checkpoint-safetensors
[INFO] Adding extra search path configs F:\AI\Automatic1111\stable-diffusion-webui\models\Stable-diffusion
[INFO] Adding extra search path vae F:\AI\Automatic1111\stable-diffusion-webui\models\VAE
[INFO] Adding extra search path loras I:\lora-models
[INFO] Adding extra search path loras F:\AI\Automatic1111\stable-diffusion-webui\models\LyCORIS
[INFO] Adding extra search path upscale_models F:\AI\Automatic1111\stable-diffusion-webui\models\ESRGAN
[INFO] Adding extra search path upscale_models F:\AI\Automatic1111\stable-diffusion-webui\models\RealESRGAN
[INFO] Adding extra search path upscale_models F:\AI\Automatic1111\stable-diffusion-webui\models\SwinIR
[INFO] Adding extra search path embeddings F:\AI\Automatic1111\stable-diffusion-webui\embeddings
[INFO] Adding extra search path hypernetworks F:\AI\Automatic1111\stable-diffusion-webui\models\hypernetworks
[INFO] Adding extra search path controlnet F:\AI\Automatic1111\stable-diffusion-webui\models\ControlNet
E:\AI\ComfyUI\.venv-pt210\Lib\site-packages\torch\cuda\__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
  import pynvml  # type: ignore[import]
[INFO] Found comfy_kitchen backend triton: {'available': True, 'disabled': True, 'unavailable_reason': None, 'capabilities': ['apply_rope', 'apply_rope1', 'apply_rope_split_half', 'apply_rope_split_half1', 'dequantize_nvfp4', 'dequantize_per_tensor_fp8', 'quantize_mxfp8', 'quantize_nvfp4', 'quantize_per_tensor_fp8']}
[INFO] Found comfy_kitchen backend eager: {'available': True, 'disabled': False, 'unavailable_reason': None, 'capabilities': ['apply_rope', 'apply_rope1', 'apply_rope_split_half', 'apply_rope_split_half1', 'dequantize_mxfp8', 'dequantize_nvfp4', 'dequantize_per_tensor_fp8', 'gemv_awq_w4a16', 'quantize_mxfp8', 'quantize_nvfp4', 'quantize_per_tensor_fp8', 'quantize_svdquant_w4a4', 'scaled_mm_mxfp8', 'scaled_mm_nvfp4', 'scaled_mm_svdquant_w4a4', 'stochastic_rounding_fp8']}
[INFO] Found comfy_kitchen backend cuda: {'available': True, 'disabled': False, 'unavailable_reason': None, 'capabilities': ['apply_rope', 'apply_rope1', 'apply_rope_split_half', 'apply_rope_split_half1', 'dequantize_nvfp4', 'dequantize_per_tensor_fp8', 'gemv_awq_w4a16', 'quantize_mxfp8', 'quantize_nvfp4', 'quantize_per_tensor_fp8', 'quantize_svdquant_w4a4', 'scaled_mm_nvfp4', 'scaled_mm_svdquant_w4a4', 'stochastic_rounding_fp8']}
[INFO] Checkpoint files will always be loaded safely.
[INFO] Total VRAM 24576 MB, total RAM 65349 MB
[INFO] pytorch version: 2.10.0+cu130
[INFO] xformers version: 0.0.35
[INFO] Set vram state to: NORMAL_VRAM
[INFO] Device: cuda:0 NVIDIA GeForce RTX 3090 : cudaMallocAsync
[INFO] Using async weight offloading with 2 streams
[INFO] Enabled pinned memory 26139.0
[INFO] working around nvidia conv3d memory bug.
[INFO] Using xformers attention
E:\AI\ComfyUI\.venv-pt210\Lib\site-packages\requests\__init__.py:113: RequestsDependencyWarning: urllib3 (2.19.907) or chardet (7.2.0)/charset_normalizer (3.4.6) doesn't match a supported version!
  warnings.warn(
aimdo: src-win/cuda-detour.c:38:INFO:aimdo_setup_hooks: installing 6 hooks
aimdo: src-win/cuda-detour.c:28:DEBUG:install_hook_entries: hooks successfully installed
aimdo: src-win/shmem-detect.c:80:INFO:comfy-aimdo WDDM adapter match: NVIDIA GeForce RTX 3090 runtime_luid=00000000:00012204 dxgi_luid=00000000:00012204
aimdo: src/control.c:241:INFO:comfy-aimdo inited for GPU: NVIDIA GeForce RTX 3090 (VRAM: 24575 MB)
[INFO] DynamicVRAM support detected and enabled
[INFO] Python version: 3.13.5 (tags/v3.13.5:6cb20a2, Jun 11 2025, 16:15:46) [MSC v.1943 64 bit (AMD64)]
[INFO] ComfyUI version: 0.24.0
[INFO] comfy-aimdo version: 0.4.8
[INFO] comfy-kitchen version: 0.2.10
[INFO] comfyui-frontend-package version: 1.44.19
[INFO] comfyui-workflow-templates version: 0.9.94
[INFO] comfyui-embedded-docs version: 0.5.2
[INFO] comfy-kitchen version: 0.2.10
[INFO] comfy-aimdo version: 0.4.8
[INFO] [Prompt Server] web root: E:\AI\ComfyUI\.venv-pt210\Lib\site-packages\comfyui_frontend_package\static
[INFO] Asset seeder disabled
[INFO] Skipping loading of custom nodes
[INFO] Context impl SQLiteImpl.
[INFO] Will assume non-transactional DDL.
[INFO] Using RAM pressure cache.
[INFO] Starting server

[INFO] To see the GUI go to: http://127.0.0.1:8188
[INFO] got prompt
[INFO] Using xformers attention in VAE
[INFO] Using xformers attention in VAE
[INFO] VAE load device: cuda:0, offload device: cpu, dtype: torch.bfloat16
[INFO] Found quantization metadata version 1
[INFO] Detected mixed precision quantization
[INFO] Using mixed precision operations
[INFO] Native ops:  , emulated ops: mxfp8, float8_e4m3fn, float8_e5m2, nvfp4
[INFO] model weight dtype torch.bfloat16, manual cast: torch.bfloat16
[INFO] model_type FLOW
[INFO] Found quantization metadata version 1
[INFO] Using MixedPrecisionOps for text encoder
[INFO] CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cpu, dtype: torch.float16
[INFO] Requested to load Ideogram4TEModel_
[INFO] Model Ideogram4TEModel_ prepared for dynamic VRAM loading. 7811MB Staged. 0 patches attached. Force pre-loaded 144 weights: 594 KB.
[ERROR] !!! Exception during processing !!! hostbuf_file_reader_read failed
[ERROR] Traceback (most recent call last):
  File "E:\AI\ComfyUI\execution.py", line 536, in execute
    output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
                                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\AI\ComfyUI\execution.py", line 336, in get_output_data
    return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\AI\ComfyUI\execution.py", line 310, in _async_map_node_over_list
    await process_inputs(input_dict, i)
  File "E:\AI\ComfyUI\execution.py", line 298, in process_inputs
    result = f(**inputs)
  File "E:\AI\ComfyUI\nodes.py", line 79, in encode
    return (clip.encode_from_tokens_scheduled(tokens), )
            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^
  File "E:\AI\ComfyUI\comfy\sd.py", line 326, in encode_from_tokens_scheduled
    pooled_dict = self.encode_from_tokens(tokens, return_pooled=return_pooled, return_dict=True)
  File "E:\AI\ComfyUI\comfy\sd.py", line 395, in encode_from_tokens
    o = self.cond_stage_model.encode_token_weights(tokens)
  File "E:\AI\ComfyUI\comfy\text_encoders\ideogram4.py", line 62, in encode_token_weights
    out, pooled, extra = super().encode_token_weights(token_weight_pairs)
                         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "E:\AI\ComfyUI\comfy\sd1_clip.py", line 737, in encode_token_weights
    out = getattr(self, self.clip).encode_token_weights(token_weight_pairs)
  File "E:\AI\ComfyUI\comfy\sd1_clip.py", line 45, in encode_token_weights
    o = self.encode(to_encode)
  File "E:\AI\ComfyUI\comfy\sd1_clip.py", line 306, in encode
    return self(tokens)
  File "E:\AI\ComfyUI\.venv-pt210\Lib\site-packages\torch\nn\modules\module.py", line 1776, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "E:\AI\ComfyUI\.venv-pt210\Lib\site-packages\torch\nn\modules\module.py", line 1787, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\AI\ComfyUI\comfy\sd1_clip.py", line 279, in forward
    outputs = self.transformer(None, attention_mask_model, embeds=embeds, num_tokens=num_tokens, intermediate_output=intermediate_output, final_layer_norm_intermediate=self.layer_norm_hidden_state, dtype=torch.float32, embeds_info=embeds_info)
  File "E:\AI\ComfyUI\.venv-pt210\Lib\site-packages\torch\nn\modules\module.py", line 1776, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "E:\AI\ComfyUI\.venv-pt210\Lib\site-packages\torch\nn\modules\module.py", line 1787, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\AI\ComfyUI\comfy\text_encoders\llama.py", line 834, in forward
    return self.model(input_ids, *args, **kwargs)
           ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\AI\ComfyUI\.venv-pt210\Lib\site-packages\torch\nn\modules\module.py", line 1776, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "E:\AI\ComfyUI\.venv-pt210\Lib\site-packages\torch\nn\modules\module.py", line 1787, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\AI\ComfyUI\comfy\text_encoders\llama.py", line 759, in forward
    x, current_kv = layer(
                    ~~~~~^
        x=x,
        ^^^^
    ...<3 lines>...
        past_key_value=past_kv,
        ^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "E:\AI\ComfyUI\.venv-pt210\Lib\site-packages\torch\nn\modules\module.py", line 1776, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "E:\AI\ComfyUI\.venv-pt210\Lib\site-packages\torch\nn\modules\module.py", line 1787, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\AI\ComfyUI\comfy\text_encoders\llama.py", line 580, in forward
    x, present_key_value = self.self_attn(
                           ~~~~~~~~~~~~~~^
        hidden_states=x,
        ^^^^^^^^^^^^^^^^
    ...<3 lines>...
        past_key_value=past_key_value,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "E:\AI\ComfyUI\.venv-pt210\Lib\site-packages\torch\nn\modules\module.py", line 1776, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "E:\AI\ComfyUI\.venv-pt210\Lib\site-packages\torch\nn\modules\module.py", line 1787, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\AI\ComfyUI\comfy\text_encoders\llama.py", line 501, in forward
    xq = self.q_proj(hidden_states)
  File "E:\AI\ComfyUI\.venv-pt210\Lib\site-packages\torch\nn\modules\module.py", line 1776, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "E:\AI\ComfyUI\.venv-pt210\Lib\site-packages\torch\nn\modules\module.py", line 1787, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\AI\ComfyUI\comfy\ops.py", line 1244, in forward
    output = self.forward_comfy_cast_weights(input, compute_dtype, want_requant=isinstance(input, QuantizedTensor))
  File "E:\AI\ComfyUI\comfy\ops.py", line 1187, in forward_comfy_cast_weights
    weight, bias, offload_stream = cast_bias_weight(self, input, offloadable=True, compute_dtype=compute_dtype, want_requant=want_requant)
                                   ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\AI\ComfyUI\comfy\ops.py", line 321, in cast_bias_weight
    offload_stream = cast_modules_with_vbar([s], dtype, device, bias_dtype, non_blocking)
  File "E:\AI\ComfyUI\comfy\ops.py", line 188, in cast_modules_with_vbar
    handle_pin(s, pin, xfer_source, xfer_dest, size=dest_size)
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\AI\ComfyUI\comfy\ops.py", line 186, in handle_pin
    cast_maybe_lowvram_patch(source, pin, offload_stream, xfer_dest2=dest)
    ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\AI\ComfyUI\comfy\ops.py", line 177, in cast_maybe_lowvram_patch
    comfy.model_management.cast_to_gathered(xfer_source, xfer_dest, non_blocking=non_blocking, stream=stream, r2=xfer_dest2)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\AI\ComfyUI\comfy\model_management.py", line 1439, in cast_to_gathered
    if comfy.memory_management.read_tensor_file_slice_into(tensor, dest_view, stream=stream, destination2=dest2_view):
       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\AI\ComfyUI\comfy\memory_management.py", line 21, in read_tensor_file_slice_into
    if not read_tensor_file_slice_into(tensor._qdata,
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
                                       destination._qdata if destination is not None else None, stream=stream,
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                       destination2=(destination2._qdata if destination2 is not None else None)):
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\AI\ComfyUI\comfy\memory_management.py", line 59, in read_tensor_file_slice_into
    comfy_aimdo.host_buffer.read_file_to_device(file_obj, info.offset, info.size,
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                                stream_ptr, destination2.data_ptr(),
                                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                                destination2.device.index,
                                                ^^^^^^^^^^^^^^^^^^^^^^^^^^
                                                mark_cold=False)
                                                ^^^^^^^^^^^^^^^^
  File "E:\AI\ComfyUI\.venv-pt210\Lib\site-packages\comfy_aimdo\host_buffer.py", line 71, in read_file_to_device
    raise RuntimeError("hostbuf_file_reader_read failed")
RuntimeError: hostbuf_file_reader_read failed

[INFO] Prompt executed in 14.90 seconds

[RuneGjerde]

If its only my end, i could try reinstall fresh. If the error above is not something general

[rattus128]

if you start comfy in --verbose mode do you get any error messages from aimdo leading up to the crash?

[RuneGjerde]

was a couple of updates at comfy when i started up now. And no errors (-verbose debug is on).
Let me just turn off -verbose as well just to make sure. But seems ok again

[RuneGjerde]

Seems to work well again. I didnt take note of what the updates were just now, but at least all seems to be working ;-)

Edit: turned off disable custom node as well, all is good ;-)

(would even say its better than its ever been, the model loading is blazing fast)

[RuneGjerde]

btw, a huge thank you for all the optimization you have done.

I used to rely on gguf and hacks to be able to run often, now i feel i can run anything in comfy ;-)