vGPU windows can't use comfy-aimdo

[AllentDan]

Hi, thanks for the great job. We've recently found that the latest ComfyUI can't work in vGPU windows!!! It turns out that there is cuMemAddressReserve at https://github.com/Comfy-Org/comfy-aimdo/blob/v0.2.10/src/model-vbar.c#L199 . It means that all the cloud providers can't provide ComfyUI for windows Users. If possible, please consider using other cuda api to achieve the same result. cc @metebalci

[rattus128]

What happens in your case?

After a bit of searching, im not reading any specific incompatibility regarding cuMemAddressReserve+vGPU.

[AllentDan]

Well，I tried latest ComfyUI in both windows and ubuntu vGPU. In Windows, it was found that cuMemAddressReserve . The test cuda demo is below:

#include <stdio.h>
#include <cuda.h>

#define TEST_SIZE (512ULL * 1024 * 1024) // 512 MB

int main() {
    CUresult res;
    CUdeviceptr d_ptr = 0;

    // 1. 初始化 CUDA Driver API
    res = cuInit(0);
    if (res != CUDA_SUCCESS) {
        printf("cuInit failed: %d\n", res);
        return 1;
    }

    size_t size = TEST_SIZE;
    size_t alignment = 0;  // 0 表示驱动默认对齐
    CUdeviceptr addr = 0;
    unsigned long long addr_out = 0;

    // 2. 调用 cuMemAddressReserve
    res = cuMemAddressReserve(&addr, size, alignment, 0, 0);
    if (res == CUDA_SUCCESS) {
        printf("cuMemAddressReserve succeeded! Virtual address: 0x%llx (%llu bytes)\n", (unsigned long long)addr, (unsigned long long)size);
        // 记得释放虚拟空间  
        cuMemAddressFree(addr, size);
        return 0;
    } else {
        const char* errStr = NULL;
        cuGetErrorString(res, &errStr);
        printf("cuMemAddressReserve failed! Error code: %d, msg: %s\n", res, errStr ? errStr : "unknown");
        return 1;
    }
}

In Native Tools Command Prompt console, I used nvcc -o test_cumemaddressreserve.exe test_cumemaddressreserve.c -lcuda to compile the above codes. The execution result is below:

C: \Users \admin \Desktop>test_cumemaddressreserve. exe
cuMemAddressReserve failed! Error code: 801, msg: operation not supported

[AllentDan]

@rattus128 I consulted NVIDIA group, it was said that Windows vGPU can't open UVM.

[Aanerud]

We have the same problem, and needs a way to ensure its disabled.

[Aanerud]

Running into this on an NVIDIA L40S-48Q vGPU (SR-IOV / mediated device) on Proxmox VE. Plenty of VRAM available (47.7GB free), but the C-level call fails immediately:

aimdo: src/model-vbar.c:200:ERROR:Could not reserve Virtual Address space for VBAR

The root cause is that vGPU guests have a limited virtual address space window exposed by the hypervisor. vbar_allocate() tries to reserve model_size * 10 upfront as a contiguous VA reservation, which exceeds what the vGPU guest allows regardless of how much VRAM is actually free. This is not an OOM condition — it’s a VA space limitation specific to virtualised GPU environments.

Workaround: Wrapping the ModelVBAR instantiation in model_patcher.py with a try/except MemoryError and falling back to None is sufficient, since the rest of the code already handles vbar is None correctly (e.g. loaded_size() returns 0, etc.):

try:
    vbar = comfy_aimdo.model_vbar.ModelVBAR(self.model_size() * 10, self.load_device.index)
except MemoryError:
    logging.warning("VBAR allocation failed (vGPU environment), falling back to standard model management")
    vbar = None

With this patch ComfyUI runs fine on the vGPU — it just falls back to standard model management without VBAR optimisations, which is acceptable.

Would be great to have this handled gracefully upstream so vGPU users don’t need to patch model_patcher.py manually after every git pull.

[rattus128]

Running into this on an NVIDIA L40S-48Q vGPU (SR-IOV / mediated device) on Proxmox VE. Plenty of VRAM available (47.7GB free), but the C-level call fails immediately:

aimdo: src/model-vbar.c:200:ERROR:Could not reserve Virtual Address space for VBAR

The root cause is that vGPU guests have a limited virtual address space window exposed by the hypervisor. vbar_allocate() tries to reserve model_size * 10 upfront as a contiguous VA reservation, which exceeds what the vGPU guest allows regardless of how much VRAM is actually free. This is not an OOM condition — it’s a VA space limitation specific to virtualised GPU environments.

Workaround: Wrapping the ModelVBAR instantiation in model_patcher.py with a try/except MemoryError and falling back to None is sufficient, since the rest of the code already handles vbar is None correctly (e.g. loaded_size() returns 0, etc.):

try:
vbar = comfy_aimdo.model_vbar.ModelVBAR(self.model_size() * 10, self.load_device.index)
except MemoryError:
logging.warning("VBAR allocation failed (vGPU environment), falling back to standard model management")
vbar = None

With this patch ComfyUI runs fine on the vGPU — it just falls back to standard model management without VBAR optimisations, which is acceptable.

Would be great to have this handled gracefully upstream so vGPU users don’t need to patch model_patcher.py manually after every git pull.

Thanks for the deep dive. Aimdo does have a clamp on the address space reservations to the total VRAM of the GPU:

https://github.com/Comfy-Org/comfy-aimdo/blob/master/src/model-vbar.c#L189

If you are still tripping up it means you must have a lot of models creating VBARs or the virtual address quota is extremely limited. Do you have much in the way of numbers?

On your setup if you just make 128GB VBARs in an infinate loop (should clamp to your VRAM with ^^), keep the refs and count them how far do you get?

We should get a sense of if your case is supportable with Aimdo and just look to detect and wholesale switch off depending on what the numbers are and how workable they are.

Core comfy has options and can free vbars more aggressively that it does to conserve VA space too.

[Aanerud]

Thanks for the reply — based on your suggestions I did some more digging, this is what I found, looks like it may be an initialization order issue rather than VA space limits.

Setup: NVIDIA L40S-48Q (SR-IOV vGPU slice), driver 570.133.20, CUDA 12.8, Proxmox VE, Ubuntu 24.04.

When I called init_device(0) directly I got this:

aimdo: src/plat.h:142:DEBUG:CUDA API FAILED : cuDeviceGet(&dev, cuda_device_id) : initialization error
init_device(0) returned: False

So init() succeeds and the .so loads fine, but init_device() returns False — which then leaves everything in a broken state downstream when vbar_allocate() gets called.

My best guess is that on vGPU the CUDA Driver API isn't available until PyTorch has first initialized the CUDA context. If aimdo's init_device() fires before that happens it fails silently and never recovers. Doesn't seem to be a CUDA version thing — we're on 12.8 / driver 570 which is current.

The workaround we implemented yesterday does the trick — a try/except MemoryError around the ModelVBAR call in model_patcher.py which falls back to standard model management cleanly — so we'll keep it as is for now.

Happy to test anything on the vGPU setup if useful 👍

[rattus128]

Thanks for the reply — based on your suggestions I did some more digging, this is what I found, looks like it may be an initialization order issue rather than VA space limits.

Setup: NVIDIA L40S-48Q (SR-IOV vGPU slice), driver 570.133.20, CUDA 12.8, Proxmox VE, Ubuntu 24.04.

When I called init_device(0) directly I got this:

aimdo: src/plat.h:142:DEBUG:CUDA API FAILED : cuDeviceGet(&dev, cuda_device_id) : initialization error
init_device(0) returned: False

So init() succeeds and the .so loads fine, but init_device() returns False — which then leaves everything in a broken state downstream when vbar_allocate() gets called.

If init_device returns false, comfy should completely disable the aimdo system:

    if (not args.enable_dynamic_vram) and (comfy.model_management.torch_version_numeric < (2, 8)):
        logging.warning("Unsupported Pytorch detected. DynamicVRAM support requires Pytorch version 2.8 or later. Falling back to legacy ModelPatcher. VRAM estimates may be unreliable especially on Windows")
    elif comfy_aimdo.control.init_device(comfy.model_management.get_torch_device().index):
        if args.verbose == 'DEBUG':
            comfy_aimdo.control.set_log_debug()
        elif args.verbose == 'CRITICAL':
            comfy_aimdo.control.set_log_critical()
        elif args.verbose == 'ERROR':
            comfy_aimdo.control.set_log_error()
        elif args.verbose == 'WARNING':
            comfy_aimdo.control.set_log_warning()
        else: #INFO
            comfy_aimdo.control.set_log_info()

        comfy.model_patcher.CoreModelPatcher = comfy.model_patcher.ModelPatcherDynamic
        comfy.memory_management.aimdo_enabled = True
        logging.info("DynamicVRAM support detected and enabled")
    else:
        logging.warning("No working comfy-aimdo install detected. DynamicVRAM support disabled. Falling back to legacy ModelPatcher. VRAM estimates may be unreliable especially on Windows")

You should be seeing that message at the bottom about "No working ..."

My best guess is that on vGPU the CUDA Driver API isn't available until PyTorch has first initialized the CUDA context. If aimdo's init_device() fires before that happens it fails silently and never recovers. Doesn't seem to be a CUDA version thing — we're on 12.8 / driver 570 which is current.

init_device() is already called after torch import for this reason. The sequence is:

init() # Aimdo need to exist before torch for the hook installation
import torch # Aimdo will catch and hook torch cuda load
init_device() # Aimdo sets it self up on the device etc with cuda fully accessible

The workaround we implemented yesterday does the trick — a try/except MemoryError around the ModelVBAR call in model_patcher.py which falls back to standard model management cleanly — so we'll keep it as is for now.

Yeah im considering a universal fallback to handle your case. The problem is its hard to get it in the right place. There are corner cases with that approach as you mix DynamicVRAM file loader with the legacy ModelPatcher when you do it that way so you might have issues with some rarer models. You are probably ok with all the big name models.

Happy to test anything on the vGPU setup if useful 👍

[rattus128]

I have build a new wheel with some features that should help.

@Aanerud I have added a cuInit() which is the usual prescription for the bug you face. If we are lucky it switches everything on for you.

@AllentDan I have added a probe of the features aimdo need to init() so if they just outright fail, comfyUI will fall out of the dynamic VRAM mode completely.

Implementing Dynamic VRAM without the VMM features is possible but its not an overnighter and quite difficult. It wont reach the same memory performance as it will be quite a bit more complex and can fragment memory unless we fully implement caching allocators. Im happy for it to be a separate implementaiton of Aimdo, but we will keep with the VMM APIs when available so its very much a new feature level project.

Please try:

https://github.com/Comfy-Org/comfy-aimdo/actions/runs/23231132334

[AllentDan]

Thanks for your reply @rattus128 . I will try the new wheel ASAP.

[Aanerud]

Tested the new wheel — good progress! Here's what we're seeing:

What's working:
The async allocator is fully active and handling memory operations during inference — seeing lots of pyt-cu-plug-alloc-async activity in the logs. Prompt completed successfully (185 seconds, no crash).

What's still failing:
cuMemAddressReserve returns operation not supported on the vGPU, so VBAR/Dynamic VRAM still can't initialise:

aimdo: src/plat.h:142:DEBUG:CUDA API FAILED : cuMemAddressReserve(&vaddr, test_size, 0, 0, 0) : operation not supported
No working comfy-aimdo install detected. DynamicVRAM support disabled. Falling back to legacy ModelPatcher.

Summary:
The graceful fallback is working perfectly — ComfyUI no longer crashes, it just runs without Dynamic VRAM. The cuMemAddressReserve API is simply not exposed by the vGPU driver to guest VMs, so VBAR will likely never work in this environment regardless of what we do on the software side. That's a hypervisor-level restriction.

For our use case the fallback is totally fine — running smoothly on an L40S-48Q vGPU slice. Thanks for the quick turnaround on this! 👍