[Bug]: RuntimeError: UVA is not available

[Malone-AI]

Your current environment

Details

On wsl2 ubuntu22.04

Collecting environment information...
uv is set
==============================
        System Info
==============================
OS                           : Ubuntu 22.04.5 LTS (x86_64)
GCC version                  : (Ubuntu 11.4.0-1ubuntu1~22.04.3) 11.4.0
Clang version                : 14.0.0-1ubuntu1.1
CMake version                : version 3.22.1
Libc version                 : glibc-2.35

==============================
       PyTorch Info
==============================
PyTorch version              : 2.11.0+cu130
Is debug build               : False
CUDA used to build PyTorch   : 13.0
ROCM used to build PyTorch   : N/A
XPU used to build PyTorch    : N/A

==============================
      Python Environment
==============================
Python version               : 3.12.11 (main, Jun 12 2025, 12:40:51) [Clang 20.1.4 ] (64-bit runtime)
Python platform              : Linux-5.15.167.4-microsoft-standard-WSL2-x86_64-with-glibc2.35
    
==============================
       CUDA / GPU Info
==============================
Is CUDA available            : True
CUDA runtime version         : 12.8.61
CUDA_MODULE_LOADING set to   : 
GPU models and configuration : GPU 0: NVIDIA GeForce RTX 4060 Laptop GPU
Nvidia driver version        : 595.97
cuDNN version                : Could not collect
HIP runtime version          : N/A
MIOpen runtime version       : N/A
Is XNNPACK available         : True

==============================
          CPU Info
==============================
Architecture:                         x86_64
CPU op-mode(s):                       32-bit, 64-bit
Address sizes:                        39 bits physical, 48 bits virtual
Byte Order:                           Little Endian
CPU(s):                               10
On-line CPU(s) list:                  0-9
Vendor ID:                            GenuineIntel
Model name:                           13th Gen Intel(R) Core(TM) i7-13650HX
CPU family:                           6
Model:                                183
Thread(s) per core:                   2
Core(s) per socket:                   5
Socket(s):                            1
Stepping:                             1
BogoMIPS:                             5606.39
Flags:                                fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology tsc_reliable nonstop_tsc cpuid pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves avx_vnni umip waitpkg gfni vaes vpclmulqdq rdpid movdiri movdir64b fsrm md_clear serialize flush_l1d arch_capabilities
Virtualization:                       VT-x
Hypervisor vendor:                    Microsoft
Virtualization type:                  full
L1d cache:                            240 KiB (5 instances)
L1i cache:                            160 KiB (5 instances)
L2 cache:                             6.3 MiB (5 instances)
L3 cache:                             24 MiB (1 instance)
Vulnerability Gather data sampling:   Not affected
Vulnerability Itlb multihit:          Not affected
Vulnerability L1tf:                   Not affected
Vulnerability Mds:                    Not affected
Vulnerability Meltdown:               Not affected
Vulnerability Mmio stale data:        Not affected
Vulnerability Reg file data sampling: Mitigation; Clear Register File
Vulnerability Retbleed:               Mitigation; Enhanced IBRS
Vulnerability Spec rstack overflow:   Not affected
Vulnerability Spec store bypass:      Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Vulnerability Spectre v1:             Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:             Mitigation; Enhanced / Automatic IBRS; IBPB conditional; RSB filling; PBRSB-eIBRS SW sequence; BHI BHI_DIS_S
Vulnerability Srbds:                  Not affected
Vulnerability Tsx async abort:        Not affected

==============================
Versions of relevant libraries
==============================
[pip3] flashinfer-python==0.6.11.post2
[pip3] numpy==2.2.6
[pip3] nvidia-cublas==13.1.0.3
[pip3] nvidia-cuda-cccl==13.2.75
[pip3] nvidia-cuda-crt==13.2.78
[pip3] nvidia-cuda-cupti==13.0.85
[pip3] nvidia-cuda-nvcc==13.2.78
[pip3] nvidia-cuda-nvrtc==13.0.88
[pip3] nvidia-cuda-runtime==13.0.96
[pip3] nvidia-cudnn-cu13==9.19.0.56
[pip3] nvidia-cudnn-frontend==1.18.0
[pip3] nvidia-cufft==12.0.0.61
[pip3] nvidia-cufile==1.15.1.6
[pip3] nvidia-curand==10.4.0.35
[pip3] nvidia-cusolver==12.0.4.66
[pip3] nvidia-cusparse==12.6.3.3
[pip3] nvidia-cusparselt-cu13==0.8.0
[pip3] nvidia-cutlass-dsl==4.5.0
[pip3] nvidia-cutlass-dsl-libs-base==4.5.0
[pip3] nvidia-cutlass-dsl-libs-cu13==4.5.0
[pip3] nvidia-ml-py==13.590.48
[pip3] nvidia-nccl-cu13==2.28.9
[pip3] nvidia-nvjitlink==13.0.88
[pip3] nvidia-nvshmem-cu13==3.4.5
[pip3] nvidia-nvtx==13.0.85
[pip3] nvidia-nvvm==13.2.78
[pip3] pyzmq==27.1.0
[pip3] tokenspeed-triton==3.7.10.post20260505
[pip3] torch==2.11.0
[pip3] torch-c-dlpack-ext==0.1.5
[pip3] torchaudio==2.11.0
[pip3] torchvision==0.26.0
[pip3] transformers==4.57.6
[pip3] triton==3.6.0
[conda] Could not collect

==============================
         vLLM Info
==============================
ROCM Version                 : Could not collect
vLLM Version                 : 0.20.2rc1.dev562+g1c78f76c2.d20260521 (git sha: 1c78f76c2, date: 20260521)
vLLM Build Flags:
  CUDA Archs: Not Set; ROCm: Disabled; XPU: Disabled
GPU Topology:
        GPU0    CPU Affinity    NUMA Affinity   GPU NUMA ID
GPU0     X                              N/A

Legend:

  X    = Self
  SYS  = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
  NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
  PHB  = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
  PXB  = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
  PIX  = Connection traversing at most a single PCIe bridge
  NV#  = Connection traversing a bonded set of # NVLinks

==============================
     Environment Variables
==============================
LD_LIBRARY_PATH=/usr/local/cuda-12.8/lib64:/usr/local/cuda-12.8/lib64::/usr/local/lib:/usr/local/lib
CUDA_HOME=/usr/local/cuda-12.8
CUDA_HOME=/usr/local/cuda-12.8
PYTORCH_NVML_BASED_CUDA_CHECK=1
TORCHINDUCTOR_COMPILE_THREADS=1
TORCHINDUCTOR_CACHE_DIR=/tmp/torchinductor_malone

🐛 Describe the bug

running

vllm serve Qwen/Qwen3-0.6B --gpu-memory-utilization 0.8

on wsl2, but got RuntimeError: UVA is not available, the detailed message is

WARNING 05-22 10:02:30 [config.py:70] Support for Transformers v4 is deprecated. The Transformers v4 codepath will become unmaintained in vLLM v0.22.0 and will be removed in vLLM v0.24.0. Please upgrade to Transformers v5: pip install --upgrade transformers
WARNING 05-22 10:02:31 [interface.py:729] Using 'pin_memory=False' as WSL is detected. This may slow down the performance.
(APIServer pid=17649) INFO 05-22 10:02:31 [utils.py:344] 
(APIServer pid=17649) INFO 05-22 10:02:31 [utils.py:344]        █     █     █▄   ▄█
(APIServer pid=17649) INFO 05-22 10:02:31 [utils.py:344]  ▄▄ ▄█ █     █     █ ▀▄▀ █  version 0.20.2rc1.dev562+g1c78f76c2.d20260521
(APIServer pid=17649) INFO 05-22 10:02:31 [utils.py:344]   █▄█▀ █     █     █     █  model   /home/malone/.cache/huggingface/hub/models--Qwen--Qwen3-0.6B/snapshots/c1899de289a04d12100db370d81485cdf75e47ca/
(APIServer pid=17649) INFO 05-22 10:02:31 [utils.py:344]    ▀▀  ▀▀▀▀▀ ▀▀▀▀▀ ▀     ▀
(APIServer pid=17649) INFO 05-22 10:02:31 [utils.py:344] 
(APIServer pid=17649) INFO 05-22 10:02:31 [utils.py:278] non-default args: {'model_tag': '/home/malone/.cache/huggingface/hub/models--Qwen--Qwen3-0.6B/snapshots/c1899de289a04d12100db370d81485cdf75e47ca/', 'model': '/home/malone/.cache/huggingface/hub/models--Qwen--Qwen3-0.6B/snapshots/c1899de289a04d12100db370d81485cdf75e47ca/', 'gpu_memory_utilization': 0.8}
(APIServer pid=17649) INFO 05-22 10:02:31 [model.py:616] Resolved architecture: Qwen3ForCausalLM
(APIServer pid=17649) INFO 05-22 10:02:31 [model.py:1751] Using max model len 40960
(APIServer pid=17649) INFO 05-22 10:02:31 [vllm.py:965] Asynchronous scheduling is enabled.
(APIServer pid=17649) INFO 05-22 10:02:31 [kernel.py:270] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'])
WARNING 05-22 10:02:35 [config.py:70] Support for Transformers v4 is deprecated. The Transformers v4 codepath will become unmaintained in vLLM v0.22.0 and will be removed in vLLM v0.24.0. Please upgrade to Transformers v5: pip install --upgrade transformers
(EngineCore pid=17815) INFO 05-22 10:02:36 [core.py:109] Initializing a V1 LLM engine (v0.20.2rc1.dev562+g1c78f76c2.d20260521) with config: model='/home/malone/.cache/huggingface/hub/models--Qwen--Qwen3-0.6B/snapshots/c1899de289a04d12100db370d81485cdf75e47ca/', speculative_config=None, tokenizer='/home/malone/.cache/huggingface/hub/models--Qwen--Qwen3-0.6B/snapshots/c1899de289a04d12100db370d81485cdf75e47ca/', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=False, dtype=torch.bfloat16, max_seq_len=40960, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, decode_context_parallel_size=1, dcp_comm_backend=ag_rs, disable_custom_all_reduce=False, quantization=None, quantization_config=None, enforce_eager=False, enable_return_routed_experts=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False), seed=0, served_model_name=/home/malone/.cache/huggingface/hub/models--Qwen--Qwen3-0.6B/snapshots/c1899de289a04d12100db370d81485cdf75e47ca/, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'mode': <CompilationMode.VLLM_COMPILE: 3>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['none'], 'ir_enable_torch_wrap': True, 'splitting_ops': ['vllm::unified_attention_with_output', 'vllm::unified_mla_attention_with_output', 'vllm::mamba_mixer2', 'vllm::mamba_mixer', 'vllm::short_conv', 'vllm::linear_attention', 'vllm::plamo2_mamba_mixer', 'vllm::gdn_attention_core', 'vllm::gdn_attention_core_xpu', 'vllm::olmo_hybrid_gdn_full_forward', 'vllm::kda_attention', 'vllm::sparse_attn_indexer', 'vllm::rocm_aiter_sparse_attn_indexer', 'vllm::deepseek_v4_attention', 'vllm::unified_kv_cache_update', 'vllm::unified_mla_kv_cache_update'], 'compile_mm_encoder': False, 'cudagraph_mm_encoder': False, 'encoder_cudagraph_token_budgets': [], 'encoder_cudagraph_max_vision_items_per_batch': 0, 'encoder_cudagraph_max_frames_per_batch': None, 'compile_sizes': [], 'compile_ranges_endpoints': [2048], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'size_asserts': False, 'alignment_asserts': False, 'scalar_asserts': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 1, 'cudagraph_capture_sizes': [1, 2, 4, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 128, 136, 144, 152, 160, 168, 176, 184, 192, 200, 208, 216, 224, 232, 240, 248, 256, 272, 288, 304, 320, 336, 352, 368, 384, 400, 416, 432, 448, 464, 480, 496, 512], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False, 'fuse_rope_kvcache_cat_mla': False, 'fuse_act_padding': False}, 'max_cudagraph_capture_size': 512, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': False, 'static_all_moe_layers': []}, kernel_config=KernelConfig(ir_op_priority=IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native']), enable_flashinfer_autotune=True, moe_backend='auto', linear_backend='auto')
(EngineCore pid=17815) WARNING 05-22 10:02:37 [interface.py:729] Using 'pin_memory=False' as WSL is detected. This may slow down the performance.
(EngineCore pid=17815) INFO 05-22 10:02:37 [parallel_state.py:1416] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://172.31.173.80:37771 backend=nccl
(EngineCore pid=17815) INFO 05-22 10:02:39 [parallel_state.py:1729] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank N/A, EPLB rank N/A
(EngineCore pid=17815) INFO 05-22 10:02:39 [gpu_worker.py:292] Using V2 Model Runner
(EngineCore pid=17815) ERROR 05-22 10:02:39 [core.py:1159] EngineCore failed to start.
(EngineCore pid=17815) ERROR 05-22 10:02:39 [core.py:1159] Traceback (most recent call last):
(EngineCore pid=17815) ERROR 05-22 10:02:39 [core.py:1159]   File "/home/malone/vllm/vllm/v1/engine/core.py", line 1133, in run_engine_core
(EngineCore pid=17815) ERROR 05-22 10:02:39 [core.py:1159]     engine_core = EngineCoreProc(*args, engine_index=dp_rank, **kwargs)
(EngineCore pid=17815) ERROR 05-22 10:02:39 [core.py:1159]                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=17815) ERROR 05-22 10:02:39 [core.py:1159]   File "/home/malone/vllm/vllm/tracing/otel.py", line 178, in sync_wrapper
(EngineCore pid=17815) ERROR 05-22 10:02:39 [core.py:1159]     return func(*args, **kwargs)
(EngineCore pid=17815) ERROR 05-22 10:02:39 [core.py:1159]            ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=17815) ERROR 05-22 10:02:39 [core.py:1159]   File "/home/malone/vllm/vllm/v1/engine/core.py", line 899, in __init__
(EngineCore pid=17815) ERROR 05-22 10:02:39 [core.py:1159]     super().__init__(
(EngineCore pid=17815) ERROR 05-22 10:02:39 [core.py:1159]   File "/home/malone/vllm/vllm/v1/engine/core.py", line 118, in __init__
(EngineCore pid=17815) ERROR 05-22 10:02:39 [core.py:1159]     self.model_executor = executor_class(vllm_config)
(EngineCore pid=17815) ERROR 05-22 10:02:39 [core.py:1159]                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=17815) ERROR 05-22 10:02:39 [core.py:1159]   File "/home/malone/vllm/vllm/tracing/otel.py", line 178, in sync_wrapper
(EngineCore pid=17815) ERROR 05-22 10:02:39 [core.py:1159]     return func(*args, **kwargs)
(EngineCore pid=17815) ERROR 05-22 10:02:39 [core.py:1159]            ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=17815) ERROR 05-22 10:02:39 [core.py:1159]   File "/home/malone/vllm/vllm/v1/executor/abstract.py", line 109, in __init__
(EngineCore pid=17815) ERROR 05-22 10:02:39 [core.py:1159]     self._init_executor()
(EngineCore pid=17815) ERROR 05-22 10:02:39 [core.py:1159]   File "/home/malone/vllm/vllm/v1/executor/uniproc_executor.py", line 60, in _init_executor
(EngineCore pid=17815) ERROR 05-22 10:02:39 [core.py:1159]     self.driver_worker.init_device()
(EngineCore pid=17815) ERROR 05-22 10:02:39 [core.py:1159]   File "/home/malone/vllm/vllm/v1/worker/worker_base.py", line 325, in init_device
(EngineCore pid=17815) ERROR 05-22 10:02:39 [core.py:1159]     self.worker.init_device()  # type: ignore
(EngineCore pid=17815) ERROR 05-22 10:02:39 [core.py:1159]     ^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=17815) ERROR 05-22 10:02:39 [core.py:1159]   File "/home/malone/vllm/vllm/tracing/otel.py", line 178, in sync_wrapper
(EngineCore pid=17815) ERROR 05-22 10:02:39 [core.py:1159]     return func(*args, **kwargs)
(EngineCore pid=17815) ERROR 05-22 10:02:39 [core.py:1159]            ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=17815) ERROR 05-22 10:02:39 [core.py:1159]   File "/home/malone/vllm/vllm/v1/worker/gpu_worker.py", line 322, in init_device
(EngineCore pid=17815) ERROR 05-22 10:02:39 [core.py:1159]     self.model_runner: GPUModelRunner = GPUModelRunnerV2(  # type: ignore
(EngineCore pid=17815) ERROR 05-22 10:02:39 [core.py:1159]                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=17815) ERROR 05-22 10:02:39 [core.py:1159]   File "/home/malone/vllm/vllm/v1/worker/gpu/model_runner.py", line 193, in __init__
(EngineCore pid=17815) ERROR 05-22 10:02:39 [core.py:1159]     self.req_states = RequestState(
(EngineCore pid=17815) ERROR 05-22 10:02:39 [core.py:1159]                       ^^^^^^^^^^^^^
(EngineCore pid=17815) ERROR 05-22 10:02:39 [core.py:1159]   File "/home/malone/vllm/vllm/v1/worker/gpu/states.py", line 33, in __init__
(EngineCore pid=17815) ERROR 05-22 10:02:39 [core.py:1159]     self.all_token_ids = StagedWriteTensor(
(EngineCore pid=17815) ERROR 05-22 10:02:39 [core.py:1159]                          ^^^^^^^^^^^^^^^^^^
(EngineCore pid=17815) ERROR 05-22 10:02:39 [core.py:1159]   File "/home/malone/vllm/vllm/v1/worker/gpu/buffer_utils.py", line 126, in __init__
(EngineCore pid=17815) ERROR 05-22 10:02:39 [core.py:1159]     self._uva_buf = UvaBuffer(size, dtype)
(EngineCore pid=17815) ERROR 05-22 10:02:39 [core.py:1159]                     ^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=17815) ERROR 05-22 10:02:39 [core.py:1159]   File "/home/malone/vllm/vllm/v1/worker/gpu/buffer_utils.py", line 39, in __init__
(EngineCore pid=17815) ERROR 05-22 10:02:39 [core.py:1159]     raise RuntimeError("UVA is not available")
(EngineCore pid=17815) ERROR 05-22 10:02:39 [core.py:1159] RuntimeError: UVA is not available
(EngineCore pid=17815) Process EngineCore:
(EngineCore pid=17815) Traceback (most recent call last):
(EngineCore pid=17815)   File "/home/malone/.local/share/uv/python/cpython-3.12.11-linux-x86_64-gnu/lib/python3.12/multiprocessing/process.py", line 314, in _bootstrap
(EngineCore pid=17815)     self.run()
(EngineCore pid=17815)   File "/home/malone/.local/share/uv/python/cpython-3.12.11-linux-x86_64-gnu/lib/python3.12/multiprocessing/process.py", line 108, in run
(EngineCore pid=17815)     self._target(*self._args, **self._kwargs)
(EngineCore pid=17815)   File "/home/malone/vllm/vllm/v1/engine/core.py", line 1163, in run_engine_core
(EngineCore pid=17815)     raise e
(EngineCore pid=17815)   File "/home/malone/vllm/vllm/v1/engine/core.py", line 1133, in run_engine_core
(EngineCore pid=17815)     engine_core = EngineCoreProc(*args, engine_index=dp_rank, **kwargs)
(EngineCore pid=17815)                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=17815)   File "/home/malone/vllm/vllm/tracing/otel.py", line 178, in sync_wrapper
(EngineCore pid=17815)     return func(*args, **kwargs)
(EngineCore pid=17815)            ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=17815)   File "/home/malone/vllm/vllm/v1/engine/core.py", line 899, in __init__
(EngineCore pid=17815)     super().__init__(
(EngineCore pid=17815)   File "/home/malone/vllm/vllm/v1/engine/core.py", line 118, in __init__
(EngineCore pid=17815)     self.model_executor = executor_class(vllm_config)
(EngineCore pid=17815)                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=17815)   File "/home/malone/vllm/vllm/tracing/otel.py", line 178, in sync_wrapper
(EngineCore pid=17815)     return func(*args, **kwargs)
(EngineCore pid=17815)            ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=17815)   File "/home/malone/vllm/vllm/v1/executor/abstract.py", line 109, in __init__
(EngineCore pid=17815)     self._init_executor()
(EngineCore pid=17815)   File "/home/malone/vllm/vllm/v1/executor/uniproc_executor.py", line 60, in _init_executor
(EngineCore pid=17815)     self.driver_worker.init_device()
(EngineCore pid=17815)   File "/home/malone/vllm/vllm/v1/worker/worker_base.py", line 325, in init_device
(EngineCore pid=17815)     self.worker.init_device()  # type: ignore
(EngineCore pid=17815)     ^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=17815)   File "/home/malone/vllm/vllm/tracing/otel.py", line 178, in sync_wrapper
(EngineCore pid=17815)     return func(*args, **kwargs)
(EngineCore pid=17815)            ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=17815)   File "/home/malone/vllm/vllm/v1/worker/gpu_worker.py", line 322, in init_device
(EngineCore pid=17815)     self.model_runner: GPUModelRunner = GPUModelRunnerV2(  # type: ignore
(EngineCore pid=17815)                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=17815)   File "/home/malone/vllm/vllm/v1/worker/gpu/model_runner.py", line 193, in __init__
(EngineCore pid=17815)     self.req_states = RequestState(
(EngineCore pid=17815)                       ^^^^^^^^^^^^^
(EngineCore pid=17815)   File "/home/malone/vllm/vllm/v1/worker/gpu/states.py", line 33, in __init__
(EngineCore pid=17815)     self.all_token_ids = StagedWriteTensor(
(EngineCore pid=17815)                          ^^^^^^^^^^^^^^^^^^
(EngineCore pid=17815)   File "/home/malone/vllm/vllm/v1/worker/gpu/buffer_utils.py", line 126, in __init__
(EngineCore pid=17815)     self._uva_buf = UvaBuffer(size, dtype)
(EngineCore pid=17815)                     ^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=17815)   File "/home/malone/vllm/vllm/v1/worker/gpu/buffer_utils.py", line 39, in __init__
(EngineCore pid=17815)     raise RuntimeError("UVA is not available")
(EngineCore pid=17815) RuntimeError: UVA is not available
[rank0]:[W522 10:02:40.678313503 ProcessGroupNCCL.cpp:1575] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())
(APIServer pid=17649) Traceback (most recent call last):
(APIServer pid=17649)   File "/home/malone/vllm/.venv/bin/vllm", line 10, in <module>
(APIServer pid=17649)     sys.exit(main())
(APIServer pid=17649)              ^^^^^^
(APIServer pid=17649)   File "/home/malone/vllm/vllm/entrypoints/cli/main.py", line 92, in main
(APIServer pid=17649)     args.dispatch_function(args)
(APIServer pid=17649)   File "/home/malone/vllm/vllm/entrypoints/cli/serve.py", line 132, in cmd
(APIServer pid=17649)     uvloop.run(run_server(args))
(APIServer pid=17649)   File "/home/malone/vllm/.venv/lib/python3.12/site-packages/uvloop/__init__.py", line 96, in run
(APIServer pid=17649)     return __asyncio.run(
(APIServer pid=17649)            ^^^^^^^^^^^^^^
(APIServer pid=17649)   File "/home/malone/.local/share/uv/python/cpython-3.12.11-linux-x86_64-gnu/lib/python3.12/asyncio/runners.py", line 195, in run
(APIServer pid=17649)     return runner.run(main)
(APIServer pid=17649)            ^^^^^^^^^^^^^^^^
(APIServer pid=17649)   File "/home/malone/.local/share/uv/python/cpython-3.12.11-linux-x86_64-gnu/lib/python3.12/asyncio/runners.py", line 118, in run
(APIServer pid=17649)     return self._loop.run_until_complete(task)
(APIServer pid=17649)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=17649)   File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
(APIServer pid=17649)   File "/home/malone/vllm/.venv/lib/python3.12/site-packages/uvloop/__init__.py", line 48, in wrapper
(APIServer pid=17649)     return await main
(APIServer pid=17649)            ^^^^^^^^^^
(APIServer pid=17649)   File "/home/malone/vllm/vllm/entrypoints/openai/api_server.py", line 679, in run_server
(APIServer pid=17649)     await run_server_worker(listen_address, sock, args, **uvicorn_kwargs)
(APIServer pid=17649)   File "/home/malone/vllm/vllm/entrypoints/openai/api_server.py", line 693, in run_server_worker
(APIServer pid=17649)     async with build_async_engine_client(
(APIServer pid=17649)                ^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=17649)   File "/home/malone/.local/share/uv/python/cpython-3.12.11-linux-x86_64-gnu/lib/python3.12/contextlib.py", line 210, in __aenter__
(APIServer pid=17649)     return await anext(self.gen)
(APIServer pid=17649)            ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=17649)   File "/home/malone/vllm/vllm/entrypoints/openai/api_server.py", line 100, in build_async_engine_client
(APIServer pid=17649)     async with build_async_engine_client_from_engine_args(
(APIServer pid=17649)                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=17649)   File "/home/malone/.local/share/uv/python/cpython-3.12.11-linux-x86_64-gnu/lib/python3.12/contextlib.py", line 210, in __aenter__
(APIServer pid=17649)     return await anext(self.gen)
(APIServer pid=17649)            ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=17649)   File "/home/malone/vllm/vllm/entrypoints/openai/api_server.py", line 136, in build_async_engine_client_from_engine_args
(APIServer pid=17649)     async_llm = AsyncLLM.from_vllm_config(
(APIServer pid=17649)                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=17649)   File "/home/malone/vllm/vllm/v1/engine/async_llm.py", line 217, in from_vllm_config
(APIServer pid=17649)     return cls(
(APIServer pid=17649)            ^^^^
(APIServer pid=17649)   File "/home/malone/vllm/vllm/v1/engine/async_llm.py", line 146, in __init__
(APIServer pid=17649)     self.engine_core = EngineCoreClient.make_async_mp_client(
(APIServer pid=17649)                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=17649)   File "/home/malone/vllm/vllm/tracing/otel.py", line 178, in sync_wrapper
(APIServer pid=17649)     return func(*args, **kwargs)
(APIServer pid=17649)            ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=17649)   File "/home/malone/vllm/vllm/v1/engine/core_client.py", line 130, in make_async_mp_client
(APIServer pid=17649)     return AsyncMPClient(*client_args)
(APIServer pid=17649)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=17649)   File "/home/malone/vllm/vllm/tracing/otel.py", line 178, in sync_wrapper
(APIServer pid=17649)     return func(*args, **kwargs)
(APIServer pid=17649)            ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=17649)   File "/home/malone/vllm/vllm/v1/engine/core_client.py", line 900, in __init__
(APIServer pid=17649)     super().__init__(
(APIServer pid=17649)   File "/home/malone/vllm/vllm/v1/engine/core_client.py", line 535, in __init__
(APIServer pid=17649)     with launch_core_engines(
(APIServer pid=17649)          ^^^^^^^^^^^^^^^^^^^^
(APIServer pid=17649)   File "/home/malone/.local/share/uv/python/cpython-3.12.11-linux-x86_64-gnu/lib/python3.12/contextlib.py", line 144, in __exit__
(APIServer pid=17649)     next(self.gen)
(APIServer pid=17649)   File "/home/malone/vllm/vllm/v1/engine/utils.py", line 1133, in launch_core_engines
(APIServer pid=17649)     wait_for_engine_startup(
(APIServer pid=17649)   File "/home/malone/vllm/vllm/v1/engine/utils.py", line 1192, in wait_for_engine_startup
(APIServer pid=17649)     raise RuntimeError(
(APIServer pid=17649) RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {}

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[opiateblush]

Ran into the same issue using the Docker image vllm/vllm-openai:v0.22.0 with Docker on Windows using the WSL2 kernel.

[erikperkins]

Same error on WSL2 with meta-llama/Llama-3.2-1B-Instruct.

[rs38]

WSL2 with old vllm/vllm-openai:v0.21.0 works fine

[DarkLight1337]

Should be fixed by #41496, feel free to try it

[rs38]

@DarkLight1337 pulling vllm/vllm-openai:nightly-x86_64 ... it takes ages as the layers are never cached. :)

docker run --runtime nvidia --gpus all -v ~/.cache/huggingface:/root/.cache/huggingface vllm/vllm-openai:nightly-x86_64 --model Qwen/Qwen3-0.6B
vLLM version 0.23.1rc1.dev101+g4c6266331

the warning WARNING 06-17 07:12:23 [interface.py:757] Using 'pin_memory=False' as WSL is detected. This may slow down the performance. is now gone, but

out of the box it still returns UVA is not available

btw: the error seems unrelated to
--cpu-offload-gb

[DarkLight1337]

Sorry I forgot to mention, you should set VLLM_WSL2_ENABLE_PIN_MEMORY=1 explicitly

[rs38]

docker run --runtime nvidia --gpus all -v ~/.cache/huggingface:/root/.cache/huggingface --env "VLLM_WSL2_ENABLE_PIN_MEMORY=1" vllm/vllm-openai:cu129-nightly-x86_64 --model Qwen/Qwen3-0.6B

sorry, edited.

now runs as expected!