[Bug]: [CPU-only][Graviton3] oneDNN matmul primitive creation fails running examples/offline_inference/basic/basic.py #24976
Description
Your current environment
The output of python collect_env.py
Collecting environment information...
==============================
System Info
==============================
OS : Ubuntu 22.04.5 LTS (aarch64)
GCC version : (Ubuntu 12.3.0-1ubuntu1~22.04.2) 12.3.0
Clang version : 16.0.6 (++20231112100510+7cbf1a259152-1~exp1~20231112100554.106)
CMake version : version 4.1.0
Libc version : glibc-2.35
==============================
PyTorch Info
==============================
PyTorch version : 2.8.0+cpu
Is debug build : False
CUDA used to build PyTorch : Could not collect
ROCM used to build PyTorch : N/A
==============================
Python Environment
==============================
Python version : 3.12.11 (main, Sep 2 2025, 14:20:15) [Clang 20.1.4 ] (64-bit runtime)
Python platform : Linux-6.8.0-1036-aws-aarch64-with-glibc2.35
==============================
CUDA / GPU Info
==============================
Is CUDA available : False
CUDA runtime version : 11.5.119
CUDA_MODULE_LOADING set to : N/A
GPU models and configuration : Could not collect
Nvidia driver version : Could not collect
cuDNN version : Could not collect
HIP runtime version : N/A
MIOpen runtime version : N/A
Is XNNPACK available : True
==============================
CPU Info
==============================
Architecture: aarch64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model: 1
Thread(s) per core: 1
Core(s) per socket: 16
Socket(s): 1
Stepping: r1p1
BogoMIPS: 2100.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm ssbs paca pacg dcpodp svei8mm svebf16 i8mm bf16 dgh rng
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 16 MiB (16 instances)
L3 cache: 32 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Not affected
==============================
Versions of relevant libraries
==============================
[pip3] numpy==2.2.6
[pip3] pyzmq==27.1.0
[pip3] torch==2.8.0+cpu
[pip3] torchaudio==2.8.0
[pip3] torchvision==0.23.0
[pip3] transformers==4.56.1
[conda] Could not collect
==============================
vLLM Info
==============================
ROCM Version : Could not collect
vLLM Version : 0.10.2rc3.dev104+g04ad0dc27 (git sha: 04ad0dc27)
vLLM Build Flags:
CUDA Archs: Not Set; ROCm: Disabled
GPU Topology:
Could not collect
==============================
Environment Variables
==============================
PYTORCH_NVML_BASED_CUDA_CHECK=1
TORCHINDUCTOR_COMPILE_THREADS=1
🐛 Describe the bug
Running python examples/offline_inference/basic/basic.py on CPU-only (AWS Graviton3) fails with:
RuntimeError: could not create a primitive descriptor for the matmul primitive.
Complete Error Trace
INFO 09-16 12:42:12 [__init__.py:216] Automatically detected platform cpu.
INFO 09-16 12:42:13 [utils.py:328] non-default args: {'disable_log_stats': True, 'model': 'facebook/opt-125m'}
INFO 09-16 12:42:18 [__init__.py:706] Resolved architecture: OPTForCausalLM
`torch_dtype` is deprecated! Use `dtype` instead!
INFO 09-16 12:42:18 [__init__.py:1782] Using max model len 2048
WARNING 09-16 12:42:18 [cpu.py:114] Environment variable VLLM_CPU_KVCACHE_SPACE (GiB) for CPU backend is not set, using 4 by default.
INFO 09-16 12:42:18 [arg_utils.py:1160] Chunked prefill is not supported for ARM and POWER and S390X CPUs; disabling it for V1 backend.
INFO 09-16 12:42:22 [__init__.py:216] Automatically detected platform cpu.
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m INFO 09-16 12:42:23 [core.py:648] Waiting for init message from front-end.
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m INFO 09-16 12:42:23 [core.py:75] Initializing a V1 LLM engine (v0.10.2rc3.dev104+g04ad0dc27) with config: model='facebook/opt-125m', speculative_config=None, tokenizer='facebook/opt-125m', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=False, dtype=torch.float16, max_seq_len=2048, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=True, quantization=None, enforce_eager=False, kv_cache_dtype=auto, device_config=cpu, decoding_config=DecodingConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_backend=''), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None), seed=0, served_model_name=facebook/opt-125m, enable_prefix_caching=True, chunked_prefill_enabled=False, use_async_output_proc=False, pooler_config=None, compilation_config={"level":2,"debug_dump_path":"","cache_dir":"","backend":"inductor","custom_ops":["none"],"splitting_ops":null,"use_inductor":true,"compile_sizes":null,"inductor_compile_config":{"enable_auto_functionalized_v2":false,"dce":true,"size_asserts":false,"nan_asserts":false,"epilogue_fusion":true},"inductor_passes":{},"cudagraph_mode":0,"use_cudagraph":true,"cudagraph_num_of_warmups":0,"cudagraph_capture_sizes":[],"cudagraph_copy_inputs":false,"full_cuda_graph":false,"pass_config":{},"max_capture_size":null,"local_cache_dir":null}
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m INFO 09-16 12:42:24 [importing.py:63] Triton not installed or not compatible; certain GPU-related functions will not be available.
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m WARNING 09-16 12:42:24 [cpu.py:312] Pin memory is not supported on CPU.
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m INFO 09-16 12:42:25 [parallel_state.py:1206] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, TP rank 0, EP rank 0
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m INFO 09-16 12:42:25 [cpu_model_runner.py:101] Starting to load model facebook/opt-125m...
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m INFO 09-16 12:42:25 [cpu.py:101] Using Torch SDPA backend.
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m INFO 09-16 12:42:25 [weight_utils.py:348] Using model weights format ['*.safetensors', '*.bin', '*.pt']
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m
Loading pt checkpoint shards: 0% Completed | 0/1 [00:00<?, ?it/s]
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m
Loading pt checkpoint shards: 100% Completed | 1/1 [00:00<00:00, 1.01it/s]
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m
Loading pt checkpoint shards: 100% Completed | 1/1 [00:00<00:00, 1.01it/s]
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m INFO 09-16 12:42:26 [default_loader.py:268] Loading weights took 1.00 seconds
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] EngineCore failed to start.
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] Traceback (most recent call last):
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] File "vLLM_issue/.venv/lib/python3.12/site-packages/vllm-0.10.2rc3.dev104+g04ad0dc27.cpu-py3.12-linux-aarch64.egg/vllm/v1/engine/core.py", line 703, in run_engine_core
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] engine_core = EngineCoreProc(*args, **kwargs)
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] File "vLLM_issue/.venv/lib/python3.12/site-packages/vllm-0.10.2rc3.dev104+g04ad0dc27.cpu-py3.12-linux-aarch64.egg/vllm/v1/engine/core.py", line 502, in __init__
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] super().__init__(vllm_config, executor_class, log_stats,
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] File "vLLM_issue/.venv/lib/python3.12/site-packages/vllm-0.10.2rc3.dev104+g04ad0dc27.cpu-py3.12-linux-aarch64.egg/vllm/v1/engine/core.py", line 81, in __init__
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] self.model_executor = executor_class(vllm_config)
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] File "vLLM_issue/.venv/lib/python3.12/site-packages/vllm-0.10.2rc3.dev104+g04ad0dc27.cpu-py3.12-linux-aarch64.egg/vllm/executor/executor_base.py", line 55, in __init__
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] self._init_executor()
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] File "vLLM_issue/.venv/lib/python3.12/site-packages/vllm-0.10.2rc3.dev104+g04ad0dc27.cpu-py3.12-linux-aarch64.egg/vllm/executor/uniproc_executor.py", line 55, in _init_executor
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] self.collective_rpc("load_model")
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] File "vLLM_issue/.venv/lib/python3.12/site-packages/vllm-0.10.2rc3.dev104+g04ad0dc27.cpu-py3.12-linux-aarch64.egg/vllm/executor/uniproc_executor.py", line 83, in collective_rpc
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] return [run_method(self.driver_worker, method, args, kwargs)]
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] File "vLLM_issue/.venv/lib/python3.12/site-packages/vllm-0.10.2rc3.dev104+g04ad0dc27.cpu-py3.12-linux-aarch64.egg/vllm/utils/__init__.py", line 3067, in run_method
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] return func(*args, **kwargs)
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] ^^^^^^^^^^^^^^^^^^^^^
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] File "vLLM_issue/.venv/lib/python3.12/site-packages/vllm-0.10.2rc3.dev104+g04ad0dc27.cpu-py3.12-linux-aarch64.egg/vllm/v1/worker/gpu_worker.py", line 214, in load_model
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] self.model_runner.load_model(eep_scale_up=eep_scale_up)
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] File "vLLM_issue/.venv/lib/python3.12/site-packages/vllm-0.10.2rc3.dev104+g04ad0dc27.cpu-py3.12-linux-aarch64.egg/vllm/v1/worker/cpu_model_runner.py", line 102, in load_model
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] self.model = get_model(vllm_config=self.vllm_config)
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] File "vLLM_issue/.venv/lib/python3.12/site-packages/vllm-0.10.2rc3.dev104+g04ad0dc27.cpu-py3.12-linux-aarch64.egg/vllm/model_executor/model_loader/__init__.py", line 119, in get_model
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] return loader.load_model(vllm_config=vllm_config,
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] File "vLLM_issue/.venv/lib/python3.12/site-packages/vllm-0.10.2rc3.dev104+g04ad0dc27.cpu-py3.12-linux-aarch64.egg/vllm/model_executor/model_loader/base_loader.py", line 51, in load_model
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] process_weights_after_loading(model, model_config, target_device)
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] File "vLLM_issue/.venv/lib/python3.12/site-packages/vllm-0.10.2rc3.dev104+g04ad0dc27.cpu-py3.12-linux-aarch64.egg/vllm/model_executor/model_loader/utils.py", line 113, in process_weights_after_loading
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] quant_method.process_weights_after_loading(module)
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] File "vLLM_issue/.venv/lib/python3.12/site-packages/vllm-0.10.2rc3.dev104+g04ad0dc27.cpu-py3.12-linux-aarch64.egg/vllm/model_executor/layers/vocab_parallel_embedding.py", line 47, in process_weights_after_loading
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] dispatch_cpu_unquantized_gemm(layer, remove_weight=False)
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] File "vLLM_issue/.venv/lib/python3.12/site-packages/vllm-0.10.2rc3.dev104+g04ad0dc27.cpu-py3.12-linux-aarch64.egg/vllm/model_executor/layers/utils.py", line 175, in dispatch_cpu_unquantized_gemm
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] handler = ops.create_onednn_mm(origin_weight.t(), 32)
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] File "vLLM_issue/.venv/lib/python3.12/site-packages/vllm-0.10.2rc3.dev104+g04ad0dc27.cpu-py3.12-linux-aarch64.egg/vllm/_custom_ops.py", line 1928, in create_onednn_mm
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] handler.handler = torch.ops._C.create_onednn_mm_handler(
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] File "vLLM_issue/.venv/lib/python3.12/site-packages/torch/_ops.py", line 1243, in __call__
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] return self._op(*args, **kwargs)
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] ^^^^^^^^^^^^^^^^^^^^^^^^^
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ERROR 09-16 12:42:26 [core.py:712] RuntimeError: could not create a primitive descriptor for the matmul primitive. Run workload with environment variable ONEDNN_VERBOSE=all to get additional diagnostic information.
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m Process EngineCore_DP0:
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m Traceback (most recent call last):
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m File ".local/share/uv/python/cpython-3.12.11-linux-aarch64-gnu/lib/python3.12/multiprocessing/process.py", line 314, in _bootstrap
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m self.run()
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m File ".local/share/uv/python/cpython-3.12.11-linux-aarch64-gnu/lib/python3.12/multiprocessing/process.py", line 108, in run
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m self._target(*self._args, **self._kwargs)
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m File "vLLM_issue/.venv/lib/python3.12/site-packages/vllm-0.10.2rc3.dev104+g04ad0dc27.cpu-py3.12-linux-aarch64.egg/vllm/v1/engine/core.py", line 716, in run_engine_core
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m raise e
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m File "vLLM_issue/.venv/lib/python3.12/site-packages/vllm-0.10.2rc3.dev104+g04ad0dc27.cpu-py3.12-linux-aarch64.egg/vllm/v1/engine/core.py", line 703, in run_engine_core
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m engine_core = EngineCoreProc(*args, **kwargs)
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m File "vLLM_issue/.venv/lib/python3.12/site-packages/vllm-0.10.2rc3.dev104+g04ad0dc27.cpu-py3.12-linux-aarch64.egg/vllm/v1/engine/core.py", line 502, in __init__
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m super().__init__(vllm_config, executor_class, log_stats,
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m File "vLLM_issue/.venv/lib/python3.12/site-packages/vllm-0.10.2rc3.dev104+g04ad0dc27.cpu-py3.12-linux-aarch64.egg/vllm/v1/engine/core.py", line 81, in __init__
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m self.model_executor = executor_class(vllm_config)
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ^^^^^^^^^^^^^^^^^^^^^^^^^^^
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m File "vLLM_issue/.venv/lib/python3.12/site-packages/vllm-0.10.2rc3.dev104+g04ad0dc27.cpu-py3.12-linux-aarch64.egg/vllm/executor/executor_base.py", line 55, in __init__
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m self._init_executor()
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m File "vLLM_issue/.venv/lib/python3.12/site-packages/vllm-0.10.2rc3.dev104+g04ad0dc27.cpu-py3.12-linux-aarch64.egg/vllm/executor/uniproc_executor.py", line 55, in _init_executor
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m self.collective_rpc("load_model")
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m File "vLLM_issue/.venv/lib/python3.12/site-packages/vllm-0.10.2rc3.dev104+g04ad0dc27.cpu-py3.12-linux-aarch64.egg/vllm/executor/uniproc_executor.py", line 83, in collective_rpc
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m return [run_method(self.driver_worker, method, args, kwargs)]
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m File "vLLM_issue/.venv/lib/python3.12/site-packages/vllm-0.10.2rc3.dev104+g04ad0dc27.cpu-py3.12-linux-aarch64.egg/vllm/utils/__init__.py", line 3067, in run_method
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m return func(*args, **kwargs)
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ^^^^^^^^^^^^^^^^^^^^^
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m File "vLLM_issue/.venv/lib/python3.12/site-packages/vllm-0.10.2rc3.dev104+g04ad0dc27.cpu-py3.12-linux-aarch64.egg/vllm/v1/worker/gpu_worker.py", line 214, in load_model
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m self.model_runner.load_model(eep_scale_up=eep_scale_up)
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m File "vLLM_issue/.venv/lib/python3.12/site-packages/vllm-0.10.2rc3.dev104+g04ad0dc27.cpu-py3.12-linux-aarch64.egg/vllm/v1/worker/cpu_model_runner.py", line 102, in load_model
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m self.model = get_model(vllm_config=self.vllm_config)
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m File "vLLM_issue/.venv/lib/python3.12/site-packages/vllm-0.10.2rc3.dev104+g04ad0dc27.cpu-py3.12-linux-aarch64.egg/vllm/model_executor/model_loader/__init__.py", line 119, in get_model
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m return loader.load_model(vllm_config=vllm_config,
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m File "vLLM_issue/.venv/lib/python3.12/site-packages/vllm-0.10.2rc3.dev104+g04ad0dc27.cpu-py3.12-linux-aarch64.egg/vllm/model_executor/model_loader/base_loader.py", line 51, in load_model
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m process_weights_after_loading(model, model_config, target_device)
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m File "vLLM_issue/.venv/lib/python3.12/site-packages/vllm-0.10.2rc3.dev104+g04ad0dc27.cpu-py3.12-linux-aarch64.egg/vllm/model_executor/model_loader/utils.py", line 113, in process_weights_after_loading
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m quant_method.process_weights_after_loading(module)
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m File "vLLM_issue/.venv/lib/python3.12/site-packages/vllm-0.10.2rc3.dev104+g04ad0dc27.cpu-py3.12-linux-aarch64.egg/vllm/model_executor/layers/vocab_parallel_embedding.py", line 47, in process_weights_after_loading
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m dispatch_cpu_unquantized_gemm(layer, remove_weight=False)
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m File "vLLM_issue/.venv/lib/python3.12/site-packages/vllm-0.10.2rc3.dev104+g04ad0dc27.cpu-py3.12-linux-aarch64.egg/vllm/model_executor/layers/utils.py", line 175, in dispatch_cpu_unquantized_gemm
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m handler = ops.create_onednn_mm(origin_weight.t(), 32)
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m File "vLLM_issue/.venv/lib/python3.12/site-packages/vllm-0.10.2rc3.dev104+g04ad0dc27.cpu-py3.12-linux-aarch64.egg/vllm/_custom_ops.py", line 1928, in create_onednn_mm
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m handler.handler = torch.ops._C.create_onednn_mm_handler(
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m File "vLLM_issue/.venv/lib/python3.12/site-packages/torch/_ops.py", line 1243, in __call__
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m return self._op(*args, **kwargs)
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m ^^^^^^^^^^^^^^^^^^^^^^^^^
�[1;36m(EngineCore_DP0 pid=647832)�[0;0m RuntimeError: could not create a primitive descriptor for the matmul primitive. Run workload with environment variable ONEDNN_VERBOSE=all to get additional diagnostic information.
Traceback (most recent call last):
File "vLLM_issue/vllm_source/examples/offline_inference/basic/basic.py", line 35, in <module>
main()
File "vLLM_issue/vllm_source/examples/offline_inference/basic/basic.py", line 19, in main
llm = LLM(model="facebook/opt-125m")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "vLLM_issue/.venv/lib/python3.12/site-packages/vllm-0.10.2rc3.dev104+g04ad0dc27.cpu-py3.12-linux-aarch64.egg/vllm/entrypoints/llm.py", line 282, in __init__
self.llm_engine = LLMEngine.from_engine_args(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "vLLM_issue/.venv/lib/python3.12/site-packages/vllm-0.10.2rc3.dev104+g04ad0dc27.cpu-py3.12-linux-aarch64.egg/vllm/engine/llm_engine.py", line 506, in from_engine_args
return engine_cls.from_vllm_config(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "vLLM_issue/.venv/lib/python3.12/site-packages/vllm-0.10.2rc3.dev104+g04ad0dc27.cpu-py3.12-linux-aarch64.egg/vllm/v1/engine/llm_engine.py", line 134, in from_vllm_config
return cls(vllm_config=vllm_config,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "vLLM_issue/.venv/lib/python3.12/site-packages/vllm-0.10.2rc3.dev104+g04ad0dc27.cpu-py3.12-linux-aarch64.egg/vllm/v1/engine/llm_engine.py", line 111, in __init__
self.engine_core = EngineCoreClient.make_client(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "vLLM_issue/.venv/lib/python3.12/site-packages/vllm-0.10.2rc3.dev104+g04ad0dc27.cpu-py3.12-linux-aarch64.egg/vllm/v1/engine/core_client.py", line 80, in make_client
return SyncMPClient(vllm_config, executor_class, log_stats)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "vLLM_issue/.venv/lib/python3.12/site-packages/vllm-0.10.2rc3.dev104+g04ad0dc27.cpu-py3.12-linux-aarch64.egg/vllm/v1/engine/core_client.py", line 602, in __init__
super().__init__(
File "vLLM_issue/.venv/lib/python3.12/site-packages/vllm-0.10.2rc3.dev104+g04ad0dc27.cpu-py3.12-linux-aarch64.egg/vllm/v1/engine/core_client.py", line 448, in __init__
with launch_core_engines(vllm_config, executor_class,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".local/share/uv/python/cpython-3.12.11-linux-aarch64-gnu/lib/python3.12/contextlib.py", line 144, in __exit__
next(self.gen)
File "vLLM_issue/.venv/lib/python3.12/site-packages/vllm-0.10.2rc3.dev104+g04ad0dc27.cpu-py3.12-linux-aarch64.egg/vllm/v1/engine/utils.py", line 729, in launch_core_engines
wait_for_engine_startup(
File "vLLM_issue/.venv/lib/python3.12/site-packages/vllm-0.10.2rc3.dev104+g04ad0dc27.cpu-py3.12-linux-aarch64.egg/vllm/v1/engine/utils.py", line 782, in wait_for_engine_startup
raise RuntimeError("Engine core initialization failed. "
RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {}
Before submitting a new issue...
- Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.