/scratch/llama/ggml/src/ggml-sycl/mmvq.cpp:811: GGML_ASSERT(block_num_y % num_subgroups == 0) failed
I can run the above command with other models successfully, e.g. unsloth/gemma-3-12b-it-GGUF:Q4_K_M.
$llama-completion -hf unsloth/Olmo-3-7B-Instruct-GGUF:Q4_K_M --ctx-size 2048 -fit off
common_download_file_single_online: using cached file: /scratch/llama_cache/unsloth_Olmo-3-7B-Instruct-GGUF_Olmo-3-7B-Instruct-Q4_K_M.gguf
build: 0 (unknown) with IntelLLVM 2025.3.0 for Linux x86_64
main: llama backend init
main: load the model and apply lora adapter, if any
get_memory_info: [warning] ext_intel_free_memory is not supported (export/set ZES_ENABLE_SYSMAN=1 to support), use total memory as free memory
get_memory_info: [warning] ext_intel_free_memory is not supported (export/set ZES_ENABLE_SYSMAN=1 to support), use total memory as free memory
get_memory_info: [warning] ext_intel_free_memory is not supported (export/set ZES_ENABLE_SYSMAN=1 to support), use total memory as free memory
get_memory_info: [warning] ext_intel_free_memory is not supported (export/set ZES_ENABLE_SYSMAN=1 to support), use total memory as free memory
get_memory_info: [warning] ext_intel_free_memory is not supported (export/set ZES_ENABLE_SYSMAN=1 to support), use total memory as free memory
llama_model_load_from_file_impl: using device SYCL0 (Intel(R) Arc(TM) A770 Graphics) (unknown id) - 15473 MiB free
llama_model_load_from_file_impl: using device SYCL1 (Intel(R) Arc(TM) A770 Graphics) (unknown id) - 15473 MiB free
llama_model_loader: loaded meta data with 45 key-value pairs and 355 tensors from /scratch/llama_cache/unsloth_Olmo-3-7B-Instruct-GGUF_Olmo-3-7B-Instruct-Q4_K_M.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv 0: general.architecture str = olmo2
llama_model_loader: - kv 1: general.type str = model
llama_model_loader: - kv 2: general.name str = Olmo-3-7B-Instruct
llama_model_loader: - kv 3: general.finetune str = Instruct
llama_model_loader: - kv 4: general.basename str = Olmo-3-7B-Instruct
llama_model_loader: - kv 5: general.quantized_by str = Unsloth
llama_model_loader: - kv 6: general.size_label str = 7B
llama_model_loader: - kv 7: general.license str = apache-2.0
llama_model_loader: - kv 8: general.repo_url str = https://huggingface.co/unsloth
llama_model_loader: - kv 9: general.base_model.count u32 = 1
llama_model_loader: - kv 10: general.base_model.0.name str = Olmo 3 7B Instruct
llama_model_loader: - kv 11: general.base_model.0.organization str = Allenai
llama_model_loader: - kv 12: general.base_model.0.repo_url str = https://huggingface.co/allenai/Olmo-3...
llama_model_loader: - kv 13: general.tags arr[str,1] = ["unsloth"]
llama_model_loader: - kv 14: general.languages arr[str,1] = ["en"]
llama_model_loader: - kv 15: olmo2.block_count u32 = 32
llama_model_loader: - kv 16: olmo2.context_length u32 = 65536
llama_model_loader: - kv 17: olmo2.embedding_length u32 = 4096
llama_model_loader: - kv 18: olmo2.feed_forward_length u32 = 11008
llama_model_loader: - kv 19: olmo2.attention.head_count u32 = 32
llama_model_loader: - kv 20: olmo2.attention.head_count_kv u32 = 32
llama_model_loader: - kv 21: olmo2.rope.freq_base f32 = 500000.000000
llama_model_loader: - kv 22: olmo2.attention.layer_norm_rms_epsilon f32 = 0.000001
llama_model_loader: - kv 23: olmo2.rope.scaling.type str = yarn
llama_model_loader: - kv 24: olmo2.rope.scaling.factor f32 = 8.000000
llama_model_loader: - kv 25: olmo2.rope.scaling.attn_factor f32 = 1.207944
llama_model_loader: - kv 26: olmo2.rope.scaling.original_context_length u32 = 8192
llama_model_loader: - kv 27: olmo2.attention.sliding_window u32 = 4096
llama_model_loader: - kv 28: olmo2.attention.sliding_window_pattern arr[bool,32] = [true, true, true, false, true, true,...
llama_model_loader: - kv 29: tokenizer.ggml.model str = gpt2
llama_model_loader: - kv 30: tokenizer.ggml.pre str = dbrx
llama_model_loader: - kv 31: tokenizer.ggml.tokens arr[str,100278] = ["!", "\"", "#", "$", "%", "&", "'", ...
llama_model_loader: - kv 32: tokenizer.ggml.token_type arr[i32,100278] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv 33: tokenizer.ggml.merges arr[str,100000] = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",...
llama_model_loader: - kv 34: tokenizer.ggml.bos_token_id u32 = 100257
llama_model_loader: - kv 35: tokenizer.ggml.eos_token_id u32 = 100257
llama_model_loader: - kv 36: tokenizer.ggml.unknown_token_id u32 = 5809
llama_model_loader: - kv 37: tokenizer.ggml.padding_token_id u32 = 100277
llama_model_loader: - kv 38: tokenizer.chat_template str = {# Unsloth template fixes #}\n{%- set ...
llama_model_loader: - kv 39: general.quantization_version u32 = 2
llama_model_loader: - kv 40: general.file_type u32 = 15
llama_model_loader: - kv 41: quantize.imatrix.file str = Olmo-3-7B-Instruct-GGUF/imatrix_unslo...
llama_model_loader: - kv 42: quantize.imatrix.dataset str = unsloth_calibration_Olmo-3-7B-Instruc...
llama_model_loader: - kv 43: quantize.imatrix.entries_count u32 = 224
llama_model_loader: - kv 44: quantize.imatrix.chunks_count u32 = 689
llama_model_loader: - type f32: 129 tensors
llama_model_loader: - type q4_K: 193 tensors
llama_model_loader: - type q6_K: 33 tensors
print_info: file format = GGUF V3 (latest)
print_info: file type = Q4_K - Medium
print_info: file size = 4.16 GiB (4.90 BPW)
load: printing all EOG tokens:
load: - 100257 ('<|endoftext|>')
load: - 100265 ('<|im_end|>')
load: special tokens cache size = 23
load: token to piece cache size = 0.6143 MB
print_info: arch = olmo2
print_info: vocab_only = 0
print_info: no_alloc = 0
print_info: n_ctx_train = 65536
print_info: n_embd = 4096
print_info: n_embd_inp = 4096
print_info: n_layer = 32
print_info: n_head = 32
print_info: n_head_kv = 32
print_info: n_rot = 128
print_info: n_swa = 4096
print_info: is_swa_any = 1
print_info: n_embd_head_k = 128
print_info: n_embd_head_v = 128
print_info: n_gqa = 1
print_info: n_embd_k_gqa = 4096
print_info: n_embd_v_gqa = 4096
print_info: f_norm_eps = 0.0e+00
print_info: f_norm_rms_eps = 1.0e-06
print_info: f_clamp_kqv = 0.0e+00
print_info: f_max_alibi_bias = 0.0e+00
print_info: f_logit_scale = 0.0e+00
print_info: f_attn_scale = 0.0e+00
print_info: n_ff = 11008
print_info: n_expert = 0
print_info: n_expert_used = 0
print_info: n_expert_groups = 0
print_info: n_group_used = 0
print_info: causal attn = 1
print_info: pooling type = 0
print_info: rope type = 2
print_info: rope scaling = yarn
print_info: freq_base_train = 500000.0
print_info: freq_scale_train = 0.125
print_info: n_ctx_orig_yarn = 8192
print_info: rope_yarn_log_mul= 0.0000
print_info: rope_finetuned = unknown
print_info: model type = 7B
print_info: model params = 7.30 B
print_info: general.name = Olmo-3-7B-Instruct
print_info: vocab type = BPE
print_info: n_vocab = 100278
print_info: n_merges = 100000
print_info: BOS token = 100257 '<|endoftext|>'
print_info: EOS token = 100257 '<|endoftext|>'
print_info: EOT token = 100265 '<|im_end|>'
print_info: UNK token = 5809 '�'
print_info: PAD token = 100277 '<|pad|>'
print_info: LF token = 198 'Ċ'
print_info: FIM PRE token = 100258 '<|fim_prefix|>'
print_info: FIM SUF token = 100260 '<|fim_suffix|>'
print_info: FIM MID token = 100259 '<|fim_middle|>'
print_info: EOG token = 100257 '<|endoftext|>'
print_info: EOG token = 100265 '<|im_end|>'
print_info: max token length = 256
load_tensors: loading model tensors, this can take a while... (mmap = true)
get_memory_info: [warning] ext_intel_free_memory is not supported (export/set ZES_ENABLE_SYSMAN=1 to support), use total memory as free memory
get_memory_info: [warning] ext_intel_free_memory is not supported (export/set ZES_ENABLE_SYSMAN=1 to support), use total memory as free memory
get_memory_info: [warning] ext_intel_free_memory is not supported (export/set ZES_ENABLE_SYSMAN=1 to support), use total memory as free memory
get_memory_info: [warning] ext_intel_free_memory is not supported (export/set ZES_ENABLE_SYSMAN=1 to support), use total memory as free memory
load_tensors: offloading output layer to GPU
load_tensors: offloading 31 repeating layers to GPU
load_tensors: offloaded 33/33 layers to GPU
load_tensors: CPU_Mapped model buffer size = 220.34 MiB
load_tensors: SYCL0 model buffer size = 1968.32 MiB
load_tensors: SYCL1 model buffer size = 2072.41 MiB
..........................................................................................
common_init_result: added <|endoftext|> logit bias = -inf
common_init_result: added <|im_end|> logit bias = -inf
llama_context: constructing llama_context
llama_context: n_seq_max = 1
llama_context: n_ctx = 2048
llama_context: n_ctx_seq = 2048
llama_context: n_batch = 2048
llama_context: n_ubatch = 512
llama_context: causal_attn = 1
llama_context: flash_attn = auto
llama_context: kv_unified = false
llama_context: freq_base = 500000.0
llama_context: freq_scale = 0.125
llama_context: n_ctx_seq (2048) < n_ctx_train (65536) -- the full capacity of the model will not be utilized
Running with Environment Variables:
GGML_SYCL_DEBUG: 0
GGML_SYCL_DISABLE_OPT: 0
GGML_SYCL_DISABLE_GRAPH: 1
GGML_SYCL_DISABLE_DNN: DNN disabled by compile flag
GGML_SYCL_PRIORITIZE_DMMV: 0
Build with Macros:
GGML_SYCL_FORCE_MMQ: no
GGML_SYCL_F16: no
Found 2 SYCL devices:
| | | | |Max | |Max |Global | |
| | | | |compute|Max work|sub |mem | |
|ID| Device Type| Name|Version|units |group |group|size | Driver version|
|--|-------------------|---------------------------------------|-------|-------|--------|-----|-------|---------------------|
| 0| [opencl:gpu:0]| Intel Arc A770 Graphics| 3.0| 512| 1024| 32| 16225M| 25.48.36300|
| 1| [opencl:gpu:1]| Intel Arc A770 Graphics| 3.0| 512| 1024| 32| 16225M| 25.48.36300|
SYCL Optimization Feature:
|ID| Device Type|Reorder|
|--|-------------------|-------|
| 0| [opencl:gpu:0]| Y|
| 1| [opencl:gpu:1]| Y|
llama_context: SYCL_Host output buffer size = 0.38 MiB
llama_kv_cache_iswa: creating non-SWA KV cache, size = 2048 cells
llama_kv_cache: SYCL0 KV buffer size = 128.00 MiB
llama_kv_cache: SYCL1 KV buffer size = 128.00 MiB
llama_kv_cache: size = 256.00 MiB ( 2048 cells, 8 layers, 1/1 seqs), K (f16): 128.00 MiB, V (f16): 128.00 MiB
llama_kv_cache_iswa: creating SWA KV cache, size = 2048 cells
llama_kv_cache: SYCL0 KV buffer size = 416.00 MiB
llama_kv_cache: SYCL1 KV buffer size = 352.00 MiB
llama_kv_cache: size = 768.00 MiB ( 2048 cells, 24 layers, 1/1 seqs), K (f16): 384.00 MiB, V (f16): 384.00 MiB
get_memory_info: [warning] ext_intel_free_memory is not supported (export/set ZES_ENABLE_SYSMAN=1 to support), use total memory as free memory
get_memory_info: [warning] ext_intel_free_memory is not supported (export/set ZES_ENABLE_SYSMAN=1 to support), use total memory as free memory
llama_context: pipeline parallelism enabled (n_copies=4)
llama_context: layer 0 is assigned to device SYCL0 but the Flash Attention tensor is assigned to device CPU (usually due to missing support)
llama_context: Flash Attention was auto, set to disabled
llama_context: SYCL0 compute buffer size = 232.07 MiB
llama_context: SYCL1 compute buffer size = 267.93 MiB
llama_context: SYCL_Host compute buffer size = 40.09 MiB
llama_context: graph nodes = 1286
llama_context: graph splits = 3
common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
/scratch/llama/ggml/src/ggml-sycl/mmvq.cpp:811: GGML_ASSERT(block_num_y % num_subgroups == 0) failed
[New LWP 69604]
[New LWP 69599]
This GDB supports auto-downloading debuginfo from the following URLs:
<https://debuginfod.archlinux.org>
Enable debuginfod for this session? (y or [n]) [answered N; input not from terminal]
Debuginfod has been disabled.
To make this setting permanent, add 'set debuginfod enabled off' to .gdbinit.
warning: File "/opt/intel/oneapi/compiler/2025.3/lib/libsycl.so.8.0.0-gdb.py" auto-loading has been declined by your `auto-load safe-path' set to "$debugdir:$datadir/auto-load".
To enable execution of this file add
add-auto-load-safe-path /opt/intel/oneapi/compiler/2025.3/lib/libsycl.so.8.0.0-gdb.py
line to your configuration file "/home/nathan/.config/gdb/gdbinit".
To completely disable this security protection add
set auto-load safe-path /
line to your configuration file "/home/nathan/.config/gdb/gdbinit".
For more information about this security protection see the
"Auto-loading safe path" section in the GDB manual. E.g., run from the shell:
info "(gdb)Auto-loading safe path"
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
0x00007f1af549f002 in ?? () from /usr/lib/libc.so.6
#0 0x00007f1af549f002 in ?? () from /usr/lib/libc.so.6
#1 0x00007f1af549316c in ?? () from /usr/lib/libc.so.6
#2 0x00007f1af54931b4 in ?? () from /usr/lib/libc.so.6
#3 0x00007f1af5503d8f in wait4 () from /usr/lib/libc.so.6
#4 0x00007f1af65605da in ggml_print_backtrace () from /scratch/llama/build/bin/libggml-base.so.0
#5 0x00007f1af655f749 in ggml_abort () from /scratch/llama/build/bin/libggml-base.so.0
#6 0x00007f1af5d0f705 in ggml_sycl_op_mul_mat_vec_q(ggml_backend_sycl_context&, ggml_tensor const*, ggml_tensor const*, ggml_tensor*, char const*, float const*, char const*, float*, long, long, long, long, sycl::_V1::queue* const&) () from /scratch/llama/build/bin/libggml-sycl.so.0
#7 0x00007f1af5c58102 in void ggml_sycl_op_mul_mat<quantize_and_reorder_q8_1_soa>(ggml_backend_sycl_context&, ggml_tensor const*, ggml_tensor const*, ggml_tensor*, void (*)(ggml_backend_sycl_context&, ggml_tensor const*, ggml_tensor const*, ggml_tensor*, char const*, float const*, char const*, float*, long, long, long, long, sycl::_V1::queue* const&)) () from /scratch/llama/build/bin/libggml-sycl.so.0
#8 0x00007f1af5c4ca1a in ggml_sycl_mul_mat(ggml_backend_sycl_context&, ggml_tensor const*, ggml_tensor const*, ggml_tensor*) () from /scratch/llama/build/bin/libggml-sycl.so.0
#9 0x00007f1af5c4b0de in ggml_backend_sycl_graph_compute_impl(ggml_backend_sycl_context*, ggml_cgraph*) () from /scratch/llama/build/bin/libggml-sycl.so.0
#10 0x00007f1af5c49137 in ggml_backend_sycl_graph_compute(ggml_backend*, ggml_cgraph*) () from /scratch/llama/build/bin/libggml-sycl.so.0
#11 0x00007f1af6584e29 in ggml_backend_sched_graph_compute_async () from /scratch/llama/build/bin/libggml-base.so.0
#12 0x00007f1af628b461 in llama_context::graph_compute(ggml_cgraph*, bool) () from /scratch/llama/build/bin/libllama.so.0
#13 0x00007f1af628b0cf in llama_context::process_ubatch(llama_ubatch const&, llm_graph_type, llama_memory_context_i*, ggml_status&) () from /scratch/llama/build/bin/libllama.so.0
#14 0x00007f1af628c92e in llama_context::decode(llama_batch const&) () from /scratch/llama/build/bin/libllama.so.0
#15 0x00007f1af6290cdb in llama_decode () from /scratch/llama/build/bin/libllama.so.0
#16 0x000000000052e2fd in common_init_from_params(common_params&) ()
#17 0x000000000041dde4 in main ()
[Inferior 1 (process 69598) detached]
[1] 69598 IOT instruction (core dumped) llama-completion -hf unsloth/Olmo-3-7B-Instruct-GGUF:Q4_K_M --ctx-size 2048
Name and Version
version: b7491
built with IntelLLVM 2025.3.0 for Linux x86_64
Running
--versionreports0 (unknown).Operating systems
Linux
GGML backends
SYCL
Hardware
GPU: 2x Intel ARC A770
CPU: AMD Ryzen 5 5600
Bug also occurs when using a single A770 using the
--main-gpu 0or--main-gpu 1Models
unsloth/Olmo-3-7B-Instruct-GGUF:Q4_K_M
Problem description & steps to reproduce
If I run
llama-completion -hf unsloth/Olmo-3-7B-Instruct-GGUF:Q4_K_M --ctx-size 2048 -fit off, llama.cpp crashes with the following failed assert:I can run the above command with other models successfully, e.g.
unsloth/gemma-3-12b-it-GGUF:Q4_K_M.First Bad Commit
No response
Relevant log output