Skip to content

Eval bug: GGML_ASSERT(block_num_y % num_subgroups == 0) failed using SYCL, A770, Olmo 3 #18240

Description

@kittrydge

Name and Version

version: b7491
built with IntelLLVM 2025.3.0 for Linux x86_64

Running --version reports 0 (unknown).

Operating systems

Linux

GGML backends

SYCL

Hardware

GPU: 2x Intel ARC A770
CPU: AMD Ryzen 5 5600

Bug also occurs when using a single A770 using the --main-gpu 0 or --main-gpu 1

Models

unsloth/Olmo-3-7B-Instruct-GGUF:Q4_K_M

Problem description & steps to reproduce

If I run llama-completion -hf unsloth/Olmo-3-7B-Instruct-GGUF:Q4_K_M --ctx-size 2048 -fit off, llama.cpp crashes with the following failed assert:

/scratch/llama/ggml/src/ggml-sycl/mmvq.cpp:811: GGML_ASSERT(block_num_y % num_subgroups == 0) failed

I can run the above command with other models successfully, e.g. unsloth/gemma-3-12b-it-GGUF:Q4_K_M.

First Bad Commit

No response

Relevant log output

$llama-completion -hf unsloth/Olmo-3-7B-Instruct-GGUF:Q4_K_M --ctx-size 2048 -fit off
                                                             
common_download_file_single_online: using cached file: /scratch/llama_cache/unsloth_Olmo-3-7B-Instruct-GGUF_Olmo-3-7B-Instruct-Q4_K_M.gguf
build: 0 (unknown) with IntelLLVM 2025.3.0 for Linux x86_64
main: llama backend init
main: load the model and apply lora adapter, if any
get_memory_info: [warning] ext_intel_free_memory is not supported (export/set ZES_ENABLE_SYSMAN=1 to support), use total memory as free memory
get_memory_info: [warning] ext_intel_free_memory is not supported (export/set ZES_ENABLE_SYSMAN=1 to support), use total memory as free memory
get_memory_info: [warning] ext_intel_free_memory is not supported (export/set ZES_ENABLE_SYSMAN=1 to support), use total memory as free memory
get_memory_info: [warning] ext_intel_free_memory is not supported (export/set ZES_ENABLE_SYSMAN=1 to support), use total memory as free memory
get_memory_info: [warning] ext_intel_free_memory is not supported (export/set ZES_ENABLE_SYSMAN=1 to support), use total memory as free memory
llama_model_load_from_file_impl: using device SYCL0 (Intel(R) Arc(TM) A770 Graphics) (unknown id) - 15473 MiB free
llama_model_load_from_file_impl: using device SYCL1 (Intel(R) Arc(TM) A770 Graphics) (unknown id) - 15473 MiB free
llama_model_loader: loaded meta data with 45 key-value pairs and 355 tensors from /scratch/llama_cache/unsloth_Olmo-3-7B-Instruct-GGUF_Olmo-3-7B-Instruct-Q4_K_M.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv   0:                       general.architecture str              = olmo2
llama_model_loader: - kv   1:                               general.type str              = model
llama_model_loader: - kv   2:                               general.name str              = Olmo-3-7B-Instruct
llama_model_loader: - kv   3:                           general.finetune str              = Instruct
llama_model_loader: - kv   4:                           general.basename str              = Olmo-3-7B-Instruct
llama_model_loader: - kv   5:                       general.quantized_by str              = Unsloth
llama_model_loader: - kv   6:                         general.size_label str              = 7B
llama_model_loader: - kv   7:                            general.license str              = apache-2.0
llama_model_loader: - kv   8:                           general.repo_url str              = https://huggingface.co/unsloth
llama_model_loader: - kv   9:                   general.base_model.count u32              = 1
llama_model_loader: - kv  10:                  general.base_model.0.name str              = Olmo 3 7B Instruct
llama_model_loader: - kv  11:          general.base_model.0.organization str              = Allenai
llama_model_loader: - kv  12:              general.base_model.0.repo_url str              = https://huggingface.co/allenai/Olmo-3...
llama_model_loader: - kv  13:                               general.tags arr[str,1]       = ["unsloth"]
llama_model_loader: - kv  14:                          general.languages arr[str,1]       = ["en"]
llama_model_loader: - kv  15:                          olmo2.block_count u32              = 32
llama_model_loader: - kv  16:                       olmo2.context_length u32              = 65536
llama_model_loader: - kv  17:                     olmo2.embedding_length u32              = 4096
llama_model_loader: - kv  18:                  olmo2.feed_forward_length u32              = 11008
llama_model_loader: - kv  19:                 olmo2.attention.head_count u32              = 32
llama_model_loader: - kv  20:              olmo2.attention.head_count_kv u32              = 32
llama_model_loader: - kv  21:                       olmo2.rope.freq_base f32              = 500000.000000
llama_model_loader: - kv  22:     olmo2.attention.layer_norm_rms_epsilon f32              = 0.000001
llama_model_loader: - kv  23:                    olmo2.rope.scaling.type str              = yarn
llama_model_loader: - kv  24:                  olmo2.rope.scaling.factor f32              = 8.000000
llama_model_loader: - kv  25:             olmo2.rope.scaling.attn_factor f32              = 1.207944
llama_model_loader: - kv  26: olmo2.rope.scaling.original_context_length u32              = 8192
llama_model_loader: - kv  27:             olmo2.attention.sliding_window u32              = 4096
llama_model_loader: - kv  28:     olmo2.attention.sliding_window_pattern arr[bool,32]     = [true, true, true, false, true, true,...
llama_model_loader: - kv  29:                       tokenizer.ggml.model str              = gpt2
llama_model_loader: - kv  30:                         tokenizer.ggml.pre str              = dbrx
llama_model_loader: - kv  31:                      tokenizer.ggml.tokens arr[str,100278]  = ["!", "\"", "#", "$", "%", "&", "'", ...
llama_model_loader: - kv  32:                  tokenizer.ggml.token_type arr[i32,100278]  = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv  33:                      tokenizer.ggml.merges arr[str,100000]  = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",...
llama_model_loader: - kv  34:                tokenizer.ggml.bos_token_id u32              = 100257
llama_model_loader: - kv  35:                tokenizer.ggml.eos_token_id u32              = 100257
llama_model_loader: - kv  36:            tokenizer.ggml.unknown_token_id u32              = 5809
llama_model_loader: - kv  37:            tokenizer.ggml.padding_token_id u32              = 100277
llama_model_loader: - kv  38:                    tokenizer.chat_template str              = {# Unsloth template fixes #}\n{%- set ...
llama_model_loader: - kv  39:               general.quantization_version u32              = 2
llama_model_loader: - kv  40:                          general.file_type u32              = 15
llama_model_loader: - kv  41:                      quantize.imatrix.file str              = Olmo-3-7B-Instruct-GGUF/imatrix_unslo...
llama_model_loader: - kv  42:                   quantize.imatrix.dataset str              = unsloth_calibration_Olmo-3-7B-Instruc...
llama_model_loader: - kv  43:             quantize.imatrix.entries_count u32              = 224
llama_model_loader: - kv  44:              quantize.imatrix.chunks_count u32              = 689
llama_model_loader: - type  f32:  129 tensors
llama_model_loader: - type q4_K:  193 tensors
llama_model_loader: - type q6_K:   33 tensors
print_info: file format = GGUF V3 (latest)
print_info: file type   = Q4_K - Medium
print_info: file size   = 4.16 GiB (4.90 BPW) 
load: printing all EOG tokens:
load:   - 100257 ('<|endoftext|>')
load:   - 100265 ('<|im_end|>')
load: special tokens cache size = 23
load: token to piece cache size = 0.6143 MB
print_info: arch             = olmo2
print_info: vocab_only       = 0
print_info: no_alloc         = 0
print_info: n_ctx_train      = 65536
print_info: n_embd           = 4096
print_info: n_embd_inp       = 4096
print_info: n_layer          = 32
print_info: n_head           = 32
print_info: n_head_kv        = 32
print_info: n_rot            = 128
print_info: n_swa            = 4096
print_info: is_swa_any       = 1
print_info: n_embd_head_k    = 128
print_info: n_embd_head_v    = 128
print_info: n_gqa            = 1
print_info: n_embd_k_gqa     = 4096
print_info: n_embd_v_gqa     = 4096
print_info: f_norm_eps       = 0.0e+00
print_info: f_norm_rms_eps   = 1.0e-06
print_info: f_clamp_kqv      = 0.0e+00
print_info: f_max_alibi_bias = 0.0e+00
print_info: f_logit_scale    = 0.0e+00
print_info: f_attn_scale     = 0.0e+00
print_info: n_ff             = 11008
print_info: n_expert         = 0
print_info: n_expert_used    = 0
print_info: n_expert_groups  = 0
print_info: n_group_used     = 0
print_info: causal attn      = 1
print_info: pooling type     = 0
print_info: rope type        = 2
print_info: rope scaling     = yarn
print_info: freq_base_train  = 500000.0
print_info: freq_scale_train = 0.125
print_info: n_ctx_orig_yarn  = 8192
print_info: rope_yarn_log_mul= 0.0000
print_info: rope_finetuned   = unknown
print_info: model type       = 7B
print_info: model params     = 7.30 B
print_info: general.name     = Olmo-3-7B-Instruct
print_info: vocab type       = BPE
print_info: n_vocab          = 100278
print_info: n_merges         = 100000
print_info: BOS token        = 100257 '<|endoftext|>'
print_info: EOS token        = 100257 '<|endoftext|>'
print_info: EOT token        = 100265 '<|im_end|>'
print_info: UNK token        = 5809 ''
print_info: PAD token        = 100277 '<|pad|>'
print_info: LF token         = 198 'Ċ'
print_info: FIM PRE token    = 100258 '<|fim_prefix|>'
print_info: FIM SUF token    = 100260 '<|fim_suffix|>'
print_info: FIM MID token    = 100259 '<|fim_middle|>'
print_info: EOG token        = 100257 '<|endoftext|>'
print_info: EOG token        = 100265 '<|im_end|>'
print_info: max token length = 256
load_tensors: loading model tensors, this can take a while... (mmap = true)
get_memory_info: [warning] ext_intel_free_memory is not supported (export/set ZES_ENABLE_SYSMAN=1 to support), use total memory as free memory
get_memory_info: [warning] ext_intel_free_memory is not supported (export/set ZES_ENABLE_SYSMAN=1 to support), use total memory as free memory
get_memory_info: [warning] ext_intel_free_memory is not supported (export/set ZES_ENABLE_SYSMAN=1 to support), use total memory as free memory
get_memory_info: [warning] ext_intel_free_memory is not supported (export/set ZES_ENABLE_SYSMAN=1 to support), use total memory as free memory
load_tensors: offloading output layer to GPU
load_tensors: offloading 31 repeating layers to GPU
load_tensors: offloaded 33/33 layers to GPU
load_tensors:   CPU_Mapped model buffer size =   220.34 MiB
load_tensors:        SYCL0 model buffer size =  1968.32 MiB
load_tensors:        SYCL1 model buffer size =  2072.41 MiB
..........................................................................................
common_init_result: added <|endoftext|> logit bias = -inf
common_init_result: added <|im_end|> logit bias = -inf
llama_context: constructing llama_context
llama_context: n_seq_max     = 1
llama_context: n_ctx         = 2048
llama_context: n_ctx_seq     = 2048
llama_context: n_batch       = 2048
llama_context: n_ubatch      = 512
llama_context: causal_attn   = 1
llama_context: flash_attn    = auto
llama_context: kv_unified    = false
llama_context: freq_base     = 500000.0
llama_context: freq_scale    = 0.125
llama_context: n_ctx_seq (2048) < n_ctx_train (65536) -- the full capacity of the model will not be utilized
Running with Environment Variables:
  GGML_SYCL_DEBUG: 0
  GGML_SYCL_DISABLE_OPT: 0
  GGML_SYCL_DISABLE_GRAPH: 1
  GGML_SYCL_DISABLE_DNN: DNN disabled by compile flag
  GGML_SYCL_PRIORITIZE_DMMV: 0
Build with Macros:
  GGML_SYCL_FORCE_MMQ: no
  GGML_SYCL_F16: no
Found 2 SYCL devices:
|  |                   |                                       |       |Max    |        |Max  |Global |                     |
|  |                   |                                       |       |compute|Max work|sub  |mem    |                     |
|ID|        Device Type|                                   Name|Version|units  |group   |group|size   |       Driver version|
|--|-------------------|---------------------------------------|-------|-------|--------|-----|-------|---------------------|
| 0|     [opencl:gpu:0]|                Intel Arc A770 Graphics|    3.0|    512|    1024|   32| 16225M|          25.48.36300|
| 1|     [opencl:gpu:1]|                Intel Arc A770 Graphics|    3.0|    512|    1024|   32| 16225M|          25.48.36300|
SYCL Optimization Feature:
|ID|        Device Type|Reorder|
|--|-------------------|-------|
| 0|     [opencl:gpu:0]|      Y|
| 1|     [opencl:gpu:1]|      Y|
llama_context:  SYCL_Host  output buffer size =     0.38 MiB
llama_kv_cache_iswa: creating non-SWA KV cache, size = 2048 cells
llama_kv_cache:      SYCL0 KV buffer size =   128.00 MiB
llama_kv_cache:      SYCL1 KV buffer size =   128.00 MiB
llama_kv_cache: size =  256.00 MiB (  2048 cells,   8 layers,  1/1 seqs), K (f16):  128.00 MiB, V (f16):  128.00 MiB
llama_kv_cache_iswa: creating     SWA KV cache, size = 2048 cells
llama_kv_cache:      SYCL0 KV buffer size =   416.00 MiB
llama_kv_cache:      SYCL1 KV buffer size =   352.00 MiB
llama_kv_cache: size =  768.00 MiB (  2048 cells,  24 layers,  1/1 seqs), K (f16):  384.00 MiB, V (f16):  384.00 MiB
get_memory_info: [warning] ext_intel_free_memory is not supported (export/set ZES_ENABLE_SYSMAN=1 to support), use total memory as free memory
get_memory_info: [warning] ext_intel_free_memory is not supported (export/set ZES_ENABLE_SYSMAN=1 to support), use total memory as free memory
llama_context: pipeline parallelism enabled (n_copies=4)
llama_context: layer 0 is assigned to device SYCL0 but the Flash Attention tensor is assigned to device CPU (usually due to missing support)
llama_context: Flash Attention was auto, set to disabled
llama_context:      SYCL0 compute buffer size =   232.07 MiB
llama_context:      SYCL1 compute buffer size =   267.93 MiB
llama_context:  SYCL_Host compute buffer size =    40.09 MiB
llama_context: graph nodes  = 1286
llama_context: graph splits = 3
common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
/scratch/llama/ggml/src/ggml-sycl/mmvq.cpp:811: GGML_ASSERT(block_num_y % num_subgroups == 0) failed
[New LWP 69604]
[New LWP 69599]

This GDB supports auto-downloading debuginfo from the following URLs:
  <https://debuginfod.archlinux.org>
Enable debuginfod for this session? (y or [n]) [answered N; input not from terminal]
Debuginfod has been disabled.
To make this setting permanent, add 'set debuginfod enabled off' to .gdbinit.
warning: File "/opt/intel/oneapi/compiler/2025.3/lib/libsycl.so.8.0.0-gdb.py" auto-loading has been declined by your `auto-load safe-path' set to "$debugdir:$datadir/auto-load".
To enable execution of this file add
	add-auto-load-safe-path /opt/intel/oneapi/compiler/2025.3/lib/libsycl.so.8.0.0-gdb.py
line to your configuration file "/home/nathan/.config/gdb/gdbinit".
To completely disable this security protection add
	set auto-load safe-path /
line to your configuration file "/home/nathan/.config/gdb/gdbinit".
For more information about this security protection see the
"Auto-loading safe path" section in the GDB manual.  E.g., run from the shell:
	info "(gdb)Auto-loading safe path"
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
0x00007f1af549f002 in ?? () from /usr/lib/libc.so.6
#0  0x00007f1af549f002 in ?? () from /usr/lib/libc.so.6
#1  0x00007f1af549316c in ?? () from /usr/lib/libc.so.6
#2  0x00007f1af54931b4 in ?? () from /usr/lib/libc.so.6
#3  0x00007f1af5503d8f in wait4 () from /usr/lib/libc.so.6
#4  0x00007f1af65605da in ggml_print_backtrace () from /scratch/llama/build/bin/libggml-base.so.0
#5  0x00007f1af655f749 in ggml_abort () from /scratch/llama/build/bin/libggml-base.so.0
#6  0x00007f1af5d0f705 in ggml_sycl_op_mul_mat_vec_q(ggml_backend_sycl_context&, ggml_tensor const*, ggml_tensor const*, ggml_tensor*, char const*, float const*, char const*, float*, long, long, long, long, sycl::_V1::queue* const&) () from /scratch/llama/build/bin/libggml-sycl.so.0
#7  0x00007f1af5c58102 in void ggml_sycl_op_mul_mat<quantize_and_reorder_q8_1_soa>(ggml_backend_sycl_context&, ggml_tensor const*, ggml_tensor const*, ggml_tensor*, void (*)(ggml_backend_sycl_context&, ggml_tensor const*, ggml_tensor const*, ggml_tensor*, char const*, float const*, char const*, float*, long, long, long, long, sycl::_V1::queue* const&)) () from /scratch/llama/build/bin/libggml-sycl.so.0
#8  0x00007f1af5c4ca1a in ggml_sycl_mul_mat(ggml_backend_sycl_context&, ggml_tensor const*, ggml_tensor const*, ggml_tensor*) () from /scratch/llama/build/bin/libggml-sycl.so.0
#9  0x00007f1af5c4b0de in ggml_backend_sycl_graph_compute_impl(ggml_backend_sycl_context*, ggml_cgraph*) () from /scratch/llama/build/bin/libggml-sycl.so.0
#10 0x00007f1af5c49137 in ggml_backend_sycl_graph_compute(ggml_backend*, ggml_cgraph*) () from /scratch/llama/build/bin/libggml-sycl.so.0
#11 0x00007f1af6584e29 in ggml_backend_sched_graph_compute_async () from /scratch/llama/build/bin/libggml-base.so.0
#12 0x00007f1af628b461 in llama_context::graph_compute(ggml_cgraph*, bool) () from /scratch/llama/build/bin/libllama.so.0
#13 0x00007f1af628b0cf in llama_context::process_ubatch(llama_ubatch const&, llm_graph_type, llama_memory_context_i*, ggml_status&) () from /scratch/llama/build/bin/libllama.so.0
#14 0x00007f1af628c92e in llama_context::decode(llama_batch const&) () from /scratch/llama/build/bin/libllama.so.0
#15 0x00007f1af6290cdb in llama_decode () from /scratch/llama/build/bin/libllama.so.0
#16 0x000000000052e2fd in common_init_from_params(common_params&) ()
#17 0x000000000041dde4 in main ()
[Inferior 1 (process 69598) detached]
[1]    69598 IOT instruction (core dumped)  llama-completion -hf unsloth/Olmo-3-7B-Instruct-GGUF:Q4_K_M --ctx-size 2048

Metadata

Metadata

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions