Eval bug: image-min/max-tokens parameter setting cause IOT instruction (core dumped) when hosting Gemma 4 models

[huhanwj]

Name and Version

./llama-cli --version
ggml_cuda_init: found 1 CUDA devices (Total VRAM: 21997 MiB):
Device 0: NVIDIA GeForce RTX 2080 Ti, compute capability 7.5, VMM: yes, VRAM: 21997 MiB
version: 8667 (c08d28d)
built with GNU 15.2.1 for Linux x86_64

Operating systems

Linux

GGML backends

CUDA

Hardware

i5 12600KF + RTX 2080Ti 22G

Models

unsloth/gemma-4-E4B-it-GGUF:BF16

Problem description & steps to reproduce

When I try to host ./llama-server -hf unsloth/gemma-4-E4B-it-GGUF:BF16 --host 0.0.0.0 -ngl 99 -n 20480 --top_p 0.95 --top_k 64 --image-max-tokens 1200, the image min/max tokens will cause problem and will end in an seg fault .

The model runs fine without setting this parameter, but seems to use a low budget in image tokens, and this seems to decrease the multimodal performance. I don't know whether this is the parameter parsing issue or something else. As gemma4 should naturally support variable token budgets for images. As they write in their readme: The supported token budgets are: 70, 140, 280, 560, and 1120.

First Bad Commit

No response

Relevant log output

Logs

decoding image batch 1/1, n_tokens_batch = 735
/home/huhan/llama.cpp/src/llama-context.cpp:1597: GGML_ASSERT((cparams.causal_attn || cparams.n_ubatch >= n_tokens_all) && "non-causal attention requires n_ubatch >= n_tokens") failed
[New LWP 335059]
[New LWP 335058]
[New LWP 335057]
[New LWP 335056]
[New LWP 335055]
[New LWP 335054]
[New LWP 335053]
[New LWP 335052]
[New LWP 335051]
[New LWP 335050]
[New LWP 335049]
[New LWP 335048]
[New LWP 335047]
[New LWP 335046]
[New LWP 335045]
[New LWP 335044]
[New LWP 335043]
[New LWP 335042]
[New LWP 335039]
[New LWP 335033]

This GDB supports auto-downloading debuginfo from the following URLs:
  <ima:enforcing>
  <https://debuginfod.fedoraproject.org/>
  <ima:ignore>
Enable debuginfod for this session? (y or [n]) [answered N; input not from terminal]
Debuginfod has been disabled.
To make this setting permanent, add 'set debuginfod enabled off' to .gdbinit.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
0x00007f174ae879a2 in __syscall_cancel_arch () from /lib64/libc.so.6
#0  0x00007f174ae879a2 in __syscall_cancel_arch () from /lib64/libc.so.6
#1  0x00007f174ae7bc3c in __internal_syscall_cancel () from /lib64/libc.so.6
#2  0x00007f174ae7bc84 in __syscall_cancel () from /lib64/libc.so.6
#3  0x00007f174aeebb8f in wait4 () from /lib64/libc.so.6
#4  0x00007f175435f023 in ggml_print_backtrace () from libggml-base.so.0
#5  0x00007f175435f16f in ggml_abort () from libggml-base.so.0
#6  0x00007f1753a61dc7 in llama_context::decode(llama_batch const&) () from libllama.so.0
#7  0x00007f1753a61f2e in llama_decode () from libllama.so.0
#8  0x00007f1753da19da in mtmd_helper_decode_image_chunk () from libmtmd.so.0
#9  0x00007f1753da2ee8 in mtmd_helper_eval_chunk_single () from libmtmd.so.0
#10 0x00000000004cb1ff in server_tokens::process_chunk(llama_context*, mtmd_context*, unsigned long, int, int, unsigned long&) const ()
#11 0x000000000052825e in server_context_impl::update_slots() ()
#12 0x00000000005ae4ff in server_queue::start_loop(long) ()
#13 0x0000000000487010 in main ()
[Inferior 1 (process 335030) detached]
^C^C[1]    335030 IOT instruction (core dumped)  ./llama-server -hf unsloth/gemma-4-E4B-it-GGUF:BF16 --host 0.0.0.0 -ngl 99 -n

[wqerrewetw]

Same problem:#21427

You can add the -ub 1024 parameter to temporarily bypass the error.

[huhanwj]

Same problem:#21427

You can add the -ub 1024 parameter to temporarily bypass the error.

Thanks, will have a try.

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale.