ggml_cuda_init: found 3 ROCm devices (Total VRAM: 98256 MiB):
Device 0: AMD Instinct MI60 / MI50, gfx906:sramecc+:xnack- (0x906), VMM: no, Wave Size: 64, VRAM: 32752 MiB
Device 1: AMD Instinct MI60 / MI50, gfx906:sramecc+:xnack- (0x906), VMM: no, Wave Size: 64, VRAM: 32752 MiB
Device 2: AMD Instinct MI60 / MI50, gfx906:sramecc+:xnack- (0x906), VMM: no, Wave Size: 64, VRAM: 32752 MiB
load_backend: loaded ROCm backend from /app/libggml-hip.so
load_backend: loaded CPU backend from /app/libggml-cpu-haswell.so
main: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
build_info: b8739-d132f22fc
system_info: n_threads = 32 (n_threads_batch = 32) / 64 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
Running without SSL
init: api_keys: *******
init: using 63 threads for HTTP server
start: binding port with default address family
main: loading model
srv load_model: loading model '/models/gemma-4-E2B-it-Q8_0.gguf'
common_init_result: fitting params to device memory, for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on
/app/ggml/src/ggml-backend.cpp:1365: GGML_ASSERT(n_inputs < GGML_SCHED_MAX_SPLIT_INPUTS) failed
libggml-base.so.0(+0x1ad1b)[0x735e7a73fd1b]
libggml-base.so.0(ggml_print_backtrace+0x21c)[0x735e7a74017c]
libggml-base.so.0(ggml_abort+0x15b)[0x735e7a74035b]
libggml-base.so.0(ggml_backend_sched_split_graph+0x223b)[0x735e7a75bf6b]
libggml-base.so.0(ggml_backend_sched_reserve_size+0x45)[0x735e7a75c0d5]
libllama.so.0(_ZN13llama_context13graph_reserveEjjjPK22llama_memory_context_ibPm+0x20b)[0x735e7a8b3fdb]
libllama.so.0(_ZN13llama_context13sched_reserveEv+0x5dc)[0x735e7a8b4b2c]
libllama.so.0(_ZN13llama_contextC2ERK11llama_model20llama_context_params+0xb69)[0x735e7a8b8ff9]
libllama.so.0(llama_init_from_model+0x138)[0x735e7a8b9b78]
libllama.so.0(+0x99730)[0x735e7a88d730]
libllama.so.0(+0x9a4b5)[0x735e7a88e4b5]
libllama.so.0(llama_params_fit+0x52)[0x735e7a891b72]
/app/llama-server(+0x253dc7)[0x576d796ccdc7]
/app/llama-server(+0x25503c)[0x576d796ce03c]
/app/llama-server(+0x167bce)[0x576d795e0bce]
/app/llama-server(+0xb3f52)[0x576d7952cf52]
/lib/x86_64-linux-gnu/libc.so.6(+0x2a1ca)[0x735e7a1a81ca]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x8b)[0x735e7a1a828b]
/app/llama-server(+0xbbc25)[0x576d79534c25]
Name and Version
$ ./llama-cli --version
version: 8739 (d132f22)
built with GNU 13.3.0 for Linux x86_64
Operating systems
Linux
GGML backends
HIP
Hardware
2x EPYC 7601 + 3x Mi50 32GB
Models
Gemma 4 E4B Q8_0 (unsloth, bartowski)
Gemma 4 E2B Q8_0 (unsloth, bartowski)
Problem description & steps to reproduce
Model fails to load with the following error:
/app/ggml/src/ggml-backend.cpp:1365: GGML_ASSERT(n_inputs < GGML_SCHED_MAX_SPLIT_INPUTS) failedllama-server config:
The larger Gemma 4 models load just fine, it's something specifically with the smaller ones.
First Bad Commit
No response
Relevant log output
Logs