Name and Version
llama-cli --version
ggml_cuda_init: found 3 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 2: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
version: 23 (44008ce)
built with GNU 14.2.0 for Linux x86_64
Operating systems
Linux
Which llama.cpp modules do you know to be affected?
llama-server
Command line
llama-server
--verbose
--sleep-idle-seconds 900
--device ROCm0,ROCm1,ROCm2
--cache-ram -1
--cache-type-k q4_1
--cache-type-v q4_1
--slot-save-path /data/ai/cache
--swa-full
--flash-attn on
--kv-offload
--kv-unified
--fit on
--fit-ctx 131072
--ctx-size 131072
--split-mode layer
--tensor-split 1,1,1
--models-dir /data/ai/text
--models-max 1
--model /data/ai/text/Qwen3-Coder-Next-UD-Q8_K_XL-00001-of-00002.gguf
--temp 1.0
--top-p 0.95
--top-k 40
--min-p 0.01
--override-tensor ".ffn_(up)_exps.=CPU"
--direct-io
--alias llama
--api-key <redacted>
--host 0.0.0.0
--port 5000
--rerank
--jinja
Problem description & steps to reproduce
While the sleep mode of llama-server does indeed unload the model from VRAM, nvtop reports that my GPUs remain at 99% load and consuming 100W of their max 300W, which is the same as if it were loaded and idle (also disappointing). The machine is headless and has no other workloads of any kind to account for this load and power consumption. Only fully stopping llama-server returns load and power consumption to baseline.
What information would be useful for me to gather?
First Bad Commit
No response
Relevant log output
Logs
Name and Version
Operating systems
Linux
Which llama.cpp modules do you know to be affected?
llama-server
Command line
Problem description & steps to reproduce
While the sleep mode of
llama-serverdoes indeed unload the model from VRAM,nvtopreports that my GPUs remain at 99% load and consuming 100W of their max 300W, which is the same as if it were loaded and idle (also disappointing). The machine is headless and has no other workloads of any kind to account for this load and power consumption. Only fully stoppingllama-serverreturns load and power consumption to baseline.What information would be useful for me to gather?
First Bad Commit
No response
Relevant log output
Logs