Skip to content

Eval bug: Server freezes intermittently, /slots endpoint hangs, process ungraceful to stop #20921

Closed
Closed
Eval bug: Server freezes intermittently, /slots endpoint hangs, process ungraceful to stop#20921
Labels
bug-unconfirmed

Description

@mtms-eigen
mtms-eigen
opened on Mar 23, 2026

Name and Version

./build/bin/llama-server --version
ggml_cuda_init: found 7 CUDA devices (Total VRAM: 160593 MiB):
Device 0: NVIDIA GeForce RTX 3090, compute capability 8.6, VMM: yes, VRAM: 24124 MiB
Device 1: NVIDIA GeForce RTX 3090, compute capability 8.6, VMM: yes, VRAM: 24124 MiB
Device 2: NVIDIA GeForce RTX 3090, compute capability 8.6, VMM: yes, VRAM: 24124 MiB
Device 3: NVIDIA GeForce RTX 3090, compute capability 8.6, VMM: yes, VRAM: 24124 MiB
Device 4: NVIDIA GeForce RTX 3090, compute capability 8.6, VMM: yes, VRAM: 24124 MiB
Device 5: NVIDIA GeForce RTX 3090, compute capability 8.6, VMM: yes, VRAM: 24124 MiB
Device 6: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15847 MiB
version: 8468 (3306dba)
built with GNU 12.2.0 for Linux x86_64

Operating systems

Linux

GGML backends

CUDA

Hardware

6x RTX 3090 + 1 RTX 5060

Models

Qwen3.5-122B-A10B-GGUF-UD-Q6_K_XL
MiniMax-M2.5-MXFP4_MOE.gguf
MiniMax-M2.5-UD-Q4_K_XL.gguf
gpt-oss120.gguf
GLM-4.5-Air-GGUF_UD-Q4_K_XL.gguf

Problem description & steps to reproduce

The llama-server freezes occasionally. While the web interface remains reachable and the /health endpoint still responds with OK, the /slots endpoint hangs indefinitely without responding. New chat requests are not processed anymore.

The process cannot be stopped gracefully (e.g., via systemd stop or SIGTERM); it must be force-killed (kill -9).

### Models Tested:
    gpt-oss 120b
    Qwen3.5 122b (Unsloth Q4_K_XL)
    MiniMax M2.5 (Various Q4 quantizations)
    Multiple Unsloth Q4 models
    Issue persists across different models.

Steps to Reproduce

  • Start llama-server

  • Run requests via the Web Interface or API.

  • Wait for the issue to occur (intermittent/random).

  •     Note: The issue appears to occur more frequently when using the built-in llama.cpp Web Interface, specifically during reasoning tasks or in conjunction with Cancel actions.

  • Attempt to send a new chat request (fails).

  • Query /slots (endpoint hangs).

  • Query /health (returns OK).

  • Attempt to stop the process (fails to stop, requires kill -9).

Setup

The issue occurs with different setups (tested with -fit on and off, -np 1 and -np 2) and different model, but more frequently with Minimax and Qwen 122b.
Specially parallel requests with some stops and via Webinterface seems to trigger the bug

ExecStart=/opt/llamacpp/llama.cpp-13/build/bin/llama-server \ --host 0.0.0.0 \ -m /opt/models/MiniMax-M2.5-GGUF/MiniMax-M2.5-MXFP4_MOE.gguf \ --jinja \ --ctx-size 178000 \ -cb \ -np 1 \ -a gpt-oss-120b \ --temp 1.0 --top-p 0.95 --top-k 40 --min-p 0.01 \ -ngl 99 -fa on \ --cache-type-k q4_0 --cache-type-v q4_0 \ --fit-target 32 -ub 2048 -b 2048


ExecStart=/opt/llamacpp/llama.cpp/build/bin/llama-server \
    --host 0.0.0.0 \
    -m /opt/models/gpt-oss120.gguf \
    --ctx-size 220000 \
    --jinja \
    -ub 2048 -b 2048 \
    -np 2 \
    -a gpt-oss-120b \
    --temp 0.7 \
    --fit on

ExecStart=/opt/llamacpp/llama.cpp/build/bin/llama-server --host 0.0.0.0 --port 8080 -m /opt/models/Qwen/Qwen3.5-122B-A10B-GGUF-UD-Q6_K_XL/UD-Q6_K_XL/Qwen3.5-122B-A10B-UD-Q6_K_XL-00001-of-00004.gguf --jinja -cb -a gpt-oss-120b --temp 1.0 --top-p 0.95 --top-k 40 --min-p 0.01 -np 2 -fa on --fit on --ctx-size 400000 --mmproj /opt/models/Qwen/mmproj-122b-BF16.gguf --cache-ram 0 -v --threads-http 8 --props --metrics --slots -lv 4 --reasoning-format none

First Bad Commit

Cant say, but it happened occasionally since approx. 2 month, but in the past it did not happen so frequently with other models.

Relevant log output

Logs

Mar 23 19:43:38 ollama2 llama-server[114292]: res add_waiting_: add task 21585 to waiting list. current waiting = 1 (before add)
Mar 23 19:43:38 ollama2 llama-server[114292]: que post: new task, id = 21585/1, front = 0
Mar 23 19:43:39 ollama2 llama-server[114292]: srv log_server_r: done request: GET / 192.168.227.177 200
Mar 23 19:43:39 ollama2 llama-server[114292]: srv log_server_r: request:
Mar 23 19:43:39 ollama2 llama-server[114292]: srv log_server_r: response: Error: gzip is not supported by this browser
Mar 23 19:44:29 ollama2 llama-server[114292]: srv next: stopping wait for next result due to should_stop condition
Mar 23 19:44:29 ollama2 llama-server[114292]: srv stop: all tasks already finished, no need to cancel
Mar 23 19:44:29 ollama2 llama-server[114292]: res remove_waiti: remove task 21585 from waiting list. current waiting = 2 (before remove)
Mar 23 19:44:29 ollama2 llama-server[114292]: srv stop: cancel task, id_task = 21585
Mar 23 19:44:29 ollama2 llama-server[114292]: res remove_waiti: remove task 21585 from waiting list. current waiting = 1 (before remove)
Mar 23 19:44:29 ollama2 llama-server[114292]: que post: new task, id = 21586/1, front = 1
Mar 23 19:44:29 ollama2 llama-server[114292]: srv log_server_r: done request: POST /v1/chat/completions 192.168.237.141 200
Mar 23 19:44:29 ollama2 llama-server[114292]: srv log_server_r: request: {"messages":[{"role":"user","content":"hi"},{"role":"assistant","content":""},{"role":"user","content":"hi"}],"stream":true,"return_progress":true,"reasoning_format":"auto","temperature":1,"max_tokens":-1,"dynatemp_range":0,"dynatemp_exponent":1,"top_k":40,"top_p":0.95,"min_p":0.01,"xtc_probability":0,"xtc_threshold":0.1,"typ_p":1,"repeat_last_n":64,"repeat_penalty":1,"presence_penalty":0,"frequency_penalty":0,"dry_multiplier":0,"dry_base":1.75,"dry_allowed_length":2,"dry_penalty_last_n":-1,"samplers":["penalties","dry","top_n_sigma","top_k","typ_p","top_p","min_p","xtc","temperature"],"timings_per_token":true}
Mar 23 19:44:29 ollama2 llama-server[114292]: srv log_server_r: response:
Mar 23 19:44:30 ollama2 llama-server[114292]: srv stop: all tasks already finished, no need to cancel

-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.126.18 Driver Version: 580.126.18 CUDA Version: 13.0 |
+-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA GeForce RTX 3090 On | 00000000:00:10.0 Off | N/A |
| 0% 43C P8 17W / 350W | 22740MiB / 24576MiB | 0% Default |
+-----------------------------------------+------------------------+----------------------+
| 1 NVIDIA GeForce RTX 3090 On | 00000000:00:11.0 Off | N/A |
| 0% 36C P8 30W / 350W | 23034MiB / 24576MiB | 0% Default |
+-----------------------------------------+------------------------+----------------------+
| 2 NVIDIA GeForce RTX 3090 On | 00000000:00:1B.0 Off | N/A |
| 0% 37C P8 36W / 350W | 22836MiB / 24576MiB | 0% Default |
+-----------------------------------------+------------------------+----------------------+
| 3 NVIDIA GeForce RTX 3090 On | 00000000:00:1C.0 Off | N/A |
| 0% 53C P8 29W / 370W | 22728MiB / 24576MiB | 0% Default |
+-----------------------------------------+------------------------+----------------------+
| 4 NVIDIA GeForce RTX 3090 On | 00000000:03:0D.0 Off | N/A |
| 0% 34C P8 24W / 370W | 22732MiB / 24576MiB | 0% Default |
+-----------------------------------------+------------------------+----------------------+
| 5 NVIDIA GeForce RTX 3090 On | 00000000:03:0E.0 Off | N/A |
| 0% 34C P8 20W / 350W | 1MiB / 24576MiB | 0% Default |
+-----------------------------------------+------------------------+----------------------+
| 6 NVIDIA GeForce RTX 5060 Ti On | 00000000:03:0F.0 Off | N/A |
| 0% 46C P8 10W / 180W | 7897MiB / 16311MiB | 0% Default |
+-----------------------------------------+------------------------+----------------------+





Mar 23 19:43:38 ollama2 llama-server[114292]: common_chat_templates_apply_jinja: using differential autoparser

Mar 23 19:43:38 ollama2 llama-server[114292]: === Starting differential analysis ===

Mar 23 19:43:38 ollama2 llama-server[114292]: Phase 1: Reasoning analysis

Mar 23 19:43:38 ollama2 llama-server[114292]: Phase 2: Content analysis

Mar 23 19:43:38 ollama2 llama-server[114292]: Phase 3: Tool call analysis

Mar 23 19:43:38 ollama2 llama-server[114292]: Phase 4: Argument analysis

Mar 23 19:43:38 ollama2 llama-server[114292]:

Mar 23 19:43:38 ollama2 llama-server[114292]: --- Reasoning & Content Structure ---

Mar 23 19:43:38 ollama2 llama-server[114292]: reasoning_mode: TAG_BASED

Mar 23 19:43:38 ollama2 llama-server[114292]: reasoning_start: ''

Mar 23 19:43:38 ollama2 llama-server[114292]: reasoning_end: ''

Mar 23 19:43:38 ollama2 llama-server[114292]: content_mode: PLAIN

Mar 23 19:43:38 ollama2 llama-server[114292]: content_start: ''

Mar 23 19:43:38 ollama2 llama-server[114292]: content_end: ''

Mar 23 19:43:38 ollama2 llama-server[114292]: --- Tool Call Structure ---

Mar 23 19:43:38 ollama2 llama-server[114292]: tool_mode: TAG_WITH_TAGGED

Mar 23 19:43:38 ollama2 llama-server[114292]: supports_tools: true

Mar 23 19:43:38 ollama2 llama-server[114292]: supports_parallel_calls: true

Mar 23 19:43:38 ollama2 llama-server[114292]: tool_section_start: ''

Mar 23 19:43:38 ollama2 llama-server[114292]: tool_section_end: ''

Mar 23 19:43:38 ollama2 llama-server[114292]: per_call_start: '<tool_call>

Mar 23 19:43:38 ollama2 llama-server[114292]: '

Mar 23 19:43:38 ollama2 llama-server[114292]: per_call_end: '</tool_call>'

Mar 23 19:43:38 ollama2 llama-server[114292]: func_name_prefix: '<function='

Mar 23 19:43:38 ollama2 llama-server[114292]: func_name_suffix: '>

Mar 23 19:43:38 ollama2 llama-server[114292]: '

Mar 23 19:43:38 ollama2 llama-server[114292]: func_close: '

Mar 23 19:43:38 ollama2 llama-server[114292]: '

Mar 23 19:43:38 ollama2 llama-server[114292]: arg_name_prefix: '<parameter='

Mar 23 19:43:38 ollama2 llama-server[114292]: arg_name_suffix: '>

Mar 23 19:43:38 ollama2 llama-server[114292]: '

Mar 23 19:43:38 ollama2 llama-server[114292]: arg_value_prefix: ''

Mar 23 19:43:38 ollama2 llama-server[114292]: arg_value_suffix: '

Mar 23 19:43:38 ollama2 llama-server[114292]: '

Mar 23 19:43:38 ollama2 llama-server[114292]: name_field: 'name'

Mar 23 19:43:38 ollama2 llama-server[114292]: args_field: 'arguments'

Mar 23 19:43:38 ollama2 llama-server[114292]: id_field: ''

Mar 23 19:43:38 ollama2 llama-server[114292]: gen_id_field: ''

Mar 23 19:43:38 ollama2 llama-server[114292]: parameter_order: ''

Mar 23 19:43:38 ollama2 llama-server[114292]: === Differential analysis complete ===

Mar 23 19:43:38 ollama2 llama-server[114292]: common_chat_templates_apply_jinja: generated parser:

Mar 23 19:43:38 ollama2 llama-server[114292]: Sequence(Literal(<|im_start|>assistant

Mar 23 19:43:38 ollama2 llama-server[114292]: ), Repetition(Sequence(Literal(), Tag(reasoning, Until()), Literal(), Space), 0, 1), Tag(content, Until()), End)

Mar 23 19:43:38 ollama2 llama-server[114292]: parser generation prompt: <|im_start|>assistant

Mar 23 19:43:38 ollama2 llama-server[114292]: 

Mar 23 19:43:38 ollama2 llama-server[114292]: add_text: <|im_start|>user

Mar 23 19:43:38 ollama2 llama-server[114292]: hi<|im_end|>

Mar 23 19:43:38 ollama2 llama-server[114292]: <|im_start|>assistant

Mar 23 19:43:38 ollama2 llama-server[114292]: <|im_end|>

Mar 23 19:43:38 ollama2 llama-server[114292]: <|im_start|>user

Mar 23 19:43:38 ollama2 llama-server[114292]: hi<|im_end|>

Mar 23 19:43:38 ollama2 llama-server[114292]: <|im_start|>assistant

Mar 23 19:43:38 ollama2 llama-server[114292]: 

Mar 23 19:43:38 ollama2 llama-server[114292]: srv  params_from_: Grammar lazy: false

Mar 23 19:43:38 ollama2 llama-server[114292]: srv  params_from_: Chat format: peg-native

Mar 23 19:43:38 ollama2 llama-server[114292]: srv  params_from_: Generation prompt: '<|im_start|>assistant

Mar 23 19:43:38 ollama2 llama-server[114292]: 

Mar 23 19:43:38 ollama2 llama-server[114292]: '

Mar 23 19:43:38 ollama2 llama-server[114292]: srv  params_from_: Preserved token: 248068

Mar 23 19:43:38 ollama2 llama-server[114292]: srv  params_from_: Preserved token: 248069

Mar 23 19:43:38 ollama2 llama-server[114292]: srv  params_from_: Preserved token: 248058

Mar 23 19:43:38 ollama2 llama-server[114292]: srv  params_from_: Preserved token: 248059

Mar 23 19:43:38 ollama2 llama-server[114292]: srv  params_from_: Not preserved because more than 1 token: <function=

Mar 23 19:43:38 ollama2 llama-server[114292]: srv  params_from_: Preserved token: 29

Mar 23 19:43:38 ollama2 llama-server[114292]: srv  params_from_: Not preserved because more than 1 token: 

Mar 23 19:43:38 ollama2 llama-server[114292]: srv  params_from_: Not preserved because more than 1 token: <parameter=

Mar 23 19:43:38 ollama2 llama-server[114292]: srv  params_from_: Not preserved because more than 1 token: 

Mar 23 19:43:38 ollama2 llama-server[114292]: res  add_waiting_: add task 21585 to waiting list. current waiting = 1 (before add)

Mar 23 19:43:38 ollama2 llama-server[114292]: que          post: new task, id = 21585/1, front = 0

Mar 23 19:43:39 ollama2 llama-server[114292]: srv  log_server_r: done request: GET / 192.168.227.177 200

Mar 23 19:43:39 ollama2 llama-server[114292]: srv  log_server_r: request:

Mar 23 19:43:39 ollama2 llama-server[114292]: srv  log_server_r: response: Error: gzip is not supported by this browser

Mar 23 19:44:29 ollama2 llama-server[114292]: srv          next: stopping wait for next result due to should_stop condition

Mar 23 19:44:29 ollama2 llama-server[114292]: srv          stop: all tasks already finished, no need to cancel

Mar 23 19:44:29 ollama2 llama-server[114292]: res  remove_waiti: remove task 21585 from waiting list. current waiting = 2 (before remove)

Mar 23 19:44:29 ollama2 llama-server[114292]: srv          stop: cancel task, id_task = 21585

Mar 23 19:44:29 ollama2 llama-server[114292]: res  remove_waiti: remove task 21585 from waiting list. current waiting = 1 (before remove)

Mar 23 19:44:29 ollama2 llama-server[114292]: que          post: new task, id = 21586/1, front = 1

Mar 23 19:44:29 ollama2 llama-server[114292]: srv  log_server_r: done request: POST /v1/chat/completions 192.168.237.141 200

Mar 23 19:44:29 ollama2 llama-server[114292]: srv  log_server_r: request:  {"messages":[{"role":"user","content":"hi"},{"role":"assistant","content":""},{"role":"user","content":"hi"}],"stream":true,"return_progress":true,"reasoning_format":"auto","temperature":1,"max_tokens":-1,"dynatemp_range":0,"dynatemp_exponent":1,"top_k":40,"top_p":0.95,"min_p":0.01,"xtc_probability":0,"xtc_threshold":0.1,"typ_p":1,"repeat_last_n":64,"repeat_penalty":1,"presence_penalty":0,"frequency_penalty":0,"dry_multiplier":0,"dry_base":1.75,"dry_allowed_length":2,"dry_penalty_last_n":-1,"samplers":["penalties","dry","top_n_sigma","top_k","typ_p","top_p","min_p","xtc","temperature"],"timings_per_token":true}

Mar 23 19:44:29 ollama2 llama-server[114292]: srv  log_server_r: response:

Mar 23 19:44:30 ollama2 llama-server[114292]: srv          stop: all tasks already finished, no need to cancel
Metadata
Metadata
Assignees
No one assigned
    Labels
    bug-unconfirmed
    Type
    No type
    Projects
    No projects
    Milestone
    No milestone
    Relationships
    None yet
    Development
    No branches or pull requests
    Issue actions