opencode + llama-swap + Qwen3-Coder-30B-A3B-Instruct-IQ4_NL

[grigio]

Description

The same llama-swap api works with OpenWebUI but not with opencode

llama-swap  | prompt eval time =   10519.50 ms /   198 tokens (   53.13 ms per token,    18.82 tokens per second)
llama-swap  |        eval time =     204.17 ms /     3 tokens (   68.06 ms per token,    14.69 tokens per second)
llama-swap  |       total time =   10723.67 ms /   201 tokens
llama-swap  | slot      release: id  0 | task 32 | stop processing: n_past = 201, truncated = 0
llama-swap  | srv  update_slots: all slots are idle
llama-swap  | srv  log_server_r: request: POST /v1/chat/completions 127.0.0.1 200
llama-swap  | [INFO] Request 192.168.1.83 "POST /v1/chat/completions HTTP/1.1" 200 601 "ai-sdk/openai-compatible/1.0.24 ai-sdk/provider-utils/3.0.14 runtime/bun/1.3.0" 10.770528487s
llama-swap  | got exception: {"code":500,"message":"Value is not callable: null at row 62, column 114:\n                {%- if json_key not in handled_keys %}\n                    {%- set normed_json_key = json_key | replace(\"-\", \"_\") | replace(\" \", \"_\") | replace(\"$\", \"\") %}\n                                                                                                                 ^\n                    {%- if param_fields[json_key] is mapping %}\n at row 62, column 21:\n                {%- if json_key not in handled_keys %}\n                    {%- set normed_json_key = json_key | replace(\"-\", \"_\") | replace(\" \", \"_\") | replace(\"$\", \"\") %}\n                    ^\n                    {%- if param_fields[json_key] is mapping %}\n at row 61, column 55:\n            {%- for json_key in param_fields %}\n                {%- if json_key not in handled_keys %}\n                                                      ^\n                    {%- set normed_json_key = json_key | replace(\"-\", \"_\") | replace(\" \", \"_\") | replace(\"$\", \"\") %}\n at row 61, column 17:\n            {%- for json_key in param_fields %}\n                {%- if json_key not in handled_keys %}\n                ^\n                    {%- set normed_json_key = json_key | replace(\"-\", \"_\") | replace(\" \", \"_\") | replace(\"$\", \"\") %}\n at row 60, column 48:\n            {%- set handled_keys = ['type', 'description', 'enum', 'required'] %}\n            {%- for json_key in param_fields %}\n                                               ^\n                {%- if json_key not in handled_keys %}\n at row 60, column 13:\n            {%- set handled_keys = ['type', 'description', 'enum', 'required'] %}\n            {%- for json_key in param_fields %}\n            ^\n                {%- if json_key not in handled_keys %}\n at row 49, column 80:\n        {{- '\\n<parameters>' }}\n        {%- for param_name, param_fields in tool.parameters.properties|items %}\n                                                                               ^\n            {{- '\\n<parameter>' }}\n at row 49, column 9:\n        {{- '\\n<parameters>' }}\n        {%- for param_name, param_fields in tool.parameters.properties|items %}\n        ^\n            {{- '\\n<parameter>' }}\n at row 42, column 29:\n    {{- \"<tools>\" }}\n    {%- for tool in tools %}\n                            ^\n        {%- if tool.function is defined %}\n at row 42, column 5:\n    {{- \"<tools>\" }}\n    {%- for tool in tools %}\n    ^\n        {%- if tool.function is defined %}\n at row 39, column 51:\n{%- endif %}\n{%- if tools is iterable and tools | length > 0 %}\n                                                  ^\n    {{- \"\\n\\nYou have access to the following functions:\\n\\n\" }}\n at row 39, column 1:\n{%- endif %}\n{%- if tools is iterable and tools | length > 0 %}\n^\n    {{- \"\\n\\nYou have access to the following functions:\\n\\n\" }}\n at row 1, column 69:\n{#- Copyright 2025-present the Unsloth team. All rights reserved. #}\n                                                                    ^\n{#- Licensed under the Apache License, Version 2.0 (the \"License\") #}\n","type":"server_error"}
llama-swap  | srv  log_server_r: request: POST /v1/chat/completions 127.0.0.1 500
llama-swap  | [INFO] Request 192.168.1.83 "POST /v1/chat/completions HTTP/1.1" 500 3198 "ai-sdk/openai-compatible/1.0.24 ai-sdk/provider-utils/3.0.14 runtime/bun/1.3.0" 7.46044ms

OpenCode version

No response

Steps to reproduce

No response

Screenshot and/or share link

No response

Operating System

No response

Terminal

No response

[github-actions]

This issue might be a duplicate of existing issues. Please check:

• OpenCode sends tools (and Jinja tool template) to llama.cpp results in 500 error unless --jinja; with --jinja, template crashes (reject filter) #1890: Very similar issue with llama.cpp + Qwen3 Coder models experiencing "Value is not callable" templating errors when using --jinja flag. The error traces are nearly identical and both involve the same param_fields templating issue.
• Upload images / screenshots with vision capable models #434: Also reports "OpenAI API Streaming Error: 500 Unknown method llama.cpp with unsloth/Qwen3-Coder-480B-A35B-Instruct-GGUF" with similar Jinja template errors.

Feel free to ignore if none of these address your specific case.

[rekram1-node]

I think I need more info to help, can you share your opencode.json?

[grigio]

opencode seems to connect fine to the openapi-compatible api but then it spins forever

opencode.json

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "ollama": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "llamacpp (local)",
      "options": {
        "baseURL": "http://server.lan:9292/v1"
      },
      "models": {
        "gpt-oss-20b-mxfp4-low": {
          "name": "gpt-oss-20b-mxfp4-low"
        },
        "Qwen3-4B-Instruct-2507-Q5_K_M": {
          "name": "Qwen3-4B-Instruct-2507-Q5_K_M"
        },
        "Qwen3-Coder-30B-A3B-Instruct-IQ4_NL": {
          "name": "Qwen3-Coder-30B-A3B-Instruct-IQ4_NL"
        }
      }
    }
  },
  "mcp": {
    "searxng": {
      "type": "local",
      // Or ["bun", "x", "my-mcp-command"]
      "command": ["uvx","mcp-searxng"],
      "enabled": true,
      "environment": {
        "SEARXNG_URL": "http://server.lan:3127"
      }
    }
  }
}

qwen3 30b a3b

llama-swap  | main: server is listening on http://0.0.0.0:9999 - starting the main loop
llama-swap  | srv  update_slots: all slots are idle
llama-swap  | srv  log_server_r: request: GET /health 127.0.0.1 200
llama-swap  | [INFO] <Qwen3-Coder-30B-A3B-Instruct-IQ4_NL> Health check passed on http://127.0.0.1:9999/health
llama-swap  | srv  params_from_: Chat format: Hermes 2 Pro
llama-swap  | slot get_availabl: id  0 | task -1 | selected slot by LRU, t_last = -1
llama-swap  | slot launch_slot_: id  0 | task 0 | processing task
llama-swap  | slot update_slots: id  0 | task 0 | new prompt, n_ctx_slot = 32768, n_keep = 0, n_prompt_tokens = 199
llama-swap  | slot update_slots: id  0 | task 0 | n_past = 0, memory_seq_rm [0, end)
llama-swap  | slot update_slots: id  0 | task 0 | prompt processing progress, n_past = 199, n_tokens = 199, progress = 1.000000
llama-swap  | slot update_slots: id  0 | task 0 | prompt done, n_past = 199, n_tokens = 199
llama-swap  | [INFO] Request ::1 "GET / HTTP/1.1" 302 26 "curl/8.5.0" 183.424µs
llama-swap  | slot print_timing: id  0 | task 0 | 
llama-swap  | prompt eval time =   10636.29 ms /   199 tokens (   53.45 ms per token,    18.71 tokens per second)
llama-swap  |        eval time =     211.42 ms /     3 tokens (   70.47 ms per token,    14.19 tokens per second)
llama-swap  |       total time =   10847.71 ms /   202 tokens
llama-swap  | slot      release: id  0 | task 0 | stop processing: n_past = 201, truncated = 0
llama-swap  | srv  update_slots: all slots are idle
llama-swap  | srv  log_server_r: request: POST /v1/chat/completions 127.0.0.1 200
llama-swap  | [INFO] Request 192.168.1.83 "POST /v1/chat/completions HTTP/1.1" 200 600 "ai-sdk/openai-compatible/1.0.24 ai-sdk/provider-utils/3.0.14 runtime/bun/1.3.0" 36.343893322s
llama-swap  | got exception: {"code":500,"message":"Value is not callable: null at row 62, column 114:\n                {%- if json_key not in handled_keys %}\n                    {%- set normed_json_key = json_key | replace(\"-\", \"_\") | replace(\" \", \"_\") | replace(\"$\", \"\") %}\n                                                                                                                 ^\n                    {%- if param_fields[json_key] is mapping %}\n at row 62, column 21:\n                {%- if json_key not in handled_keys %}\n                    {%- set normed_json_key = json_key | replace(\"-\", \"_\") | replace(\" \", \"_\") | replace(\"$\", \"\") %}\n                    ^\n                    {%- if param_fields[json_key] is mapping %}\n at row 61, column 55:\n            {%- for json_key in param_fields %}\n                {%- if json_key not in handled_keys %}\n                                                      ^\n                    {%- set normed_json_key = json_key | replace(\"-\", \"_\") | replace(\" \", \"_\") | replace(\"$\", \"\") %}\n at row 61, column 17:\n            {%- for json_key in param_fields %}\n                {%- if json_key not in handled_keys %}\n                ^\n                    {%- set normed_json_key = json_key | replace(\"-\", \"_\") | replace(\" \", \"_\") | replace(\"$\", \"\") %}\n at row 60, column 48:\n            {%- set handled_keys = ['type', 'description', 'enum', 'required'] %}\n            {%- for json_key in param_fields %}\n                                               ^\n                {%- if json_key not in handled_keys %}\n at row 60, column 13:\n            {%- set handled_keys = ['type', 'description', 'enum', 'required'] %}\n            {%- for json_key in param_fields %}\n            ^\n                {%- if json_key not in handled_keys %}\n at row 49, column 80:\n        {{- '\\n<parameters>' }}\n        {%- for param_name, param_fields in tool.parameters.properties|items %}\n                                                                               ^\n            {{- '\\n<parameter>' }}\n at row 49, column 9:\n        {{- '\\n<parameters>' }}\n        {%- for param_name, param_fields in tool.parameters.properties|items %}\n        ^\n            {{- '\\n<parameter>' }}\n at row 42, column 29:\n    {{- \"<tools>\" }}\n    {%- for tool in tools %}\n                            ^\n        {%- if tool.function is defined %}\n at row 42, column 5:\n    {{- \"<tools>\" }}\n    {%- for tool in tools %}\n    ^\n        {%- if tool.function is defined %}\n at row 39, column 51:\n{%- endif %}\n{%- if tools is iterable and tools | length > 0 %}\n                                                  ^\n    {{- \"\\n\\nYou have access to the following functions:\\n\\n\" }}\n at row 39, column 1:\n{%- endif %}\n{%- if tools is iterable and tools | length > 0 %}\n^\n    {{- \"\\n\\nYou have access to the following functions:\\n\\n\" }}\n at row 1, column 69:\n{#- Copyright 2025-present the Unsloth team. All rights reserved. #}\n                                                                    ^\n{#- Licensed under the Apache License, Version 2.0 (the \"License\") #}\n","type":"server_error"}
llama-swap  | srv  log_server_r: request: POST /v1/chat/completions 127.0.0.1 500
llama-swap  | [INFO] Request 192.168.1.83 "POST /v1/chat/completions HTTP/1.1" 500 3198 "ai-sdk/openai-compatible/1.0.24 ai-sdk/provider-utils/3.0.14 runtime/bun/1.3.0" 36.345012722s

gpt-oss 20B

llama-swap  | slot print_timing: id  0 | task 33 | 
llama-swap  | prompt eval time =    9832.01 ms /   100 tokens (   98.32 ms per token,    10.17 tokens per second)
llama-swap  |        eval time =    7247.55 ms /    54 tokens (  134.21 ms per token,     7.45 tokens per second)
llama-swap  |       total time =   17079.56 ms /   154 tokens
llama-swap  | slot      release: id  0 | task 33 | stop processing: n_past = 215, truncated = 0
llama-swap  | srv  update_slots: all slots are idle
llama-swap  | srv  log_server_r: request: POST /v1/chat/completions 127.0.0.1 200
llama-swap  | [INFO] Request 172.28.0.1 "POST /v1/chat/completions HTTP/1.1" 200 781 "Python/3.11 aiohttp/3.12.15" 17.092465058s
llama-swap  | [INFO] Request 172.28.0.1 "GET /v1/models HTTP/1.1" 200 3415 "Python/3.11 aiohttp/3.12.15" 72.627µs
llama-swap  | srv  params_from_: Chat format: GPT-OSS
llama-swap  | slot get_availabl: id  0 | task -1 | selected slot by LCP similarity, sim_best = 0.261 (> 0.100 thold), f_keep = 0.298
llama-swap  | srv  get_availabl: updating prompt cache
llama-swap  | srv   prompt_save:  - saving prompt with length 215, total state size = 10.084 MiB
llama-swap  | srv          load:  - looking for better prompt, base f_keep = 0.298, sim = 0.261
llama-swap  | srv        update:  - cache state: 1 prompts, 12.382 MiB (limits: 8192.000 MiB, 4096 tokens, 142246 est)
llama-swap  | srv        update:    - prompt 0x5652226034e0:     215 tokens, checkpoints:  1,    12.382 MiB
llama-swap  | srv  get_availabl: prompt cache update took 6.57 ms
llama-swap  | slot launch_slot_: id  0 | task 89 | processing task
llama-swap  | slot update_slots: id  0 | task 89 | new prompt, n_ctx_slot = 4096, n_keep = 0, n_prompt_tokens = 245
llama-swap  | slot update_slots: id  0 | task 89 | old: ... <|end|><|start|>user<|message|> | Here is the query:
llama-swap  | cia
llama-swap  | slot update_slots: id  0 | task 89 | new: ... <|end|><|start|>user<|message|> | ### Task:
llama-swap  | Generate 1
llama-swap  | slot update_slots: id  0 | task 89 |   200007  200006    1428  200008   12253     382     290    5703     734    3565
llama-swap  | slot update_slots: id  0 | task 89 |   200007  200006    1428  200008   31639   10148     734   40569     220      16
llama-swap  | slot update_slots: id  0 | task 89 | n_past = 64, memory_seq_rm [64, end)
llama-swap  | slot update_slots: id  0 | task 89 | prompt processing progress, n_past = 181, n_tokens = 117, progress = 0.738775
llama-swap  | slot update_slots: id  0 | task 89 | n_past = 181, memory_seq_rm [181, end)
llama-swap  | slot update_slots: id  0 | task 89 | prompt processing progress, n_past = 245, n_tokens = 64, progress = 1.000000
llama-swap  | slot update_slots: id  0 | task 89 | prompt done, n_past = 245, n_tokens = 64
llama-swap  | slot update_slots: id  0 | task 89 | created context checkpoint 2 of 8 (pos_min = 0, pos_max = 180, size = 4.245 MiB)
llama-swap  | [INFO] Request ::1 "GET / HTTP/1.1" 302 26 "curl/8.5.0" 25.367µs
llama-swap  | slot print_timing: id  0 | task 89 | 
llama-swap  | prompt eval time =   11523.72 ms /   181 tokens (   63.67 ms per token,    15.71 tokens per second)
llama-swap  |        eval time =    3702.47 ms /    28 tokens (  132.23 ms per token,     7.56 tokens per second)
llama-swap  |       total time =   15226.19 ms /   209 tokens
llama-swap  | slot      release: id  0 | task 89 | stop processing: n_past = 272, truncated = 0
llama-swap  | srv  update_slots: all slots are idle
llama-swap  | srv  log_server_r: request: POST /v1/chat/completions 127.0.0.1 200
llama-swap  | [INFO] Request 172.28.0.1 "POST /v1/chat/completions HTTP/1.1" 200 685 "Python/3.11 aiohttp/3.12.15" 15.237551716s
llama-swap  | srv  params_from_: Chat format: GPT-OSS
llama-swap  | slot get_availabl: id  0 | task -1 | selected slot by LCP similarity, sim_best = 0.241 (> 0.100 thold), f_keep = 0.228
llama-swap  | srv  get_availabl: updating prompt cache
llama-swap  | srv   prompt_save:  - saving prompt with length 272, total state size = 12.757 MiB
llama-swap  | srv          load:  - looking for better prompt, base f_keep = 0.228, sim = 0.241
llama-swap  | srv        update:  - cache state: 2 prompts, 31.682 MiB (limits: 8192.000 MiB, 4096 tokens, 125925 est)
llama-swap  | srv        update:    - prompt 0x5652226034e0:     215 tokens, checkpoints:  1,    12.382 MiB
llama-swap  | srv        update:    - prompt 0x5652226036c0:     272 tokens, checkpoints:  2,    19.300 MiB
llama-swap  | srv  get_availabl: prompt cache update took 7.29 ms
llama-swap  | slot launch_slot_: id  0 | task 119 | processing task
llama-swap  | slot update_slots: id  0 | task 119 | new prompt, n_ctx_slot = 4096, n_keep = 0, n_prompt_tokens = 257
llama-swap  | slot update_slots: id  0 | task 119 | old: ...  message.<|end|><|start|> | user<|message|>### Task:
llama-swap  | Generate
llama-swap  | slot update_slots: id  0 | task 119 | new: ...  message.<|end|><|start|> | developer<|message|># Instructions
llama-swap  | 
llama-swap  | You
llama-swap  | slot update_slots: id  0 | task 119 |     3176      13  200007  200006    1428  200008   31639   10148     734   40569
llama-swap  | slot update_slots: id  0 | task 119 |     3176      13  200007  200006   77944  200008       2   68406     279    3575
llama-swap  | slot update_slots: id  0 | task 119 | n_past = 62, memory_seq_rm [62, end)
llama-swap  | slot update_slots: id  0 | task 119 | prompt processing progress, n_past = 193, n_tokens = 131, progress = 0.750973
llama-swap  | srv  log_server_r: request: POST /v1/chat/completions 127.0.0.1 400
llama-swap  | [INFO] Request 192.168.1.83 "POST /v1/chat/completions HTTP/1.1" 400 171 "ai-sdk/openai-compatible/1.0.24 ai-sdk/provider-utils/3.0.14 runtime/bun/1.3.0" 51.505583ms
llama-swap  | slot update_slots: id  0 | task 119 | n_past = 193, memory_seq_rm [193, end)
llama-swap  | slot update_slots: id  0 | task 119 | prompt processing progress, n_past = 257, n_tokens = 64, progress = 1.000000
llama-swap  | slot update_slots: id  0 | task 119 | prompt done, n_past = 257, n_tokens = 64
llama-swap  | slot print_timing: id  0 | task 119 | 
llama-swap  | prompt eval time =   11580.04 ms /   195 tokens (   59.38 ms per token,    16.84 tokens per second)
llama-swap  |        eval time =    2613.09 ms /    20 tokens (  130.65 ms per token,     7.65 tokens per second)
llama-swap  |       total time =   14193.13 ms /   215 tokens
llama-swap  | slot      release: id  0 | task 119 | stop processing: n_past = 276, truncated = 0
llama-swap  | srv  update_slots: all slots are idle
llama-swap  | srv  log_server_r: request: POST /v1/chat/completions 127.0.0.1 200
llama-swap  | [INFO] Request 192.168.1.83 "POST /v1/chat/completions HTTP/1.1" 200 689 "ai-sdk/openai-compatible/1.0.24 ai-sdk/provider-utils/3.0.14 runtime/bun/1.3.0" 14.205691081s
llama-swap  | [INFO] Request ::1 "GET / HTTP/1.1" 302 26 "curl/8.5.0" 23.575µs
llama-swap  | [INFO] Request 172.28.0.1 "GET /v1/models HTTP/1.1" 200 3415 "Python/3.11 aiohttp/3.12.15" 76.815µs
llama-swap  | [INFO] Request ::1 "GET / HTTP/1.1" 302 26 "curl/8.5.0" 22.533µs

[grigio]

DUPL #1890