Cannot use tools with qwen2.5-coder and Ollama

[leonardorame]

Hi!, I'm using opencode v0.11.1 with the qwen2.5-coder:7b model via Ollama and it looks like cannot execute external operations, for example:

The task is:
create a file test.sh with one instruction: ls -lah

And the response is:

{ "name": "Write", "arguments": { "file_path": "test.sh", content: "..."}}

And the file wasn't created.

My config is this:

cat ~/.config/opencode/opencode.json 
{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "ollama": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Ollama (local)",
      "options": {
        "baseURL": "http://localhost:11434/v1"
      },
      "models": {
        "qwen2.5-coder:7b-instruct": {
          "name": "qwen2.5-coder"
        }
      }
    }
  }
}

[rekram1-node]

@leonardorame I think this issue may be relevant for you: #1068

[rekram1-node]

Looks like:

I found out why agentic actions such as using opencode's provided tools didn't work. It's because Ollama has set the context window at 4096 for the models.

In ollama if you set:
Set parameter 'num_ctx' to '16384'

it seems to have fixed it...

I can add this to docs once we resolve your issue

[leonardorame]

Hi @rekram1-node, I don't know where to put that parameter:

I tried in opencode.json, in both ollama -> options, and ollama -> models, with quotes and without it, but it seems to be ignored by ollama:

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "ollama": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Ollama (local)",
      "options": {
        "baseURL": "http://localhost:11434/v1",
        "num_ctx": 16384
      },
      "models": {
        "qwen2.5-coder:7b-instruct": {
          "name": "qwen2.5-coder",
          "num_ctx": 16384
        }
      }
    }
  }
}

If I run ollama ps I get:

NAME                         ID              SIZE      PROCESSOR          CONTEXT    UNTIL               
qwen2.5-coder:7b-instruct    dae161e27b0e    6.0 GB    10%/90% CPU/GPU    4096       59 minutes from now 

[leonardorame]

For the record:

Create a file named, for example modelfile (in any directory) with this content:

FROM qwen2.5-coder:7b-instruct
PARAMETER num_ctx   8192
PARAMETER num_batch 384
PARAMETER num_thread 12

Run: ollama create qwen2.5-coder-7b-8k -f modelfile (of course, the modelfile must be reachable by ollama, if not on the same directory use the absolute path i.e. /home/leonardo/modelfiles/modelfile).

gathering model components 
using existing layer sha256:60e05f2100071479f596b964f89f510f057ce397ea22f2833a0cfe029bfc2463 
using existing layer sha256:66b9ea09bd5b7099cbb4fc820f31b575c0366fa439b08245566692c6784e281e 
using existing layer sha256:1e65450c30670713aa47fe23e8b9662bdf4065e81cc8e3cbfaa98924fcc0d320 
using existing layer sha256:832dd9e00a68dd83b3c3fb9f5588dad7dcf337a0db50f7d9483f310cd292e92e 
using existing layer sha256:9b2154434353e33e707107d2c1a7b7c69023aa85d8545ba3e6975a8027e406dc 
writing manifest 
success 

Then, to check, if you run ollama ls you'll get:

NAME                                                  ID              SIZE       MODIFIED      
qwen2.5-coder-7b-8k:latest                            98181fbdef2d    4.7 GB     2 minutes ago 
....

Adapt opencode.json:

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "ollama": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Ollama (local)",
      "options": {
        "baseURL": "http://localhost:11434/v1"
      },
      "models": {
        "qwen2.5-coder-7b-8k": {
          "name": "qwen2.5-coder"
        }
      }
    }
  }
}

The next time you run opencode, if you execute ollama ps you'll get:

ollama ps
NAME                          ID              SIZE      PROCESSOR          CONTEXT    UNTIL               
qwen2.5-coder-7b-8k:latest    98181fbdef2d    6.2 GB    13%/87% CPU/GPU    8192       59 minutes from now   

[leonardorame]

I tested it with a context of 8k and 16k, but none of these worked. I still get the instructions to create my file, in json format, when I need opencode to create it by itself.

[leonardorame]

This worked!!: https://github.com/p-lemonish/ollama-x-opencode

[rekram1-node]

sorry I should’ve communicated it better I was trying to relay info from that ^^

We should update our docs to make this easier for others

[Atomic-Germ]

Mine is now so concerned it suggested the crisis line. I'm starting to think the tool using is a gaslight.

[rekram1-node]

@Atomic-Germ can you elaborate what you mean?