Ollama models take +50% memory when called by Continue

[ShaunaGordon]

Before submitting your bug report

• I've tried using the "Ask AI" feature on the Continue docs site to see if the docs have an answer
• I believe this is a bug. I'll try to join the Continue Discord for questions
• I'm not able to find an open issue that reports the same bug
• I've seen the troubleshooting guide on the Continue Docs

Relevant environment info

- OS: Arch Linux (EndeavourOS)
- GPU: Radeon 7900XTX 24gb vram
- System: Ryzen 9 9950X3D, 64gb system ram
- Continue version: 1.2.1
- IDE version: VSCode 1.103.2
- Ollama version: 0.11.8
- Model: Any
- config:

%YAML 1.1
---
name: Notes Assistant
version: 1.0.0
schema: v1

context:
  - provider: code
  - provider: codebase
  - provider: currentFile
  - provider: diff
  - provider: docs
  - provider: folder
  - provider: open
  - provider: problems
  - provider: search
  - provider: terminal

rules:
  - You are a friendly assistant whose purpose is to analyze a Zettelkasten style "second brain" notes collection to do things like find patterns, surface related notes, organize and normalize tags, and other tasks related to taking and curating notes.
  - Files should default to markdown, unless otherwise specified
  - When asked to suggest tags, do not suggest existing items in the `tags` front matter, unless suggesting changes to any of them.

prompts:
  - name: Tag This File
    description: Suggest tags for the current file.
    prompt: |
      @codebase @currentFile Analyze the current file and suggest tags for it.
      Look for tags in other files that potentially match and suggest them, so that files can be linked together.

docs:
  - name: Foam
    startUrl: https://foambubble.github.io/foam/


%YAML 1.1
---
name: Qwen3 Large Context
version: 1.0.0
schema: v1

ollama_provider: &ollama_provider
  provider: ollama
  capbilities:
    - tool_use

models:
  - name: Qwen 3 30b
    <<: *ollama_provider
    model: qwen3:30b

Description

For some reason, whenever I run any model through Continue, it uses roughly half again the amount of vRAM that it's supposed to. This happens regardless of whether I'm in Chat or Agent mode, and regardless of context items or rules added.

• cogito:4b 5.1gb -> 7.5gb
• qwen3:30b 19gb -> 26gb
• command-r:latest 20gb -> 30gb

On the larger models, this pushes them beyond my max vRAM, requiring Ollama to offload some of the work onto the CPU, and drastically slowing down processing.

To reproduce

1. Install Ollama with ROCm (or possibly CUDA) support
2. Set up any model
3. Run any command on that model in Ollama's internal client
4. Run ollama ps -- Size should report the same as the listing on https://ollama.com/models
5. Stop the client and run ollama ps to ensure the model is unloaded
6. Run any command in Continue using the same model
7. Run ollama ps -- Size reports roughly +50% for all models. This is particularly obvious with models that fit on the graphics card when using other clients, but causes Ollama to resource split when using Continue.

Log output

[ShaunaGordon]

In talking with the ollama folks while trying to troubleshoot the OOM errors, I saw that any time Continue prompted Ollama to load a model, ollama ps also reported a context size of 32768, regardless of the model. This is clearly coming from Continue, because other clients set up with context sizes closer to 4096. It seems this could be handled better all around, and it would be nice if this could be configured in Continue.

[tingwai]

@ShaunaGordon Thanks for the report. I followed the steps you outlined but wasn’t able to reproduce the memory issue. Could you provide more details about your environment and any additional context that might help?

The context size is a good point, I will surface that with the team

[ShaunaGordon]

@tingwai What more information would be helpful that isn't already provided? You can find my entire Continue config here, and Dotfiles here, if you want to look at them more in depth, but everything else related to Continue and Ollama are just defaults.

[uinstinct]

hey @ShaunaGordon
just to confirm, does the models.defaultCompletionOptions.contextLength option work for you?

  - name: cogito
    provider: ollama
    model: cogito:3b
    roles:
      - chat
      - edit
    defaultCompletionOptions:
      contextLength: 4096

[ShaunaGordon]

@uinstinct It works in that Continue picks it up and successfully sets the context limit in Ollama, but in testing it, I'm noticing that even with basically all but two tiny rules (171 characters), even one isolated query is sending 18k tokens. It looks like it's sending information about all active tools and MCP servers, too (I can get the used context size down significantly by turning tools off).

I wonder if there's a way to be more intelligent about this? When doing agentic stuff, the more reasonable context sizes, combined with the way pieces are activated/deactivated, would make it tedious to use, as users would frequently run into context size limits, even from the outset.

(As an aside, if it's a general context length that gets applied to all modes, why is that option in completion options?)

[farfromrefug]

I think i have the same issue. When running a model through ollama or alpaca (for example) i can see that the models i use are using 100%GPU. But when i use the same models through continue they use 50%GPU/50%CPU and are much slower.

I am still to understand where i should configure continue with yaml files as shown in this issue.
What i can tell you is that this is my config.json

{
  "models": [
    {
      "title": "Llama 3 8B",
      "provider": "ollama",
      "model": "llama3:8b"
    }
  ],
  "tabAutocompleteModel": {
    "title": "DeepSeek Coder 6.7B",
    "provider": "ollama",
    "model": "deepseek-coder:6.7b"
  },
  "embeddingsProvider": {
    "provider": "ollama",
    "model": "nomic-embed-text"
  }
}

[ZAKOVYIKA]

I'm facing the same problem.

1. Running ollama in a Docker container, where timeout shutdown is disabled for each model.
2. Pre-launching the qwen3:4b and qwen2.5-coder:1.5b-base models.
3. The result of the ollama ps command in the console of the container with ollama may be seen below.

# ollama ps
NAME                       ID              SIZE      PROCESSOR    UNTIL   
qwen3:4b                   2bfd38a7daaf    5.2 GB    100% GPU     Forever    
qwen2.5-coder:1.5b-base    02e0f2817a89    2.0 GB    100% GPU     Forever

5. Running VSCode with the Continue plugin.
6. Writing "Hi!" in a chat with qwen3:4 in the Continue's chat-window.
7. Previous models are disabled, as may be seen below.

My config.yaml for Local Agent in Continue:

name: Local Assistant
version: 1.0.0
schema: v1.1
models:
  - name: Qwen 3 4B
    provider: ollama
    model: qwen3:4b
    roles:
      - chat
      - edit
      - apply
  - name: Qwen2.5-Coder 1.5B
    provider: ollama
    model: qwen2.5-coder:1.5b-base
    roles:
      - autocomplete
  - name: Nomic Embed
    provider: ollama
    model: nomic-embed-text:latest
    roles:
      - embed
context:
  - provider: code
  - provider: docs
  - provider: diff
  - provider: terminal
  - provider: problems
  - provider: folder
  - provider: codebase

My docker-compose file for container with ollama:

services:
  ollama-small:
    image: ollama/ollama:latest
    restart: unless-stopped
    container_name: ollama-small
    volumes:
      - ./ollama:/root/.ollama
    ports:
      - 11434:11434
    environment:
      OLLAMA_KEEP_ALIVE: -1
    runtime: nvidia