OpenCode/Ollama context window requires manual configuration

[hugo-lorenzo-mato]

Summary

When using OpenCode with Ollama backend, the default context window (2048-4096 tokens) is insufficient for code-intensive tasks. This requires manual Ollama configuration that is not immediately obvious to users.

Problem

┌─────────────────────────────────────────────────────────────┐
│  quorum-ai → OpenCode → Ollama API → Model                  │
│                                                             │
│  Limitation chain:                                          │
│                                                             │
│  1. Ollama defaults to 2048 tokens context                  │
│  2. OpenCode uses OpenAI-compatible API (/v1)               │
│  3. OpenAI-compat API does NOT support num_ctx parameter    │
│  4. quorum-ai cannot pass context config to OpenCode        │
│                                                             │
│  Result: Models run at ~6% of their capacity                │
└─────────────────────────────────────────────────────────────┘

Impact

Model	Max Capacity	Default Allocation	Utilization
qwen2.5-coder:32b	32,768	2,048	6%
deepseek-r1:32b	131,072	2,048	1.5%
qwen3-coder:30b	262,144	2,048	0.7%

Symptoms

• Truncated responses mid-generation
• Missing context in multi-file analysis
• Poor code understanding despite capable model
• "Context too long" errors in Ollama logs

Root Cause Analysis

1. Ollama default behavior: Conservatively allocates 2048 tokens to minimize VRAM usage
2. OpenCode API limitation: Uses @ai-sdk/openai-compatible which doesn't support Ollama-specific options.num_ctx
3. No passthrough mechanism: quorum-ai → OpenCode → Ollama chain has no way to configure context per-request

OpenCode upstream issue

Feature request exists: anomalyco/opencode#3250

Status: Open, labeled "help-wanted"

Technical blocker: Ollama's OpenAI-compatible endpoint (/v1/*) does not accept num_ctx in request body. Only native Ollama API (/api/generate) supports it.

Current Workaround

Users must configure Ollama server-side:

# Linux (systemd)
sudo systemctl edit ollama.service

# Add:
[Service]
Environment="OLLAMA_CONTEXT_LENGTH=32768"

# Apply:
sudo systemctl daemon-reload
sudo systemctl restart ollama

Documentation Added

• Created docs/OLLAMA.md with comprehensive integration guide
• Updated docs/CONFIGURATION.md with context window warning
• Added troubleshooting section in docs/TROUBLESHOOTING.md

Future Considerations

Option 1: Wait for OpenCode upstream fix

Monitor anomalyco/opencode#3250 for implementation.

Option 2: Direct Ollama integration

Bypass OpenCode for simple prompts using Ollama native API:

// Hypothetical direct Ollama adapter
type OllamaAdapter struct {
    baseURL string
    // Can set num_ctx per request via native API
}

Option 3: Documentation-only approach (current)

Maintain clear documentation for manual Ollama configuration.

References

• Ollama FAQ - Context Window
• OpenCode + Ollama Integration
• OpenCode Issue #3250
• Context Window Optimization Guide