Question: 24GB VRAM local models for Kanban workers - real-world results?

[kxkayser]

tl;dr: Has anyone successfully run Hermes Kanban workers on local models with 24GB VRAM? Looking for real-world data before GPU upgrade.

Background

I currently run Hermes with 12GB VRAM. Testing local models for Kanban workers:

• qwen3.5:9b (6.6GB): Crashes after 28-91s, hallucinates bash commands instead of using tools
• qwen2.5:14b (9GB): Crashes after 12-33s, overwhelmed by system prompt
• gemma4:e4b (9.6GB): Same failures

All using context_length patch (Issue #24072 applies, but model capability is the blocker).

Root cause: Hermes's massive system prompt (+10K tokens, 30+ tool schemas, memory injection) overwhelms small models. Cloud models work fine.

The Question

I'm considering a 24GB GPU upgrade specifically for local Kanban agents. Based on #523 (Local Model Setup Skill), these fit:

Model	Size	Context	Recommendation
Qwen3-30B-A3B	~17GB	128K	"Best accuracy/efficiency balance"
Qwen3-32B	~20GB	128K	"Dense model, highest accuracy"

Has anyone tested these with Kanban workers specifically?

What I need to know:

1. Do Qwen3-30B/32B reliably call tools (kanban_show, terminal, write_file, etc.) in complex chains?
2. Do they handle multi-step Kanban tasks or crash like 9B/14B models?
3. Are timeout patches needed (Issue [Feature]: Configurable timeouts for auxiliary call_llm and context compression #3404 - hardcoded 30s aux/45s compression timeout)?
4. Any specific config tweaks beyond context_length: 128000?

Environment

• Goal: Async Kanban agents with local models
• Current: 12GB VRAM → considering 24GB upgrade
• Use case: Research/data analysis tasks via analyst, researcher profiles
• Alternative: Keep cloud models (working, has costs)

Would appreciate any real-world experience before spending $1500 on a new GPU.

Related Issues

• Feature: Local Model Setup Skill — Ollama, llama.cpp & vLLM Configuration Guide with Model Recommendations (inspired by Liquid AI LocalCowork) #523: Local Model Setup Skill - model recommendations
• [Feature]: Configurable timeouts for auxiliary call_llm and context compression #3404: Configurable timeouts for local models
• # Bug Report: model.context_length persists across /model provider switches #24072: Context length override (patch works for loading but not tool reliability)
• [Bug]: Ollama Models don't recognize environment (aka hermes-agent) #2074: Ollama models not recognizing environment (similar problem)

---

If no one has tested this, I can volunteer to be the guinea pig and document results if there's interest.

[Jp-yan]

@kxkayser I've tested several setups on 24GB cards. Here's my real-world data:

Model Recommendations for 24GB

Best for Kanban workers (tool-calling):

• Qwen2.5-Coder-14B (Q4_K_M) — ~9GB VRAM, solid tool use, fast inference
• DeepSeek-Coder-V2-Lite (Q4) — ~10GB, great for code tasks

For reasoning-heavy workers:

• Qwen2.5-32B (Q3_K_M) — ~18GB, tight but fits, good for complex reasoning

Multi-worker config

With 24GB you can run 2 workers comfortably (e.g. 14B Q4 for code + 7B Q4 for simple tasks), or 1 big model. I use llama.cpp with --n-gpu-layers 99 for max offloading.

API alternative

For production, I'd actually recommend using cheap API models (DeepSeek/阿里百炼) for workers instead of local. The token cost is negligible compared to the electricity + VRAM wear. You can configure per-worker model routing in Hermes.

If you need help with the config, feel free to ask — happy to share my setup.

[BitWss]

@Jp-yan Why do you not recommend Qwen3 series of models?