KoboldCpp for DeepSeek Harness

dsh-koboldcpp-hands — give your DeepSeek Harness agent a pair of local hands.

English · 中文

A third-party tool plugin for DeepSeek Harness that lets the online model (your main conversation model) hand repetitive, token-cheap labor to a local KoboldCpp (llama.cpp) server — plain text work and vision work (image analysis / OCR / comparison).

The main model stays where your deployment puts it. When it decides a job is better done locally, it calls:

• koboldcpp_run — run one prompt on the local text model (batch rewrites, name translations, string munging, short summarization, extraction).
• koboldcpp_vision — send images to a local multimodal model (OCR, image analysis, multi-image comparison) with structured report templates.

The plugin manages the local server lifecycle: it launches your KoboldCpp binary with your .kcpps launch config (which owns the GPU backend, model, mmproj, port), waits for the model to load, and stops the server on an idle timeout and/or harness exit. An externally started KoboldCpp is reused and never killed.

---

What it does

• Two model-facing tools registered on the harness tool registry (ctx.tools), following the official dsh-tools contract (defineTool, canonical JSON values, pure render/presenters, exec.signal forwarding).
• On-demand server lifecycle: first tool call spawns exePath with your kcppsPath + --port, polls /v1/models until healthy, self-heals if the server dies, and stops per stopBehavior (exit / idle / never). Windows process-tree termination (taskkill /T) because KoboldCpp relaunches itself as a child process.
• Text + vision wire support: non-streaming OpenAI-compatible chat-completions; images sent as the standard multimodal content array.
• Three image sources for the vision tool: local file paths, data:/http(s): URLs, or the images attached to the current conversation (read through the harness attachment service). Note: the conversation-attachment source requires a main model that declares image input — with a text-only main model only image_paths / image_urls work (see current limitation).
• Structured vision prompts structured, machine-verifiable report contracts: analyze (8-section report), ocr (character-exact), compare (multi-image, 5-section) — plus a fidelity rule for the online model (relay verbatim, never invent, preserve uncertainty).
• Live config: a llm-koboldcpp: section in the harness user-settings document overrides the plugin config without a restart; KOBOOLDCPP_EXE / KOBOOLDCPP_KCPPS env fallbacks.
• Safe ownership: external KoboldCpp processes are reused, never touched; only servers the plugin spawned are stopped.

What it does NOT do

• Does not replace the harness LLM provider — the online model stays the main model; the local model is only reached through the two tools.
• Does not decide GPU backends, model paths, or templates for you. Everything about the KoboldCpp launch lives in your .kcpps file (backend usecuda/usevulkan/usecpu, model_param, mmproj, port). No probing, no auto-flags.
• Does not modify any DeepSeek Harness file; it is a pure add-on plugin.
• Does not bundle or host GGUF / mmproj model files — bring your own.
• Does not use streaming or an API key (local server; no credentials involved).
• Does not run inside the harness process as a service — it spawns a separate KoboldCpp process only when needed.

Requirements

Item	Requirement
Node.js	≥ 20
DeepSeek Harness	installed (npx @deepseek-ai/dsh web or a source checkout)
KoboldCpp binary	koboldcpp.exe (NVIDIA/CUDA) or koboldcpp-nocuda.exe (AMD/Vulkan), any release with /v1/chat/completions
GGUF model	your own; vision additionally needs a multimodal GGUF and its mmproj (set "mmproj" in the kcpps)

Install

Inside your harness project (the directory whose cordis.yml / cordis.patch.yml composes your deployment):

npm install dsh-koboldcpp-hands

From a source checkout of the harness, you can point the plugin row directly at a clone of this repo:

- insert:
    - id: koboldcpp-tool
      name: '../dsh-koboldcpp-hands'

Configure

You own the launch settings. Add the plugin row to your profile cordis.patch.yml:

- insert:
    - id: koboldcpp-tool
      name: 'dsh-koboldcpp-hands'
      config:
        baseURL: 'http://127.0.0.1:5001'                     # must match the port in your kcpps
        exePath: 'C:\path\to\koboldcpp-nocuda.exe'           # your binary (CUDA or Vulkan build)
        kcppsPath: 'C:\path\to\your-model.kcpps'             # your launch config: backend + model + mmproj + port
        autoStart: true
        stopBehavior: idle
        idleStopMinutes: 30

The launched command is exactly:

koboldcpp-nocuda.exe "C:\path\to\your-model.kcpps" --port 5001

Full config reference (all 17 fields with defaults): docs/api.md §1.2.

Using the tools

koboldcpp_run — text

param	type	required	meaning
prompt	string	yes	instruction/text sent as a user message
system	string	no	optional system instructions
temperature	number	no	sampling temperature (0–2)
max_tokens	integer	no	output cap (default maxTokens)
stop	string[]	no	stop sequences

Returns { text, reasoning?, model, usage, elapsedMs }.

koboldcpp_vision — images / OCR

param	type	required	meaning
mode	analyze/ocr/compare	no	built-in prompt template (default analyze)
prompt	string	no	custom instruction (overrides the template)
image_paths	string[]	no	local images (png/jpg/jpeg/webp/gif/bmp, ≤20 MB each)
image_urls	string[]	no	data:image/... or http(s):// URLs
temperature	number	no	sampling temperature (lower for OCR, ~0.2)
max_tokens	integer	no	output cap
stop	string[]	no	stop sequences

Image sources resolve in order: explicit image_paths + image_urls → the most recent image(s) attached to the conversation → clear error. compare sends 2–4 images in ONE request for joint reasoning.

Returns { text, reasoning?, model, images, usage, elapsedMs }.

Vision requires a multimodal GGUF and its mmproj projector in the kcpps. Without mmproj the request completes but the model cannot see the image.

Current limitation: text-only main models — images arrive by link or local path only

In the current release (plugin 0.1.0, harness 0.1.0-rc.6), when your main model is text-only, koboldcpp_vision can only receive images through the two explicit channels: image_paths (local file paths) and image_urls (online / data: links). The conversation-attachment source is unavailable in that setup — and this is a hard limit of the harness, not of this plugin:

1. When you paste or drop an image while a text-only model is selected, dsh rejects the message before it ever enters the session with attachment-error / MODEL_DOES_NOT_SUPPORT_IMAGES (the UI shows "当前模型不支持图片，请切换支持图片的模型"). The check lives in dsh-host-apiproxy: the selected model's declared input modalities (from the pi-ai model catalog) must include image; a model catalogued as input: ["text"] (e.g. deepseek-v4-flash / deepseek-v4-pro on the opencode-go route) is refused.
2. Even if an image part got through, dsh-llm-pi-ai's streaming adapter rejects image content for the same text-only models (UNSUPPORTED_CONTENT), and subagent continuation sessions block images in the browser client entirely.
3. Because the message is refused before it is durably attached, the "most recent conversation-attachment" source has nothing to read — unlike OpenCode / Pi, dsh does not currently turn a pasted image into a temporary file path for text-only models.

Workarounds that work today:

• Ask the model to call koboldcpp_vision with image_paths: ["C:\\...\\photo.png"] — any path readable by the harness process.
• Or pass an online link: image_urls: ["https://example.com/photo.png"] (also data: URLs).
• Or switch the main model to one whose catalog entry declares image input (e.g. minimax-m3, qwen3.7-plus, kimi-k2.6, kimi-k3, grok-4.5 on the opencode-go route) — the conversation-attachment source then works automatically.

Tracked upstream in deepseek-harness discussion #1378 (request: allow image attachments for text-only models and deliver them to tools as links/paths). This section will be updated when the harness relaxes the restriction.

Roadmap

Possible / planned directions:

• More vision modes and prompt templates (document layouts, table extraction).
• Multi-model autoswapmode support (kcpps-level; the wire model field is already configurable).
• Publishing to the npm registry and the dsh-plugin topic.
• Batch jobs: drive many local calls from one agent turn.

Deliberately NOT planned:

• Automatic GPU/backend detection or flag injection — your kcpps is authoritative by design.
• Becoming an LLM provider adapter — the plugin stays a tool; the online model stays the main model.
• Streaming responses — tool calls get the full answer in one round trip (simpler and sufficient).
• Bundling model files (gguf/mmproj) or modifying DeepSeek Harness itself.

Uninstall

Removing the plugin is as clean as installing it:

1. Remove the plugin row from your profile cordis.patch.yml (or cordis.yml):

   # delete this block
   - insert:
       - id: koboldcpp-tool
         name: 'dsh-koboldcpp-hands'

2. Restart the harness (or let HMR reload if you edit the config live). The two tools (koboldcpp_run, koboldcpp_vision) are unregistered automatically — the online model no longer sees them.
3. Server lifecycle after removal:
   • stopBehavior: exit — the plugin stops its spawned KoboldCpp when the harness shuts down gracefully.
   • stopBehavior: idle — the server stops after the idle window.
   • stopBehavior: never — the server keeps running; stop it yourself (e.g. taskkill /PID <pid> /T /F on Windows).
   • An externally started KoboldCpp is never touched.
4. No residue: the plugin writes nothing into the harness, leaves no lingering processes on graceful shutdown, and creates no config files of its own. If you installed it via npm, remove it with npm uninstall dsh-koboldcpp-hands.

Removing the plugin package itself

• Installed via npm — one command removes the package from your project:

  npm uninstall dsh-koboldcpp-hands

• Installed from a git clone (profile row points at the checkout) — remove the profile row, then delete the checkout:

  Remove-Item -Recurse -Force C:\path\to\dsh-koboldcpp-hands

  rm -rf /path/to/dsh-koboldcpp-hands

Version & compatibility

Component	Version
This plugin	0.1.0
DeepSeek Harness	0.1.0-rc series (tested against npm @deepseek-ai/* 0.1.0-rc.6)
Node.js	≥ 20
KoboldCpp	any release exposing /v1/chat/completions

Peer dependencies (runtime): @deepseek-ai/cordis ^4.0.1, @deepseek-ai/dsh-tools/dsh-llm/dsh-session/dsh-attachment/dsh-settings/dsh-launch-environment >=0.1.0-rc.2, @deepseek-ai/schemastery ^3.18.1.

Development

npm install
npm run typecheck   # tsc --noEmit
npm test            # vitest run (45 tests: unit, tool, integration, Loader composition)
npm run build       # tsc -> lib/

Tests include a REAL-composition tier (app boot → Cordis Loader → cordis.yml) per the harness testing policy, and a real-machine scenario driver (tests/real-driver.mjs) for autostart / reuse / not-running behaviors.

Documentation

doc	content
docs/engineering.md	structure, plugin contract, commands, test tiers
docs/api.md	authoritative API reference (Config, tools, classes, error codes)
docs/glossary.md	standard terminology
docs/solutions.md	pitfalls, troubleshooting, methodology

Credits & thanks

• DeepSeek AI — the DeepSeek Harness platform this plugin plugs into, and the reference implementations (dsh-llm-deepseek, dsh-tool-todo) that define the patterns we follow.
• LostRuins / KoboldCpp — the excellent local llama.cpp server with an OpenAI-compatible API that makes all of this possible.
• Cordis — the plugin runtime that powers the harness.
• The open-source models and quantizers (llama.cpp ecosystem, GGUF) that run locally on your machine.

License

MIT. Not affiliated with DeepSeek AI or LostRuins; dsh and koboldcpp are trademarks of their respective owners.