[Plugin] dsh-vision-primitives — precise interactive visual reasoning, native, no external MCP

[zouyuanqing]

dsh-vision-primitives — a native interactive visual-reasoning plugin for DeepSeek Harness, with zero external MCP servers.

Repo: https://github.com/zouyuanqing/dsh-vision-primitives · Install: dsh plugin --profile <name> add github:zouyuanqing/dsh-vision-primitives

What it does

Gives text-only agents "precise eyes": a Set-of-Mark numbered grid overlay turns fuzzy visual perception into exact pixel coordinates (grid → vision_resolve(cell) → cell-center coordinates → vision_zoom lossless upscaling with a coordinate-mapping chain back to the original frame), then deterministic verification via annotate / measure / diff / color segmentation / native OCR.

Highlights

• 13 vision_* tools: capture (multi-monitor screen or PNG), grid, resolve, zoom, annotate, measure, diff, find_color, ocr, describe, locate, state, reset
• Pure-JS kernel, zero dependencies: PNG codec (stored-deflate), full RFC1951 inflate, 5×7 bitmap font, Bresenham, connected components, frame diff — verified against Node zlib (7/7) and byte-exact encode roundtrips on real 2560×1600 captures
• MiMo V2.5 multimodal backend (Xiaomi official API): vision_describe / vision_locate, plus a registered mimo LlmAdapter (streaming / tool calls / image input)
• Minimal OS boundary: only screen capture, OCR, and binary persistence cross into Windows PowerShell via the native subprocess service; no desktop control
• Official profile-bundle format (pure JS, no build script, no allowBuilds needed); API keys are read lazily from DSH credentials — nothing hardcoded

Design is a native re-implementation of my earlier vision-primitives-mcp.

Feedback, issues, and PRs welcome!

[zouyuanqing]

v1.1.0 — WebUI configuration card (https://github.com/zouyuanqing/dsh-vision-primitives/releases/tag/v1.1.0)

The plugin now registers the vision-primitives settings namespace and ships a Client half: a configuration card under 设置 → 插件配置 → Vision Primitives with API Key / Base URL / Model / Timeout. API key is stored through the credentials domain (MIMO_API_KEY, never echoed); baseUrl/model/timeoutMs go through the official settings transport. Resolution order: WebUI user settings → row config → defaults. CLI equivalent stays dsh credentials set MIMO_API_KEY <key>.

[zouyuanqing]

v1.3.0 — Visual Evidence Protocol (https://github.com/zouyuanqing/dsh-vision-primitives/releases/tag/v1.3.0): pasted images (and the new vision_analyze tool) now produce structured visual evidence — caption + layout + grounded elements whose normalized boxes are mapped to pixel boxes and SOM grid cells, so text-only models can reference elements precisely (vision_resolve(cell=N)). VLM reasoning is drafted internally and dropped by default.

[zouyuanqing]

v1.6.0 — send-time image bridge (https://github.com/zouyuanqing/dsh-vision-primitives/releases/tag/v1.6.0): text-only models can now paste/drag images into the composer (native thumbnails, no "model does not support images"); at send time the image is cached into the workspace and handed to the model as [Attached image: <path>] text, so the model can read_image or vision_analyze(path) on demand. Vision models are untouched. Community approach inspired by dsh-image-to-text and pi2dsh — no DSH source changes.