dsh-video-lens v0.3.1

dsh-video-lens v0.3.1

Video understanding for DeepSeek Harness — give text-only agents eyes and ears on video.

What's new

• Scene-aware frame sampling — ffmpeg scdet shot detection picks representative frames per shot; duration-adaptive budget (4–12 frames) with resolution scaling to keep VLM cost bounded
• Optional ASR transcript — any OpenAI-compatible /audio/transcriptions endpoint (SiliconFlow, OpenAI, Groq, ...); text-only and timestamped responses both supported; missing key or provider failure never breaks the visual path (transcriptError surfaced in evidence)
• Time-anchored Q&A (video_ask) — parses explicit time references ("at 3:20", "第2分钟") or locates relevant speech via transcript keyword matching, re-samples frames from matched windows, answers with confidence + supporting timestamps
• Provider-agnostic — vision and ASR are both configurable OpenAI-compatible endpoints (baseUrl + model + key env var)

Install

pnpm add dsh-video-lens

Register the bundle in your DSH profile package.json:

{
  "dependencies": { "dsh-video-lens": "^0.3" },
  "dsh": { "profile": { "bundles": ["@deepseek-ai/dsh-base", "@deepseek-ai/dsh-web-app", "dsh-video-lens"] } }
}

Export keys and restart the profile:

export VIDEO_LENS_API_KEY=sk-...   # vision
export VIDEO_LENS_ASR_KEY=sk-...   # optional, ASR

Validation

• 71 automated tests, all green on GitHub Actions CI (Ubuntu, Node 20, ffmpeg)
  • Unit: sampling plans, time parsing, keyword matching, prompts (43)
  • Network layer: 401/429/500, non-JSON, empty content, abort propagation, ASR text-only/segments (16)
  • E2E: full ffmpeg pipeline on synthetic video, graceful degradation without keys (12)
• Requires Node.js >= 20, ffmpeg >= 6 (scdet)

Links

• Repository: https://github.com/dundunhan/dsh-video-lens
• npm: https://www.npmjs.com/package/dsh-video-lens