Skip to content

dsh-video-lens v0.3.1

Latest
Latest

Choose a tag to compare

View all tags
@dundunhan dundunhan released this 18 Aug 17:39
v0.3.1
73c30b1

dsh-video-lens v0.3.1

Video understanding for DeepSeek Harness — give text-only agents eyes and ears on video.

What's new

  • Scene-aware frame sampling — ffmpeg scdet shot detection picks representative frames per shot; duration-adaptive budget (4–12 frames) with resolution scaling to keep VLM cost bounded
  • Optional ASR transcript — any OpenAI-compatible /audio/transcriptions endpoint (SiliconFlow, OpenAI, Groq, ...); text-only and timestamped responses both supported; missing key or provider failure never breaks the visual path (transcriptError surfaced in evidence)
  • Time-anchored Q&A (video_ask) — parses explicit time references ("at 3:20", "第2分钟") or locates relevant speech via transcript keyword matching, re-samples frames from matched windows, answers with confidence + supporting timestamps
  • Provider-agnostic — vision and ASR are both configurable OpenAI-compatible endpoints (baseUrl + model + key env var)

Install

pnpm add dsh-video-lens

Register the bundle in your DSH profile package.json:

{
  "dependencies": { "dsh-video-lens": "^0.3" },
  "dsh": { "profile": { "bundles": ["@deepseek-ai/dsh-base", "@deepseek-ai/dsh-web-app", "dsh-video-lens"] } }
}

Export keys and restart the profile:

export VIDEO_LENS_API_KEY=sk-...   # vision
export VIDEO_LENS_ASR_KEY=sk-...   # optional, ASR

Validation

  • 71 automated tests, all green on GitHub Actions CI (Ubuntu, Node 20, ffmpeg)
    • Unit: sampling plans, time parsing, keyword matching, prompts (43)
    • Network layer: 401/429/500, non-JSON, empty content, abort propagation, ASR text-only/segments (16)
    • E2E: full ffmpeg pipeline on synthetic video, graceful degradation without keys (12)
  • Requires Node.js >= 20, ffmpeg >= 6 (scdet)

Links

Assets 2
Loading