Repository navigation
Feature request: video content blocks (video/*) alongside ImageBlock - pass video to video-capable models #9729
Nonexistent01
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Context
DSH's typed model-content contract is image-only:
@deepseek-ai/dsh-llmlib/typert.host.jsL329-333:ImageBlock { type: 'image' }andImageMediaType = 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif'@deepseek-ai/*packages finds no realvideo/handling anywhere (the only hit is inside the vendored pdf.js bundle string — a false positive).Problem
Most current frontier models accept video input (via native video parts or frame sequences, over the same OpenAI-compatible / provider chat APIs that DSH already speaks). Because the framework's content model has no
videoblock type, there is no way to hand a video clip to a video-capable model — e.g. an MCP tool returning an mp4, or a user attaching a short screen recording. The capability is lost at the framework layer, not the model layer.Request
Add a
VideoBlock { type: 'video' }analogous toImageBlock(attachment ref +video/*media type) and plumb it end to end:video/mp4,video/webm, …)An MVP limited to pass-through on supporting adapters + a documented fallback would already unlock this.
All reactions