Skip to content

webui: support video files as input#22830

Merged
allozaur merged 1 commit into
ggml-org:masterfrom
foldl:webui-video-files
May 17, 2026
Merged

webui: support video files as input#22830
allozaur merged 1 commit into
ggml-org:masterfrom
foldl:webui-video-files

Conversation

@foldl
Copy link
Copy Markdown
Contributor

@foldl foldl commented May 8, 2026

Overview

Support adding video files as input. This can fix #20741.

Everything is done almost the same as audio files.

Detailed Modifications

  1. Add a menu item for uploading video files;
  2. Show an icon in the chat input box (ChatAttachmentsListItemThumbnailFile) like ChatAttachmentsPreviewThumbnailStrip;
  3. A new preview window for video files;
  4. Video files are sent to the server through input_video (just like input_audio for audio files);
  5. Two types of video files are defined (mp4 and ogg);
  6. On Model Information window, video modality is shown as "Vision (Video)", and the vision modality is shown as "Vision (Image)";
  7. Add a new bool field video to Modalities.

Test & Sceenshots

I have tested this with chatllm.cpp.

image image image

Additional information

Some findings or thoughts that are out of the scope of this PR.

  • How to properly show the modalities of image-only, and image-video?
  • Video files often contain audio. At present, when sending to servers, media types are inferred from file extension but not the menu item which is clicked by users.

Requirements

  • I have read and agree with the contributing guidelines
  • AI usage disclosure: NO. I coded this all by myself (copied and modified some existing codes).

@ngxson
Copy link
Copy Markdown
Contributor

ngxson commented May 8, 2026

IMO this can be an acceptable stop-gap solution. But just one concern is that we will eventually have native video support in mtmd, so we should make sure changes from this PR can be easily reverted when it happens.

@foldl
Copy link
Copy Markdown
Contributor Author

foldl commented May 8, 2026

This is for mtmd (see the issue #20741). Why this needs to be reverted when video support in mtmd is ready?

@allozaur
Copy link
Copy Markdown
Contributor

Please rebase this on latest commit on master and solve conflicts.

@foldl foldl force-pushed the webui-video-files branch from 7713550 to eb04056 Compare May 16, 2026 12:53
Copy link
Copy Markdown
Contributor

@ServeurpersoCom ServeurpersoCom left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, video has been added symmetrically to image and audio.
If refactor is required, all three can be maintained in the same way.

@allozaur allozaur merged commit 4f13cb7 into ggml-org:master May 17, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature Request: support video files in WebUI

4 participants