Add Support for Image-to-Video Models

[unicorn667]

Currently, OpenWebUI primarily supports text-based LLMs. It would be great to extend support for Image-to-Video models. This would allow users to generate videos from images and prompts directly within the OpenWebUI interface.

[rgaricano]

It's not as simple as you say, you have to take into account that you are working with different models together, each one a child of its mother, image and video formats, codecs,..., and enough hardware to move all that.
Also, that can already be done with the available options: Automatic1111 or Comfy if you want to work locally or the corresponding online ones (and more that can be added).
I imagine that in the future it will be easier but I have the impression that for now the developers are focused on other more immediate things, not to mention that by adding too many options you run the risk of the project becoming an untamable beast.

[unicorn667]

Hi,

thanks for the reply. Your admission that the project will run out of control if too much is installed is absolutely correct. I think I should be fine with the two options.

[rollingstone87]

UX will be greatly improved indeed if we could get multimedia output from chat, all I got so far is image out of text, really expecting to be able to get video preview inside of chat.

[rgaricano]

Yes, of course I agree with that too, I just noticed that working with multimodal models is a bit more complex, that there is a lot of work behind it to make the pieces work correctly and that if it is not done well it can be a source of many problems.
Personally I would prefer that before going into depth on that topic other aspects (such as searches or handling of markdown code) were solved and improved, but I am not a developer, I am just trying to collaborate a little.
In any case, I do not see much of a problem with being able to have video embeddings in the chat (with the necessary adjustments so that they do not trigger and overload the vectorization database on light systems and greatly increase the generation of tokens)