Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Video backend for Linux #2646

Open
mafiesto4 opened this issue May 24, 2024 · 3 comments
Open

Add Video backend for Linux #2646

mafiesto4 opened this issue May 24, 2024 · 3 comments
Labels
graphics help wanted Extra attention is needed platform

Comments

@mafiesto4
Copy link
Member

Add missing video backend for Linux platform. Find a good lib or 3rd party plugin for .mp4 files loading (eg. minimp4), then decode h.254 video frames into YUV format (engine can automatically convert into RGB image). Use one of the existing backends as base for proper implementation: https://github.com/FlaxEngine/FlaxEngine/tree/1.9/Source/Engine/Video

All other platforms provide system-lib for video media playback so only Linux is missing.

Video docs: https://github.com/FlaxEngine/FlaxDocs/blob/1.9/manual/graphics/video/index.md

Here is a prototype with minimp4 that loads data from a file but requires frames decoding: PrototypeMinimp4.zip

@Zode
Copy link
Contributor

Zode commented May 24, 2024

https://en.wikipedia.org/wiki/Advanced_Video_Coding#Licensing
Due to the licensing nature of mp4/h264, linux may need the use of webm/vp8 or equivalent.
Thoughts on using https://gstreamer.freedesktop.org/ to do the video decoding job? It is the most widely used in apps, and is what you would get if games were ran through proton/wine.
Could be dynamically linked so that it uses system binaries for the job, so it can use "nonfree" decoders (such as nvidia's gpu decoding).

@mafiesto4
Copy link
Member Author

Could be dynamically linked so that it uses system binaries for the job, so it can use "nonfree" decoders (such as nvidia's gpu decoding).

I think it would be a good starting point. Maybe we could use ffmpeg?

@Zode
Copy link
Contributor

Zode commented May 24, 2024

Could be dynamically linked so that it uses system binaries for the job, so it can use "nonfree" decoders (such as nvidia's gpu decoding).

I think it would be a good starting point. Maybe we could use ffmpeg?

FFmpeg is bit harder as it provides a low level API from what i gather, GStreamer provides a high level API similar to the MFPlay API used on Windows side of things so it would be way simpler to integrate (and more importantly: maintain)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
graphics help wanted Extra attention is needed platform
Projects
None yet
Development

No branches or pull requests

2 participants