FFmpeg as audio and video demuxer and decoder

[zaps166]

FFmpeg (http://ffmpeg.org/about.html) can decode almost every audio and video format and codec using one API. It is very portable. It can replace Theora, OGG, Vorbis, Opus and other libraries (Speex decoder in FFmpeg uses libspeex). It has also its own audio resampler (with audio channel expansion) and video scaler (with pixel format conversion). It has also various audio and video filters which can be used.

FFmpeg is optimized for performance, e.g. libvorbis is slower than built-in FFmpeg Vorbis decoder:
ffmpeg -i Test.ogg -acodec pcm_s16le -f wav - &> /dev/null - 0.51 sec
oggdec -o - Test.ogg &> /dev/null - 0.87 sec

FFmpeg can be linked statically or dynamically. Its configuration tool allows to enable only needed formats and codecs.

FFmpeg has also its own protocol support (http, https, file, rtsp, rtmp, ftp, etc...). They can be compiled-in and used (also in other Godot parts), or they might be disabled during compilation. Then custom IO reader can be created which will use Gotots' file and network functions.

It also can decode still images like JPEG/PNG/GIF, but I've never used FFmpeg for still images :)

See also: #5133 (comment)

[reduz]

FFMPEG is just a library to wrap around a lot of codecs. This is nice, but most of those codecs are either

1. Patent encumbered
2. too slow to decode using CPU, and need native API access

So, if anyone really needs this it could be provided by a separate module, but it's not possible for us to include it. Theora is still the only codec that can be decoded via CPU that is not patent encumbered.

[reduz]

Regarding performance, replacing a simple library like Theora by the whole ffmpeg just to gain a bit of performance is pointless IMO.

[akien-mga]

Also the license is LGPL (or GPL if you enable too much stuff), and both would be no-go to embed directly in Godot's core AFAIK. We need to stay under a permissive license.

[Keyaku]

I don't necessarily agree that FFmpeg is the better choice, but I must agree on the fact that Theora is really slow. It can't play 60 FPS videos without hanging every second, but worse yet is that it lags on 30 FPS videos.

If possible, we should really use a better library. Openh264 looks like a good choice, but I need to research more about it.

[Calinou]

Openh264 looks like a good choice, but I need to research more about it.

H.264 is patented until 2027.

H.264 is a newer video codec. The standard first came out in 2003, but continues to evolve. An automatically generated patent expiration list is available at H.264 Patent List based on the MPEG-LA patent list. The last expiration is US 7826532 on 29 nov 2027 ( note that 7835443 is divisional, but the automated program missed that). US 7826532 was first filed in 05 sep 2003 and has an impressive 1546 day extension. It will be a while before H.264 is patent free.

We can support VP8 or VP9, but both are quite expensive on the CPU (probably more so than Theora), but they do compress better.

[reduz]

vp8 and vp9 are patent free, I would like to implement them but i couldn't
find any single example demuxer

On Thu, Jun 23, 2016 at 3:31 PM, Hugo Locurcio notifications@github.com
wrote:

Openh264 looks like a good choice, but I need to research more about it.

H.264 is patented until 2029.
http://www.osnews.com/story/24954/US_Patent_Expiration_for_MP3_MPEG-2_H_264
.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#5226 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AF-Z21F6RBEqKipplW_jEuSZAghZHIQGks5qOtEfgaJpZM4I2a7P
.

[Keyaku]

H.264 is patented until 2027.

So much for "Open". Still, libtheora should be replaced whenever possible.

vp8 and vp9 are patent free, I would like to implement them but i couldn't
find any single example demuxer

I could check if I could grab/write a demuxer with libvpx.

[reduz]

as far as I understand, webm is mkv+vpx+vorbis , so a mkv demuxer with
support for vpx and vorbis needs to be used. It's a lot of work and none of
us has much experience with video codecs.

On Thu, Jun 23, 2016 at 3:38 PM, António Sarmento notifications@github.com
wrote:

H.264 is patented until 2027.

So much for "Open". To me, with patent != open, no matter what other
strings are attached to the definition of "open".

Well, still, libtheora should be replaced if possible.

vp8 and vp9 are patent free, I would like to implement them but i couldn't
find any single example demuxer

I could check if I could grab/write a demuxer with libvpx.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#5226 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AF-Z2xgOyzD_VluQYzmn65IoxlRbFyFXks5qOtKkgaJpZM4I2a7P
.

[akien-mga]

So, ffmpeg in the main tree obviously won't happen. (It would be the perfect match for a community-maintained module though :)).

[zaps166]

So if I understand it correctly - libvpx can be integrated for VP8 and VP9 (?)