Skip to content

FFmpeg Integration

Jump to bottom
Christian Lehmann edited this page Jun 27, 2024 · 7 revisions

The VVC decoder in FFmpeg is almost feature complete and is still being optimized.
As VVdeC is still faster and realtime capable this section describes how VVdeC can be added to FFmpeg.

Linux

The following procedure to patch FFmpeg with VVdeC support is tested with FFmpeg revision 41190da9e1 (June 22th 2024).

  1. Build and install VVdeC where it can be found by pkg-config
  • Build VVdeC shared and install into a system path, e.g.:
    sudo make install install-prefix=/usr/local
  1. Clone FFmpeg and go into the downloaded folder
  • git clone https://git.ffmpeg.org/ffmpeg.git ffmpeg
  • cd ffmpeg

  1. Checkout the master

  2. Download and patch with the latest VVdeC Patch from the FFmpeg patchwork:

    • wget -O libvvdec.patch https://patchwork.ffmpeg.org/project/ffmpeg/patch/20240514150936.39657-3-chris10317h5@gmail.com/mbox/

  3. Patch with the latest VVdeC Patch from the FFmpeg patchwork
    As the patch can not be applied as the file configure changed, it can only be patched partly. One line have have to be patched manually

    • patch -p 1 < libvvdec.patch

    Patch the file configure manually:

    1. Open file configure
    2. Search for line:
      enabled libvvenc && require_pkg_config libvvenc "libvvenc >= 1.6.1" "vvenc/vvenc.h" vvenc_get_version
      Insert into next line:
      enabled libvvdec && require_pkg_config libvvdec "libvvdec >= 1.6.0" "vvdec/vvdec.h" vvdec_get_version

  4. Configure and build FFmpeg

  • ./configure --enable-pthreads --enable-pic --enable-shared --enable-rpath --arch=amd64 --enable-demuxer=dash --enable-libxml2 --enable-libvvdec
  • Add more options to your configure command to enable all needed tools and codecs (e.g. --enable-libvvenc)
  • make (use make -j to compile multi-threaded)
  1. Install ffmpeg (optional step)
  • sudo make install
  1. Check if FFmpeg supports VVC
    ffmpeg -hide_banner -codecs | grep vvc
    DEV.L. vvc H.266 / VVC (Versatile Video Coding) (decoders: libvvdec ) (encoders: libvvenc )

In case you want to revert all changes the master can be cleaned up to origin/master by using:
git reset --hard
git clean -df

MacOS

In general you can follow the guidelines for Linux when build on MacOS.

VVdeC is also available in Homebrew.
If you want to skip the build process it can be installed by calling:
brew install vvdec

When building on an ARM platform please follow this guide:

ARM devices (M1, M2)

When running on an ARM device you have to build and install VVdeC and FFmpeg for ARM To build VVdeC on ARM ninja build must be used to build, as with XCode generated libraries are not working.
It can be installed by using Homebrew: brew install ninja

  1. Build and install VVdeC:
    sudo make g=ninja install install-prefix=/usr/local
    Alternatively install via Homebrew: brew install vvdec
  2. configure and build FFmpeg:
  • VVenC and VVdeC support: ./configure --enable-pthreads --enable-pic --enable-shared --enable-rpath --arch=arm64 --enable-demuxer=dash --enable-libxml2 --enable-libvvenc --enable-libvvdec
  • VVdeC support: ./configure --enable-pthreads --enable-pic --enable-shared --enable-rpath --arch=arm64 --enable-demuxer=dash --enable-libxml2 --enable-libvvdec

Windows

As building FFmpeg for Windows can be tricky, we recommend the Media Autobuild Suite.
VVdeC is already part of the suite and must be enabled during the script setup to get VVC support in FFmpeg.

Follow the guide given in the Media Autobuild Suite to build FFmpeg.

  1. Clone the media-autobuild_suite
  • git clone https://github.com/m-ab-s/media-autobuild_suite
  • cd media-autobuild_suite
  1. Patch media-autobuild_suite to enable ffmpeg with vvc support
  • Open the file media-autobuild_suite/build/media-suite_compile.sh in your favorite Editor
* Search for line `lib/cmake/vvdec/vvdecConfig.cmake)`:
```sh
if [[ $bits = 64bit && $vvdec= y ]] &&
     do_vcs "$SOURCE_REPO_LIBVVDEC"; then
 # vvdecimplementation
fi
  • change the line fi to following code:
else
    pc_exists libvvdec || do_removeOption "--enable-libvvdec"
fi
  • Search for line do_changeFFmpegConfig:
do_changeFFmpegConfig "$license"
  • Go into next line and insert the following code to apply the patchset:
if [[ $bits = 64bit ]] && enabled_any libvvdec; then
	  do_patch "https://patchwork.ffmpeg.org/project/ffmpeg/patch/20240514150936.39657-3-chris10317h5@gmail.com/mbox/" am  ||
          do_removeOptions "--enable-libvvdec"
fi
  • Save and close the file media-autobuild_suite/build/media-suite_compile.sh
  • Open the file media-autobuild_suite/media-autobuild_suite.bat in your favorite Editor
  • Search for the line:
    libdav1d libaom --disable-debug libfdk-aac
  • Change the line to:
    libdav1d libaom --disable-debug libfdk-aac libvvdec
  • Save and close the file media-autobuild_suite/media-autobuild_suite.bat

  1. Open the Windows PowerShell and run:
    media-autobuild_suite.bat

  2. Follow the instructions given by the script and enable VVdeC when asked

Troubleshooting: Media Autobuild Suite

FFmpeg may fail to build as the Media Autobuild Suite is kind of a moving target.
That means FFmpeg is depending on a bunch of core applications and codecs which are build from the current master branch per default. As a depending repository may fail to build or is not compatible with the ffmpeg version this section will show solutions or workarounds on known issues.

libjxl

  • error during ffmpeg configure step: ERROR: libjxl >= 0.7.0 not found using pkg-config
  • workaround: disable libjxl in file build/media-autobuild_suite.ini
    jpegxl=2 and in file build/ffmpeg_options.txt by comment line
    #--enable-libjxl

libavif

  • workaround: disable libavif in file build/media-autobuild_suite.ini
    libavif=2

texinfo 7.1 not compatible with current ffmpeg head

  • error during ffmpeg build step: /build/ffmpeg-git/doc/t2h.pm: Undefined subroutine &Texinfo::Config::set_from_init_file ...
  • workaround: disable html generation in file build/ffmpeg_options.txt by adding
    --disable-doc

libplacebo

  • workaround: disable in file build/ffmpeg_options.txt by comment line
    #--enable-libplacebo or set
    --disable-libplacebo

set specific revisions

If a dependent library has changed the interface and is not compatible with ffmpeg anymore a particular revision can be set in the file build/media-suite_deps.sh, e.g. for mpv player
SOURCE_REPO_MPV=https://github.com/mpv-player/mpv.git#tag=v0.36.0
The particular FFmpeg revision can also be set in the file build/media-autobuild_suite.ini, e.g.
ffmpegPath=https://git.ffmpeg.org/ffmpeg.git#1b41631185

Usage

Check for valid FFmpeg installation

ffmpeg -hide_banner -codecs | grep vvc
DEV.L. vvc H.266 / VVC (Versatile Video Coding) (decoders: libvvdec ) (encoders: libvvenc )

Available options

Run FFmpeg help to see available options for decoder:
ffmpeg -h decoder=libvvdec

Decoder libvvdec [H.266 / VVC Decoder VVdeC]:
General capabilities: delay threads
Threading capabilities: other
Supported pixel formats: gray gray10le yuv420p yuv422p yuv444p yuv420p10le yuv422p10le yuv444p10le

Usage

Decoding with VVdeC

Per default the FFmpeg VVC decoder is used. To run VVdeC it must be specified:
ffmpeg -c:v libvvdec -i input.mp4 output.yuv
ffmpeg -c:v libvvdec -i input.mp4 -strict -1 output.y4m

Extract RAW ES from mp4:

ffmpeg -i input.mp4 -an -vcodec copy -bsf vvc_mp4toannexb output.266

Measure performance

ffmpeg -c:v libvvdec -benchmark -i input.mp4 -f null NULL

Playback

ffplay is part of ffmpeg and can be used to playback VVC streams.
Following file formats are supported and can be played with ffplay:

  • RAW elementary streams (.266, .vvc)
  • ISO base media file format (.mp4, .mpd)
  • MPEG transport streams (.ts)

Per default the FFmpeg VVC decoder is used.
To use VVdeC it must be specified in the ffplay call:
ffplay -codec:v:1 libvvdec input

general usage of ffplay

ffplay input
To use VVdeC use:
ffplay -codec:v:1 libvvdec input

To playback a specific video track the option -vst v:id can be used. e.g.:
ffplay -codec:v:1 libvvdec dashfile.mpd -vst v:2
To toggle between different tracks during playback use v

Test streams

A variety of transport streams and dash streams according to the DVB specification can be found at:
https://dvb.org/specifications/verification-validation/vvc-test-content/

All test bitstreams from JVET can be found at:
https://www.itu.int/wftp3/av-arch/jvet-site/bitstream_exchange/VVC/draft_conformance/draft6/

Disclaimer

The used libvvdec patchset is an community submitted patch and not merged into the master nor checked by the developer team.

Home | Build | Usage | Copyright © 2018-2024 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V.

Clone this wiki locally