How to use VVdeC

To use the decoder, please download and build the repository as explained in the Build section. This will build a library and an application.

The application can be used to decompress raw VVC bitstreams into y4m or raw YUV files. Please see the help (vvdecapp --help) for more information.

Usage with FFmpeg

Please see the guide in VVenC wiki for instruction on building FFmpeg with VVenC and VVdeC support.

Obsolete: Please refer to [1] for instructions on how to include VVC support into standard open source software.

Build MPV-Android with VVC support

Quick build using a rather old VVC FFmpeg integration

• Install build prerequisites (debian/ubuntu)

  sudo apt-get install default-jdk-headless wget
  

• Get the MPV-Android source with patched buildscripts

  git clone https://github.com/tbiat/mpv-android
  cd mpv-android/buildscripts/
  git checkout vvdec
  

• The build scripts try to install the latest version of Meson system wide using pip. This is not allowed in recent python versions (at least on debian), so we setup a virtual environment, and patch the scripts accordingly. You can skip this step if you have a recent version of Meson installed already.

  sudo apt-get install python3-venv
  python3 -m venv venv
  . ./venv/bin/activate
  sed 's/sudo pip/pip/' -i include/download-sdk.sh
  

• Download all remaining dependencies and Android SDK & NDK

  ./download.sh
  

  [!NOTE] Recommended: replace the FFmpeg sources as explained here

• We need to disable python support in the libxml build

  sed 's/--disable-require-system-font-provider/\0 --without-python/' -i scripts/libxml2.sh
  

• Switch the MPV-Android build to use Meson

  git fetch https://github.com/mpv-android/mpv-android.git
  git cherry-pick f7d284fcb
  

• Start the actual build. This builds MPV-Android for arm64 only.

  ./buildall.sh
  

  • to add further architectures to the generated APK (e.g. x86_64), call the build script again like this:

    ./buildall.sh --arch x86_64
    

Note about old version of the FFmpeg VVC patchset

These instructions use a rather old version of the patches integrating VVdeC into FFmpeg.

The MPV-Android build scripts described here download the FFmpeg sources to mpv-android/buildscripts/deps/ffmpeg/. You can simply replace these FFmpeg sources after the download step with a more recent patched version.

The VVenC wiki explains how to apply the latest VVC patchset for FFmpeg: (steps 1.-4. only, don't build anything)
https://github.com/fraunhoferhhi/vvenc/wiki/FFmpeg-Integration#linux

Standalone decoder application (deprecated)

In addition to the C-library, the VVdeC project provides a sample application, which allows to decode raw VVC bitstreams into YUV4MPEG2(y4m) or raw YUV files (for 10-bit input, the output is yuv420p10le format in ffmpeg; for 8-bit input, the output is yuv420p).

Table I: List of important decoder options. For full list please use the --help option.

OPTION	DEFAULT	DESCRIPTION
--help,-h	-	Show help
--bitstream,-b	-	Raw bitstream input file
--output,-o	-	The name of the raw yuv output file (yuv420p10le format), use - to pipe output (e.g. -o -)
--y4m	-	Force y4m output (for pipe output; auto enable for .y4m output file extension)
--threads,-t	-1	Size of the threadpool allocated for decoding. Set to 0 for single-threaded execution and -1 to allocate one thread per core available
--verbosity,-v	3	Verbosity level
--parsedelay,-p	-1	Expert: maximal number of parsed frames waiting for reconstruction. Increases the decoding latency but improves MT scaling. Set to -1 to allow one parse frame delay per allocated thread.
--SEIDecodedPictureHash,-dph	not set	Expert: if the bitstream contains Decoded Picture Hash SEI information, the switch enables the decoder to check the values against the reconstruction
--loops,-L	0	Expert: decode the bitstream file multiple times
--CheckYuvMD5,-md5	-	Expert: check the decoded YUV against a given MD5 hash value (used for testing)

Example usage

• Given a compliant VVC input file str.266, the following call will decode the file into raw YUV raw_yuv.yuv:

vvdecapp -b str.266 -o raw_yuv.yuv

• Decode into YUV4MPEG2 (y4m):

vvdecapp -b str.266 -o raw_yuv.y4m

• Pipe decoded output (e.g. into VVenC)

vvdecapp -b str.266 --y4m -o - | vvencapp -i - -o newstr.266 --y4m

Library

The VVdeC project provides an easy to use C-library. This section gives a rough overview of how to use the VVdeC library. For simplicity, it assumes the vvdec/vvdec.h header is included and the vvdec.lib or libvvdec.a library is linked statically into the application.

For a full application example, see vvdecapp sources.

The following steps are required to use the decoder:

1. Initialize the decoder:

vvdecParams params;
vvdec_params_default( &params );
params.logLevel = VVDEC_INFO;
vvdecDecoder* decoder = vvdec_decoder_open( &params );

2. Allocate and initialize the access unit storage. We assume MaxNaluSize is set to the maximum size of a NAL unit appearing in the bitstream.

vvdecAccessUnit* au = vvdec_accessUnit_alloc();
vvdec_accessUnit_default( au ); 
vvdec_accessUnit_alloc_payload( au, MaxNaluSize );

3. Read one NAL unit from the bitstream (not shown here), assign it to the access unit payload, and set the payloadUsedSize accordingly.

memcpy( au->payload, &bitstream[naluStart], naluSize );
au->payloadUsedSize = naluSize;

4. Pass the access unit to the decoder. The decoder will return VVDEC_TRY_AGAIN if it needs more data, or VVDEC_OK when a decoded frame was produced. In that case, the provided frame pointer points to the latter. After the application is done processing the frame it calls vvdec_frame_unref() to allow the decoder to reuse the frame storage.

vvdecFrame* frame = nullptr;
int ret   = vvdec_decode( decoder, au, &frame );
if( ret != VVDEC_OK && ret != VVDEC_TRY_AGAIN ) {
    return -1;    // abort on error for simplicity
}
if ( frame ) {
  // TODO:
  //   process the decoded frame (e.g. display, write to file)

  vvdec_frame_unref( decoder, frame );
}

5. Repeat from steps 3 and 4 until all NAL units have been passed to the decoder. Then start flushing the decoder (step 6).
6. Wait for and extract next decoded frame from the decoder. As before, the decoder will return VVDEC_OK, when a frame is produced. When all frames have been decoded VVDEC_EOF will signal the end of the sequence.

vvdecFrame* frame = nullptr;
int ret = vvdec_flush( decoder, &frame );
if( ret != VVDEC_OK && ret != VVDEC_EOF ) {
  return -1;    // abort on error for simplicity
}
if( frame ) {
  // TODO:
  //   process the decoded frame (e.g. display, write to file)
  vvdec_frame_unref( decoder, frame );
}

7. Repeat step 6, while the decoder returns VVDEC_OK.
8. Free decoder and access unit storage.

vvdec_accessUnit_free( au );
vvdec_decoder_close( decoder );

WebAssembly Runtime

VVdeC explicitly supports WebAssembly as a compilation target using Emscripten. The Emscripten SDK needs to be installed, activated and in the PATH as documented on the Emscripten site (https://emscripten.org/). Then, building VVdeC for the WebAssembly runtime is straightforward:

emcmake cmake -B build/wasm
cmake --build build/wasm

The resulting binary and support files vvdecapp.wasm, vvdeapp.worker.js, and vvdecapp.js can be included in a website to build a browser based VVC player. The WASM module exposes an interface similar to the native library but slightly more object-oriented to be easier to use from Javascript code running in the browser.

For a full application example, see vvcWebPlayer sources.

The objects like Decoder, AccessUnit, and decoder Parameters are instantiated using the JavaScript “new” operator and deleted using the attached .delete() functions. The interface can be used like this:

1. Instantiate the WASM module and wrapper:

const module_config = {
    mainScriptUrlOrBlob: "path/to/vvdecapp.js",
};
const VVdeC = await CreateVVdeC(module_config);

2. Create the decoder instance, AccessUnit, and FrameHandle. The latter is a tiny class to receive frames from the decoder instead of the double pointer in the native library.

const params = new VVdeC.Params();
params.threads = 10;
const dec = new VVdeC.Decoder(params);
params.delete();
const au = new VVdeC.AccessUnit();
au.alloc_payload(100000);
const frameHandle = new VVdeC.FrameHandle();

3. Copy NAL unit data to AccessUnit payload, and pass to the decoder. The decode() function call accepts a FrameHandle object to receive the produced fame. The data pointers of the frame’s planes are represented as Uint8- or Uint16Arrays depending of the bit depth of the Frame.

au.payload.set(nalu);
au.payloadUsedSize = nalu.byteLength;
let ret = dec.decode(au, frameHandle);
if (ret !== 0 && ret !== -40) {
  return -1;
}
if (frameHandle.frame) {
  // TODO:
  //   process the decoded frame

  dec.frame_unref(frameHandle.frame);
}

4. Repeat step 3 until all NAL units have been passed to the decoder.
5. Repeatedly flush the decoder, until it returns -50 (EOF).

int ret = dec.flush(frameHandle);
if (ret !== 0 && ret !== -50) {
  return -1;
}

if (frameHandle.frame) {
  // TODO:
  //   process the decoded frame

  dec.frame_unref(frameHandle.frame);
}

6. Release the decoder, access unit and frame handle.

dec.delete();
au.delete();
frameHandle.delete();

Known issues

Single-threaded execution

Problem: when starting the decoder with --threads 1, I notice that at times more than one core is utilized.

Solution: because of the semantics of the threads parameter, a thread pool of size 1 is allocated. Use --threads 0 not to allocate a thread-pool and execute the decoding in the main thread (possible since version v0.1.2.0).

References

• [1] A. Wieckowski, C. Lehmann, B. Bross, D. Marpe, T. Biatek, M. Raulet, and J. Le Feuvre, "A Complete End-To-End Open Source Toolchain for the Versatile Video Coding (VVC) Standard," 29th ACM International Conference on Multimedia (MM’21), 2021. doi: 10.1145/3474085.3478320