Intel ffmpeg patches upstream 2023q4 queue

ffmpeg-vaapi and ffmpeg-qsv upstream patches commits tested:

• cartwheel-ffmpeg: https://github.com/intel/cartwheel-ffmpeg/releases/tag/2023q4 (tag: 2023q4)

Supported Intel Platforms:

• MTL (Meteor Lake)
• ATS-M (Arctic Sound)
• DG2 (Discrete Graphics 2)
• RPL (Raptor Lake-S/P)
• ADL-S (Alder Lake-S/P/N)
• SG1/DG1
• RKL (Rocket Lake)
• TGL (Tiger Lake)
• ICL (Ice Lake)
• JSL (Jasper Lake)/EHL (Elkhart Lake)
• KBLx (KBL/Kaby Lake; CFL/Coffee Lake; WHL/Whiskey Lake; CML/Comet Lake; AML/Amber Lake)

Tested Features:

• Decode: AVC/H264, HEVC/H265 (8, 10, 12bit), AV1 (8, 10bit), VP9 (8/10/12bit), VP8, JPEG/MJPEG, MPEG2
• Encode: AV1(8, 10bit), AVC/H264, HEVC/H265 (8, 10bit), VP9 (8, 10bit), VP8, JPEG/MJPEG, MPEG2
• VPP: procamp (brightness/contrast/saturation/hue), csc, deinterlace, denoise, scale, sharpen, mirroring, rotation, transpose

Supported Features among Intel platforms:

For features supported on each Intel platform, please refer to links below:

• decode/encode features: https://github.com/intel/media-driver/#decodingencoding-features
• VPP features: https://github.com/intel/media-driver/#video-processing-features

Build configures

FFmpeg build configure:

• #./configure --prefix=$ROOT_INSTALL_DIR --enable-shared --enable-vaapi --enable-libvpl
• #make
• #make install

Reference Configure Used: Intel Libva/iHD driver, MediaSDK, VPL and VPL GPU Runtime

• VPL Dispatcher: https://github.com/intel/libvpl/commits/v2023.4.0/
• VPL Runtime: https://github.com/oneapi-src/oneVPL-intel-gpu/releases/tag/intel-onevpl-23.4.3
• Media SDK: Intel-Media-SDK/MediaSDK@7a72de3
• Media driver: https://github.com/intel/media-driver/commits/intel-media-23.4.3/
• Libva: https://github.com/intel/libva/releases/tag/2.20.0
• Libva-utils: https://github.com/intel/libva-utils/commits/2.20.1
• Gmmlib: https://github.com/intel/gmmlib/commits/intel-gmmlib-22.3.15/
  Binary:
  these components binary of media stack (LibVA/iHD/VPL/GmmLib) can be found from https://github.com/oneapi-src/oneVPL-intel-gpu/releases/tag/intel-onevpl-23.4.3

Contactors:

Please contact open.source.media.framework@intel.com, if you have any question.

New Detail

2023Q4:

• FFmpeg added Intel new MTL platform support
• ffmpeg-dnn added support of openvino yolov1 v2 v3 v4
• ffmpeg-dnn added support of SSD model with two outputs tensors
• ffmpeg-dnn added support of model with dynamic output tensor shape
• ffmpeg-dnn libtorch backend added support for using multi-devices to accelerate inference
• both ffmpeg-vaapi and ffmpeg-qsv can use user-specified card on multi-gpu platform (such as dGPU+iGPU configure)
• ffmpeg-vaapi and ffmpeg-qsv added icq encode mode support

2023Q3:

• ffmpeg-vaapi and qsv support dynamic frame buffer pool for vpp and decode instead of using fixed number buffer
• ffmpeg-dnn: add scale and layout options to do pre/post-processing for openvino
• ffmpeg-dnn: add multi-frame support for libtorch to support BasicVSR, BasicVSR++ and RVRT

2023Q2:

• ffmpeg-vaapi added passthrough mode for lavfi/{denoise, procamp, scale, sharpness}_vaapi filters
• ffmpeg-vaapi/qsv add auto hwmap filter
• ffmpeg-qsv add openvino2.0 support

2023Q1:
FFmpeg:

• ffmpeg-qsv and ffmpeg-vaapi added hstack_qsv, vstack_qsv and xstack_qsv filter support
• ffmpeg-qsv and ffmpeg-vaapi added gopconcat bsf support. This filter will concat gop from different streams to make a new stream, which helps to run encoder on multi-gpu platform
• ffmpeg-vaapi support RGB* format clips for vp9 decode

2022Q4:

• ffmpeg added new RPL-P platform support
• ffmpeg added new ADL-N platform support
• ffmpeg-qsv added av1 encode adaptive_i/b support
• ffmpeg-qsv added hevc encode support HDR info support
• ffmpeg-qsv added hevc encode 12bit (p012, y212, xv36) support
• ffmpeg-qsv decoder support to ouput hdr info
• ffmpeg-qsv change gop option to ”open“ by default
• ffmpeg-qsv added vpp_qsv scale mode can be applied to color conversion
• ffmpeg-qsv added vpp_qsv support p010 system memory input
• ffmpeg-vaapi refined overlay vaapi filter and added more options like framesync
• ffmpeg-vaapi added hevc encode 12bit (p012, y212, xv36) support

2022Q3:

• FFmpeg added new RPL-S platform support
• ffmpeg-vaapi added av1 encode support
• ffmpeg-vaapi can select related hw codec to do decode automatically, a example cmd line for decode "ffmpeg -hwccel vaapi -i av1.mp4 -f null -"
• ffmpeg-qsv h265 encode added adaptive_i/b support
• ffmpeg-qsv vpp scaling add model select through oneVPL which supports EU, VDBOX or VEBOX. User can use option of "scale_mode" to select which hardware model
• ffmpeg dnn: OpenVINO full GPU detect and classify support
• ffmpeg dnn: enabled libtorch as one of FFmpeg DNN backend

2022Q2:

• ffmpeg-vaapi added VDSFC Scaling support
• ffmpeg-vaapi added max_frame_size for encode
• ffmpeg-vaapi added GPB frame support for encode
• ffmpeg-qsv added adaptive_i and adaptive_b support for hevc encode
• ffmpeg-qsv added min/max qp control for I/P/B frames
• ffmpeg-qsv added ROI support for h264/h265 encode
• ffmpeg-qsv supported yuyv422/y210 format direct input mode for encode
• ffmpeg added gaussian filter through Vulkan
• ffmpeg added stitching filter through libxcam library
• ffmpeg video analysis added BasicVSR super resolution support
• ffmpeg video analysis added FRVSR super resolution

2022Q1:

• ffmpeg_qsv added av1 encode support
• supported av1 decode frame split bsf feature in ffmpeg core
• fixed a hevc 10bit run2run issue on some legacy platforms for ffmpeg_qsv

2021Q4:

• buffer sharing between media and vulkan
• support 0YUV, YUY2, Y210, Y410, P012, Y212, Y412 on Windows when using d3d11 device
• hevc main10 sp encoding