VP8 decoder drop frame on ICL

[sunxinpeng]

For the same test clip, I use ffmpeg-vaapi vp8 decoder to decode on SKL and ICL. My OS is Ubuntu 18.04 and decoding passed on SKL but dropped frame on ICL. The cmd line log is as below:

1. For SKL:

sakura@skl-sakura:~/sxp-workdir/ffmpeg$ ffmpeg -hwaccel vaapi -init_hw_device vaapi=hw:/dev/dri/renderD128 -filter_hw_device hw -v verbose -i /mnt/vp8/vp8_rev05_InterFrame_Profile1_121_shields0.webm -pix_fmt yuv420p -f rawvideo -vsync passthrough -vframes 2 -y /home/sakura/sxp-workdir/test_sample/dump.yuv
ffmpeg version N-95347-gc8813b1a98 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.4.0-1ubuntu1~18.04.1)
  configuration: --enable-vaapi --prefix=/home/sakura/sxp-workdir/opt/X11R7/iHD/ffmpeg_mainline --pkg-config-flags=--static --enable-nonfree --enable-libmfx --enable-gpl --enable-libx264 --enable-libx265 --disable-optimizations --enable-debug=3
  libavutil      56. 35.100 / 56. 35.100
  libavcodec     58. 59.102 / 58. 59.102
  libavformat    58. 33.100 / 58. 33.100
  libavdevice    58.  9.100 / 58.  9.100
  libavfilter     7. 62.100 /  7. 62.100
  libswscale      5.  6.100 /  5.  6.100
  libswresample   3.  6.100 /  3.  6.100
  libpostproc    55.  6.100 / 55.  6.100
[AVHWDeviceContext @ 0x55b61e10f340] libva: VA-API version 1.6.0
[AVHWDeviceContext @ 0x55b61e10f340] libva: va_getDriverName() returns 0
[AVHWDeviceContext @ 0x55b61e10f340] libva: User requested driver 'iHD'
[AVHWDeviceContext @ 0x55b61e10f340] libva: Trying to open /home/sakura/sxp-workdir/opt/X11R7/iHD/vpg_driver/lib/dri//iHD_drv_video.so
[AVHWDeviceContext @ 0x55b61e10f340] libva: Found init function __vaDriverInit_1_6
[AVHWDeviceContext @ 0x55b61e10f340] libva: va_openDriver() returns 0
[AVHWDeviceContext @ 0x55b61e10f340] Initialised VAAPI connection: version 1.6
[AVHWDeviceContext @ 0x55b61e10f340] VAAPI driver: Intel iHD driver - 1.0.0.
[AVHWDeviceContext @ 0x55b61e10f340] Driver not found in known nonstandard list, using standard behaviour.
Input #0, matroska,webm, from '/mnt/vp8/vp8_rev05_InterFrame_Profile1_121_shields0.webm':
  Metadata:
    encoder         : vpxenc v0.9.6
  Duration: 00:00:00.07, start: 0.000000, bitrate: 162 kb/s
    Stream #0:0(eng): Video: vp8, 1 reference frame, yuv420p(progressive), 48x48, SAR 1:1 DAR 1:1, 30 fps, 30 tbr, 1k tbn, 1k tbc (default)
Stream mapping:
  Stream #0:0 -> #0:0 (vp8 (native) -> rawvideo (native))
Press [q] to stop, [?] for help
[LIBVA]:CRITICAL - DdiMedia_SyncSurface:3425: nullptr codecHal->pDecoder
[graph 0 input from stream 0:0 @ 0x55b61e17f980] w:48 h:48 pixfmt:nv12 tb:1/1000 fr:30/1 sar:1/1 sws_param:flags=2
[auto_scaler_0 @ 0x55b61e183080] w:iw h:ih flags:'bicubic' interl:0
[format @ 0x55b61e180340] auto-inserting filter 'auto_scaler_0' between the filter 'Parsed_null_0' and the filter 'format'
[auto_scaler_0 @ 0x55b61e183080] w:48 h:48 fmt:nv12 sar:1/1 -> w:48 h:48 fmt:yuv420p sar:1/1 flags:0x4
Output #0, rawvideo, to '/home/sakura/sxp-workdir/test_sample/dump.yuv':
  Metadata:
    encoder         : Lavf58.33.100
    Stream #0:0(eng): Video: rawvideo, 1 reference frame (I420 / 0x30323449), yuv420p, 48x48 [SAR 1:1 DAR 1:1], q=2-31, 829 kb/s, 30 fps, 30 tbn, 30 tbc (default)
    Metadata:
      encoder         : Lavc58.59.102 rawvideo
No more output streams to write to, finishing.
frame=    2 fps=0.0 q=-0.0 Lsize=       7kB time=00:00:00.06 bitrate= 829.4kbits/s speed=17.3x    
video:7kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%
Input file #0 (/mnt/vp8/vp8_rev05_InterFrame_Profile1_121_shields0.webm):
  Input stream #0:0 (video): 2 packets read (981 bytes); 2 frames decoded; 
  Total: 2 packets (981 bytes) demuxed
Output file #0 (/home/sakura/sxp-workdir/test_sample/dump.yuv):
  Output stream #0:0 (video): 2 frames encoded; 2 packets muxed (6912 bytes); 
  Total: 2 packets (6912 bytes) muxed
[AVIOContext @ 0x55b61e15ff80] Statistics: 0 seeks, 1 writeouts
[AVIOContext @ 0x55b61e1572c0] Statistics: 1341 bytes read, 0 seeks

2. For ICL:

root@media_libva_icl2:~/build/ffmpeg# ffmpeg -hwaccel vaapi -init_hw_device vaapi=hw:/dev/dri/renderD128 -filter_hw_device hw -v verbose -i /root/xinpeng_test/vp8_rev05_InterFrame_Profile1_121_shields0.webm -pix_fmt yuv420p -f rawvideo -vsync passthrough -vframes 2 -y /root/xinpeng_test/dump.yuv
ffmpeg version N-95347-gc8813b1a98 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.10) 20160609
  configuration: --enable-vaapi --prefix=/opt/X11R7/iHD/ffmpeg_mainline --enable-nonfree --enable-shared --enable-libmfx --enable-gpl --enable-libx264 --enable-libx265 --disable-optimizations --enable-debug=3 --disable-stripping
  libavutil      56. 35.100 / 56. 35.100
  libavcodec     58. 59.102 / 58. 59.102
  libavformat    58. 33.100 / 58. 33.100
  libavdevice    58.  9.100 / 58.  9.100
  libavfilter     7. 62.100 /  7. 62.100
  libswscale      5.  6.100 /  5.  6.100
  libswresample   3.  6.100 /  3.  6.100
  libpostproc    55.  6.100 / 55.  6.100
[AVHWDeviceContext @ 0x1551340] libva: VA-API version 1.6.0
[AVHWDeviceContext @ 0x1551340] libva: va_getDriverName() returns 0
[AVHWDeviceContext @ 0x1551340] libva: User requested driver 'iHD'
[AVHWDeviceContext @ 0x1551340] libva: Trying to open /opt/X11R7/iHD/vpg_driver/lib/dri//iHD_drv_video.so
[AVHWDeviceContext @ 0x1551340] libva: Found init function __vaDriverInit_1_6
[AVHWDeviceContext @ 0x1551340] libva: va_openDriver() returns 0
[AVHWDeviceContext @ 0x1551340] Initialised VAAPI connection: version 1.6
[AVHWDeviceContext @ 0x1551340] VAAPI driver: Intel iHD driver - 1.0.0.
[AVHWDeviceContext @ 0x1551340] Driver not found in known nonstandard list, using standard behaviour.
Input #0, matroska,webm, from '/root/xinpeng_test/vp8_rev05_InterFrame_Profile1_121_shields0.webm':
  Metadata:
    encoder         : vpxenc v0.9.6
  Duration: 00:00:00.07, start: 0.000000, bitrate: 162 kb/s
    Stream #0:0(eng): Video: vp8, 1 reference frame, yuv420p(progressive), 48x48, SAR 1:1 DAR 1:1, 30 fps, 30 tbr, 1k tbn, 1k tbc (default)
Stream mapping:
  Stream #0:0 -> #0:0 (vp8 (native) -> rawvideo (native))
Press [q] to stop, [?] for help
[graph 0 input from stream 0:0 @ 0x15b9040] w:48 h:48 pixfmt:nv12 tb:1/1000 fr:30/1 sar:1/1 sws_param:flags=2
[auto_scaler_0 @ 0x15bda40] w:iw h:ih flags:'bicubic' interl:0
[format @ 0x15ba280] auto-inserting filter 'auto_scaler_0' between the filter 'Parsed_null_0' and the filter 'format'
[auto_scaler_0 @ 0x15bda40] w:48 h:48 fmt:nv12 sar:1/1 -> w:48 h:48 fmt:yuv420p sar:1/1 flags:0x4
Output #0, rawvideo, to '/root/xinpeng_test/dump.yuv':
  Metadata:
    encoder         : Lavf58.33.100
    Stream #0:0(eng): Video: rawvideo, 1 reference frame (I420 / 0x30323449), yuv420p, 48x48 [SAR 1:1 DAR 1:1], q=2-31, 829 kb/s, 30 fps, 30 tbn, 30 tbc (default)
    Metadata:
      encoder         : Lavc58.59.102 rawvideo
[MOS]:  CRITICAL - Mos_Specific_GetResourceInfo:2401: Invalid (nullptr) Pointer.
[MHW]:  CRITICAL - AddMfxPipeBufAddrCmd:999: MOS returned error, eStatus = 0x5
[CODEC]:CRITICAL - DecodeStateLevel:215: MOS returned error, eStatus = 0x5
[CODEC]:CRITICAL - Execute:1281: State level decoding failed.
[LIBVA]:CRITICAL - EndPicture:809: DDI:DdiDecode_DecodeInCodecHal return failure.
[vp8 @ 0x15a9a40] Failed to end picture decode issue: 23 (internal decoding error).
Error while decoding stream #0:0: Input/output error
No more output streams to write to, finishing.
frame=    1 fps=0.0 q=-0.0 Lsize=       3kB time=00:00:00.03 bitrate= 829.4kbits/s speed=  11x    
video:3kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%
Input file #0 (/root/xinpeng_test/vp8_rev05_InterFrame_Profile1_121_shields0.webm):
  Input stream #0:0 (video): 2 packets read (981 bytes); 1 frames decoded; 
  Total: 2 packets (981 bytes) demuxed
Output file #0 (/root/xinpeng_test/dump.yuv):
  Output stream #0:0 (video): 1 frames encoded; 1 packets muxed (3456 bytes); 
  Total: 1 packets (3456 bytes) muxed
[AVIOContext @ 0x15a6780] Statistics: 0 seeks, 1 writeouts
[AVIOContext @ 0x156fc40] Statistics: 1341 bytes read, 0 seeks

[ss22ongithub]

Hi
I observed the same error on JSL (gen 11) as well here is the log

localhost /media # ffmpeg -hide_banner -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -v verbose -i  /usr/local/autotest/cros/video/files/720_vp8.webm  -pix_fmt yuv420p  -f rawvideo -vsync passthrough -y 720_vp8.yuv >ffmpeg.txt
[AVHWDeviceContext @ 0x57fff5c28610] Opened VA display via DRM device /dev/dri/renderD128.
[AVHWDeviceContext @ 0x57fff5c28610] libva: Open new log file /home/chronos/user/libva.trace.165353.thd-0x0000230e for the thread 0x0000230e
[AVHWDeviceContext @ 0x57fff5c28610] libva: LIBVA_TRACE is on, save log into /home/chronos/user/libva.trace.165353.thd-0x0000230e
[AVHWDeviceContext @ 0x57fff5c28610] libva: VA-API version 1.6.0
[AVHWDeviceContext @ 0x57fff5c28610] libva: va_getDriverName() returns 0
[AVHWDeviceContext @ 0x57fff5c28610] libva: Trying to open /usr/lib64/va/drivers/iHD_drv_video.so
[AVHWDeviceContext @ 0x57fff5c28610] libva: Found init function __vaDriverInit_1_6
[AVHWDeviceContext @ 0x57fff5c28610] libva: va_openDriver() returns 0
[AVHWDeviceContext @ 0x57fff5c28610] Initialised VAAPI connection: version 1.6
[AVHWDeviceContext @ 0x57fff5c28610] VAAPI driver: Intel iHD driver - 19.4.nov12.
[AVHWDeviceContext @ 0x57fff5c28610] Driver not found in known nonstandard list, using standard behaviour.
Input #0, matroska,webm, from '/usr/local/autotest/cros/video/files/720_vp8.webm':
  Metadata:
    encoder         : Lavf54.20.4
  Duration: 00:00:20.02, start: 0.000000, bitrate: 842 kb/s
    Stream #0:0(eng): Video: vp8, 1 reference frame, yuv420p(progressive), 1280x720, SAR 1:1 DAR 16:9, 1k fps, 29.97 tbr, 1k tbn, 1k tbc (default)
Stream mapping:
  Stream #0:0 -> #0:0 (vp8 (native) -> rawvideo (native))
Press [q] to stop, [?] for help
[AVHWDeviceContext @ 0x57fff5c28610] libva: Open new log file /home/chronos/user/libva.trace.165353.thd-0x00002311 for the thread 0x00002311
[AVHWDeviceContext @ 0x57fff5c28610] libva: Save context 0x10000000 into log file /home/chronos/user/libva.trace.165353.thd-0x00002311
[AVHWDeviceContext @ 0x57fff5c28610] libva: Open new log file /home/chronos/user/libva.trace.165353.thd-0x00002312 for the thread 0x00002312
[AVHWDeviceContext @ 0x57fff5c28610] libva: Save context 0x10000000 into log file /home/chronos/user/libva.trace.165353.thd-0x00002312
[vp8 @ 0x57fff5c92030] Failed to end picture decode issue: 23 (internal decoding error).
[AVHWDeviceContext @ 0x57fff5c28610] libva: Open new log file /home/chronos/user/libva.trace.165353.thd-0x00002313 for the thread 0x00002313
[AVHWDeviceContext @ 0x57fff5c28610] libva: Save context 0x10000000 into log file /home/chronos/user/libva.trace.165353.thd-0x00002313
[vp8 @ 0x57fff5c94c70] Failed to end picture decode issue: 23 (internal decoding error).
[AVHWDeviceContext @ 0x57fff5c28610] libva: Open new log file /home/chronos/user/libva.trace.165353.thd-0x00002314 for the thread 0x00002314
[AVHWDeviceContext @ 0x57fff5c28610] libva: Save context 0x10000000 into log file /home/chronos/user/libva.trace.165353.thd-0x00002314
[vp8 @ 0x57fff5c97990] Failed to end picture decode issue: 23 (internal decoding error).
[AVHWDeviceContext @ 0x57fff5c28610] libva: Open new log file /home/chronos/user/libva.trace.165353.thd-0x00002315 for the thread 0x00002315
[graph 0 input from stream 0:0 @ 0x57fff5c9eee0] w:1280 h:720 pixfmt:nv12 tb:1/1000 fr:30000/1001 sar:1/1 sws_param:flags=2
[auto_scaler_0 @ 0x57fff5ca0c20] w:iw h:ih flags:'bicubic' interl:0
[format @ 0x57fff5c9f8a0] auto-inserting filter 'auto_scaler_0' between the filter 'Parsed_null_0' and the filter 'format'
[auto_scaler_0 @ 0x57fff5ca0c20] w:1280 h:720 fmt:nv12 sar:1/1 -> w:1280 h:720 fmt:yuv420p sar:1/1 flags:0x4
Output #0, rawvideo, to '720_vp8.yuv':
  Metadata:
    encoder         : Lavf58.20.100
    Stream #0:0(eng): Video: rawvideo, 1 reference frame (I420 / 0x30323449), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 331444 kb/s, 29.97 fps, 29.97 tbn, 29.97 tbc (default)
    Metadata:
      encoder         : Lavc58.35.100 rawvideo
[AVHWDeviceContext @ 0x57fff5c28610] libva: Save context 0x10000000 into log file /home/chronos/user/libva.trace.165353.thd-0x00002315
Error while decoding stream #0:0: Input/output error
[vp8 @ 0x57fff5c9a6b0] Failed to end picture decode issue: 23 (internal decoding error).
Error while decoding stream #0:0: Input/output error
[vp8 @ 0x57fff5c74c70] Failed to end picture decode issue: 23 (internal decoding error).
Error while decoding stream #0:0: Input/output error
[vp8 @ 0x57fff5c92030] Failed to end picture decode issue: 23 (internal decoding error).
Error while decoding stream #0:0: Input/output error
[vp8 @ 0x57fff5c94c70] Failed to end picture decode issue: 23 (internal decoding error).
Error while decoding stream #0:0: Input/output error
[vp8 @ 0x57fff5c97990] Failed to end picture decode issue: 23 (internal decoding error).
Error while decoding stream #0:0: Input/output error
[vp8 @ 0x57fff5c9a6b0] Failed to end picture decode issue: 23 (internal decoding error).
Error while decoding stream #0:0: Input/output error
[vp8 @ 0x57fff5c74c70] Failed to end picture decode issue: 23 (internal decoding error).
Error while decoding stream #0:0: Input/output error
[vp8 @ 0x57fff5c92030] Failed to end picture decode issue: 23 (internal decoding error).
Error while decoding stream #0:0: Input/output error
[vp8 @ 0x57fff5c94c70] Failed to end picture decode issue: 23 (internal decoding error).
Error while decoding stream #0:0: Input/output error
[vp8 @ 0x57fff5c97990] Failed to end picture decode issue: 23 (internal decoding error).
Error while decoding stream #0:0: Input/output error
[vp8 @ 0x57fff5c9a6b0] Failed to end picture decode issue: 23 (internal decoding error).
Error while decoding stream #0:0: Input/output error
[vp8 @ 0x57fff5c74c70] Failed to end picture decode issue: 23 (internal decoding error).
Error while decoding stream #0:0: Input/output error
[vp8 @ 0x57fff5c92030] Failed to end picture decode issue: 23 (internal decoding error).
Error while decoding stream #0:0: Input/output error
[vp8 @ 0x57fff5c94c70] Failed to end picture decode issue: 23 (internal decoding error).
Error while decoding stream #0:0: Input/output error
[vp8 @ 0x57fff5c97990] Failed to end picture decode issue: 23 (internal decoding error).
Error while decoding stream #0:0: Input/output error
    Last message repeated 2 times
No more output streams to write to, finishing.e=00:00:19.01 bitrate=320874.5kbits/s speed=3.78x
frame=  582 fps=110 q=-0.0 Lsize=  785700kB time=00:00:20.02 bitrate=321501.2kbits/s speed=3.77x
video:785700kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%
Input file #0 (/usr/local/autotest/cros/video/files/720_vp8.webm):
  Input stream #0:0 (video): 600 packets read (2103789 bytes); 582 frames decoded;
  Total: 600 packets (2103789 bytes) demuxed
Output file #0 (720_vp8.yuv):
  Output stream #0:0 (video): 582 frames encoded; 582 packets muxed (804556800 bytes);
  Total: 582 packets (804556800 bytes) muxed
[AVIOContext @ 0x57fff5c73fb0] Statistics: 0 seeks, 3070 writeouts
[AVIOContext @ 0x57fff5c468f0] Statistics: 2108575 bytes read, 0 seeks

The error :

[MOS]:  CRITICAL - Mos_Specific_GetResourceInfo:2440: Invalid (nullptr) Pointer.
[MHW]:  CRITICAL - AddMfxPipeBufAddrCmd:998: MOS returned error, eStatus = 0x5
[CODEC]:CRITICAL - DecodeStateLevel:215: MOS returned error, eStatus = 0x5
[CODEC]:CRITICAL - Execute:1281: State level decoding failed.
[LIBVA]:CRITICAL - EndPicture:809: DDI:DdiDecode_DecodeInCodecHal return failure.
[LIBVA]:EXIT     - DdiDecode_EndPicture: eStatus = 0x17

The complete logs are here
ffmpeg.txt

[weizhu-intel]

Hi Sunxinpeng,
I used your command to reproduce, and found that:

1. VA context has 16 target buffer, but vaBeginPicture function pass an VASurfaceID with 31 when decoding second frame
2. vaCreateContext has been called again when decoding second frame, this is not expected

Could you double check the call sequence on VAAPI in ffmpeg plugin?

Thanks
zhu wei

[sunxinpeng]

This issue has been fixed in driver. Closed.