Skip to content
This repository has been archived by the owner on May 17, 2023. It is now read-only.

[SKL][VDENC] FFMPEG AVC VDENC unsupported #1010

Closed
fulinjie opened this issue Dec 13, 2018 · 8 comments
Closed

[SKL][VDENC] FFMPEG AVC VDENC unsupported #1010

fulinjie opened this issue Dec 13, 2018 · 8 comments

Comments

@fulinjie
Copy link
Contributor

AVC VDENC failed on SKL, and same cmdline works on KBL.

CMD:

ffmpeg -report -v verbose -hwaccel qsv -c:v h264_qsv -i input.mp4 -c:v h264_qsv -low_power 1 -flags qscale -global_quality 20 -y out_lowpower.mp4

or

ffmpeg -hwaccel qsv -init_hw_device qsv=hw -filter_hw_device hw -v verbose -f rawvideo -video_size 1280x720 -pix_fmt yuv420p -i /root/media_streams//encoderbitstreams//stockholm_1280x720_604.yuv -an -vf hwupload=extra_hw_frames=64,format=qsv -c:v h264_qsv -low_power 1 -profile:v main -flags qscale -global_quality 20 -vframes 100 -y ./h264e_cqp_1280x720_gop30_refdist3_qp40.h264

Environment:
ffmpeg: 1046cba24be41c624489f40513aeaba7e52ca9ff
VDENC patch: https://patchwork.ffmpeg.org/patch/11217/
(apply will be reject because of the update in qsvenc.c, you can cope with the conflicts or reset ffmpeg version to b44a571dc4af929f19715c9faad47792a0812bf1 then apply the patch )

Media-driver: 43282360d2ff251f051cc2d751503715c7177300
MediaSDK: 5c1c312

Error message:
[h264_qsv @ 0x1561b80] Error initializing the encoder: unsupported (-3)
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
@fulinjie
Copy link
Contributor Author

And Vaapi works on SKL environment:
ffmpeg -v verbose -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -f rawvideo -video_size 1280x720 -pix_fmt yuv420p -i /root/media_streams//encoderbitstreams//stockholm_1280x720_604.yuv -an -vf forma t=nv12,hwupload -c:v h264_vaapi -low_power 1 -profile:v main -flags qscale -global_quality 20 -vframes 100 -y ./lowpower.h264

@onabiull
Copy link
Contributor

I suppose that vaapi implementation just silently skip low_power option. According to encoder support page https://github.com/intel/media-driver only E1 supported on SKL, which is VME-based encode. Low power introduced in APL and KBL.

@fulinjie
Copy link
Contributor Author

https://github.com/intel/media-driver/blob/master/README.md#supported-codecs
Supported Codecs
CODEC | BDW | SKL | BXT/APL | KBL | CFL | CNL | ICL*
AVC | D/E1 | D/E1 | D/E1/E2 | D/E1/E2 | D/E1/E2 | D/E1 | D/E1/E2

D - decoding
E1 - VME based encoding
E2 - Low power encoding

Thanks.

@dvrogozh
Copy link
Contributor

dvrogozh commented Dec 13, 2018 •
edited

That's most definitely a mismatch between driver's readme and driver's code:
https://github.com/intel/media-driver/blob/master/media_driver/linux/gen9/ddi/media_sku_wa_g9.cpp#L47
So, AVC VDENC should be supported on SKL (on all Gen9 SKUs according to the code). I have filed intel/media-driver#465 to correct this in driver's readme.

The reason why ffmpeg doesn't work is MFE. Looks like mediasdk miss a check for the Low Power encoding on the MFE Auto path used by ffmpeg. So, you can add -mfmode 0 to check that VDENC works fine (@fulinjie: could you, please, verify). The reason why KBL works fine is probably that MFE is not enable on this platform at all. @artem-shaporenko : can you, please, fix a conflict between MFE Auto and VDENC?

# perf stat -a -e i915/bcs0-busy/,i915/rcs0-busy/,i915/vcs0-busy/,i915/vcs1-busy/,i915/vecs0-busy/ \
/home/dvrogozh/git/github/install/bin/ffmpeg \
   -hwaccel qsv -init_hw_device qsv=hw -filter_hw_device hw \
   -v verbose -f rawvideo -video_size 720x480 -pix_fmt yuv420p \
   -i 720x480.yuv \
   -an -vf hwupload=extra_hw_frames=64,format=qsv -c:v h264_qsv \
   -low_power 1 -profile:v main -flags qscale -global_quality 20 -mfmode 0 \
   -vframes 100 -y ./out.h264

ffmpeg version N-92695-gce3f7c1 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 4.8.5 (GCC) 20150623 (Red Hat 4.8.5-28)
  configuration: --prefix=/home/dvrogozh/git/github/install --enable-libmfx
  libavutil      56. 24.101 / 56. 24.101
  libavcodec     58. 42.101 / 58. 42.101
  libavformat    58. 24.101 / 58. 24.101
  libavdevice    58.  6.101 / 58.  6.101
  libavfilter     7. 46.101 /  7. 46.101
  libswscale      5.  4.100 /  5.  4.100
  libswresample   3.  4.100 /  3.  4.100
[AVHWDeviceContext @ 0x3568c40] Opened VA display via DRM device /dev/dri/renderD128.
[AVHWDeviceContext @ 0x3568c40] libva: Open new log file /home/dvrogozh/trace/file.142020.thd-0x00006b2f for the thread 0x00006b2f        
[AVHWDeviceContext @ 0x3568c40] libva: LIBVA_TRACE is on, save log into /home/dvrogozh/trace/file.142020.thd-0x00006b2f                   
[AVHWDeviceContext @ 0x3568c40] libva: VA-API version 1.4.0                                                                               
[AVHWDeviceContext @ 0x3568c40] libva: va_getDriverName() returns 0                                                                       
[AVHWDeviceContext @ 0x3568c40] libva: User requested driver 'iHD'                                                                        
[AVHWDeviceContext @ 0x3568c40] libva: Trying to open /usr/lib64/dri/iHD_drv_video.so                                                     
[AVHWDeviceContext @ 0x3568c40] libva: Found init function __vaDriverInit_1_4                                                             
[AVHWDeviceContext @ 0x3568c40] libva: va_openDriver() returns 0                                                                          
[AVHWDeviceContext @ 0x3568c40] Initialised VAAPI connection: version 1.4                                                                 
[AVHWDeviceContext @ 0x3568c40] VAAPI driver: Intel iHD driver - 1.0.0.                                                                   
[AVHWDeviceContext @ 0x3568c40] Driver not found in known nonstandard list, using standard behaviour.                                     
[AVHWDeviceContext @ 0x3568840] Initialize MFX session: API version is 1.28, implementation version is 1.28                               
[AVHWDeviceContext @ 0x3568840] MFX compile/runtime API: 1.28/1.28                                                                        
[rawvideo @ 0x35c09c0] Estimating duration from bitrate, this may be inaccurate
Input #0, rawvideo, from '/msdk/MEDIASDK_STREAMS/YUV/iceage_720x480_491.yuv':
  Duration: 00:00:19.64, start: 0.000000, bitrate: 103680 kb/s
    Stream #0:0: Video: rawvideo, 1 reference frame (I420 / 0x30323449), yuv420p, 720x480, 103680 kb/s, 25 tbr, 25 tbn, 25 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (h264_qsv))
Press [q] to stop, [?] for help
[graph 0 input from stream 0:0 @ 0x35d1880] w:720 h:480 pixfmt:yuv420p tb:1/25 fr:25/1 sar:0/1 sws_param:flags=2
[auto_scaler_0 @ 0x35d22c0] w:iw h:ih flags:'bicubic' interl:0                                                                            
[Parsed_hwupload_0 @ 0x35c9740] auto-inserting filter 'auto_scaler_0' between the filter 'graph 0 input from stream 0:0' and the filter 'Parsed_hwupload_0'                                                                                                                         
[auto_scaler_0 @ 0x35d22c0] w:720 h:480 fmt:yuv420p sar:0/1 -> w:720 h:480 fmt:nv12 sar:0/1 flags:0x4                                     
[AVHWDeviceContext @ 0x35d1600] VAAPI driver: Intel iHD driver - 1.0.0.                                                                   
[AVHWDeviceContext @ 0x35d1600] Driver not found in known nonstandard list, using standard behaviour.                                     
[AVHWDeviceContext @ 0x3568c40] libva: Save context 0x40000000 into log file /home/dvrogozh/trace/file.142020.thd-0x00006b2f              
[AVHWDeviceContext @ 0x3568c40] libva: Open new log file /home/dvrogozh/trace/file.142020.thd-0x00006b48 for the thread 0x00006b48        
[h264_qsv @ 0x35cc6c0] Using the constant quantization parameter (CQP) ratecontrol method
[h264_qsv @ 0x35cc6c0] MFMode:0
[AVHWDeviceContext @ 0x360ae80] VAAPI driver: Intel iHD driver - 1.0.0.
[AVHWDeviceContext @ 0x360ae80] Driver not found in known nonstandard list, using standard behaviour.
[AVHWDeviceContext @ 0x3568c40] libva: Save context 0x20000000 into log file /home/dvrogozh/trace/file.142020.thd-0x00006b2f
[h264_qsv @ 0x35cc6c0] profile: main; level: 30
[h264_qsv @ 0x35cc6c0] GopPicSize: 250; GopRefDist: 1; GopOptFlag: ; IdrInterval: 0
[h264_qsv @ 0x35cc6c0] TargetUsage: 4; RateControlMethod: CQP
[h264_qsv @ 0x35cc6c0] QPI: 10; QPP: 10; QPB: 10
[h264_qsv @ 0x35cc6c0] NumSlice: 1; NumRefFrame: 2
[h264_qsv @ 0x35cc6c0] RateDistortionOpt: OFF
[h264_qsv @ 0x35cc6c0] RecoveryPointSEI: OFF IntRefType: 0; IntRefCycleSize: 0; IntRefQPDelta: 0
[h264_qsv @ 0x35cc6c0] MaxFrameSize: 259200; MaxSliceSize: 0; 
[h264_qsv @ 0x35cc6c0] BitrateLimit: ON; MBBRC: OFF; ExtBRC: OFF
[h264_qsv @ 0x35cc6c0] Trellis: auto
[h264_qsv @ 0x35cc6c0] VDENC: ON
[h264_qsv @ 0x35cc6c0] RepeatPPS: OFF; NumMbPerSlice: 0; LookAheadDS: 2x
[h264_qsv @ 0x35cc6c0] AdaptiveI: OFF; AdaptiveB: OFF; BRefType: off
[h264_qsv @ 0x35cc6c0] MinQPI: 0; MaxQPI: 0; MinQPP: 0; MaxQPP: 0; MinQPB: 0; MaxQPB: 0
[h264_qsv @ 0x35cc6c0] Entropy coding: CABAC; MaxDecFrameBuffering: 2
[h264_qsv @ 0x35cc6c0] NalHrdConformance: OFF; SingleSeiNalUnit: ON; VuiVclHrdParameters: OFF VuiNalHrdParameters: OFF
[h264_qsv @ 0x35cc6c0] FrameRateExtD: 1; FrameRateExtN: 25 
Output #0, h264, to './out.h264':
  Metadata:
    encoder         : Lavf58.24.101
    Stream #0:0: Video: h264 (h264_qsv), 1 reference frame, qsv, 720x480, q=-1--1, 1000 kb/s, 25 fps, 25 tbn, 25 tbc
    Metadata:
      encoder         : Lavc58.42.101 h264_qsv
    Side data:
      cpb: bitrate max/min/avg: 0/0/1000000 buffer size: 0 vbv_delay: -1
[AVHWDeviceContext @ 0x3568c40] libva: Open new log file /home/dvrogozh/trace/file.142020.thd-0x00006b38 for the thread 0x00006b38
No more output streams to write to, finishing.
frame=  100 fps=0.0 q=10.0 Lsize=    9525kB time=00:00:04.00 bitrate=19506.2kbits/s speed=50.5x    
video:9525kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%
Input file #0 (/msdk/MEDIASDK_STREAMS/YUV/iceage_720x480_491.yuv):
  Input stream #0:0 (video): 100 packets read (51840000 bytes); 100 frames decoded; 
  Total: 100 packets (51840000 bytes) demuxed
Output file #0 (./out.h264):
  Output stream #0:0 (video): 100 frames encoded; 100 packets muxed (9753091 bytes); 
  Total: 100 packets (9753091 bytes) muxed
[AVIOContext @ 0x35cdb40] Statistics: 0 seeks, 38 writeouts
[AVIOContext @ 0x35c1140] Statistics: 51840000 bytes read, 0 seeks

 Performance counter stats for 'system wide':

                 0 ns   i915/bcs0-busy/                                             
                 0 ns   i915/rcs0-busy/                                             
        62,664,748 ns   i915/vcs0-busy/                                             
                 0 ns   i915/vcs1-busy/                                             
        11,003,235 ns   i915/vecs0-busy/

You can also use sample_encode to check whether AVC VDENC works (in this way you avoid MFE entirely):

perf stat -a -e i915/bcs0-busy/,i915/rcs0-busy/,i915/vcs0-busy/,i915/vcs1-busy/,i915/vecs0-busy/ ./share/mfx/samples/sample_encode h264 -i 720x480.yuv -o a.264 -w 720 -h 480 -f 30 -cqp -qpi 40 -qpp 30 -qpb 30 -qsv-ff
libva info: Open new log file /home/dvrogozh/trace/file.142833.thd-0x00006e2e for the thread 0x00006e2e
libva info: LIBVA_TRACE is on, save log into /home/dvrogozh/trace/file.142833.thd-0x00006e2e
libva info: VA-API version 1.4.0
libva info: va_getDriverName() returns 0
libva info: User requested driver 'iHD'
libva info: Trying to open /usr/lib64/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_4
libva info: va_openDriver() returns 0
libva info: Save context 0x20000000 into log file /home/dvrogozh/trace/file.142833.thd-0x00006e2e
Encoding Sample Version 8.3.26.

Input file format       YUV420
Output video            AVC 
Source picture:
        Resolution      720x480
        Crop X,Y,W,H    0,0,720,480
Destination picture:
        Resolution      720x480
        Crop X,Y,W,H    0,0,720,480
Frame rate      30.00
QPI     40
QPP     30
QPB     30
Gop size        0
Ref dist        0
Ref number      0
Idr Interval    0
Target usage    balanced
Memory type     system
Media SDK impl          hw
Media SDK version       1.28

Processing started
libva info: Open new log file /home/dvrogozh/trace/file.142833.thd-0x00006e2f for the thread 0x00006e2f
Frame number: 491

Processing finished

 Performance counter stats for 'system wide':

                 0 ns   i915/bcs0-busy/                                             
                 0 ns   i915/rcs0-busy/                                             
       213,727,617 ns   i915/vcs0-busy/                                             
                 0 ns   i915/vcs1-busy/                                             
                 0 ns   i915/vecs0-busy/                                            

       0.230169915 seconds time elapsed

@fulinjie
Copy link
Contributor Author

Verified, AVC VDENC works on SKL when add -mfmode 0.
Thanks.

@artem-shaporenko
Copy link
Contributor

Will add a fix till monday.

@lizhong1008
Copy link
Contributor

Since MFE is just support for very limited HW platform(Skylake) and conflict with many other encoding parameters. So I believe would better expose the capability and limitation via MFXVideoENCODE_Query().

Unfortunately, currently MFXVideoENCODE_Query can't expose anything of MFE. @dvrogozh @artem-shaporenko , would you please consider this, instead of simply turn off MFE for some case just like ICQ.

This was referenced Dec 17, 2018
Closed
Merged
artem-shaporenko pushed a commit to artem-shaporenko/MediaSDK that referenced this issue Dec 19, 2018
Silently Disable MFE for VDEnc
6aeba18 
Fixes Intel-Media-SDK#1010
artem-shaporenko pushed a commit to artem-shaporenko/MediaSDK that referenced this issue Dec 19, 2018
Silently Disable MFE for VDEnc
0f49000 
Fixes Intel-Media-SDK#1010

Signed-off-by: Artem Shaporenko artem.shaporenko@intel.com
@artem-shaporenko artem-shaporenko mentioned this issue Dec 19, 2018
Merged
vilichev pushed a commit that referenced this issue Dec 20, 2018
@vilichev
Silently Disable MFE for VDEnc
ba4518d 
Fixes #1010
vilichev pushed a commit that referenced this issue Dec 20, 2018
@vilichev
Silently Disable MFE for VDEnc
0b589e2 
Fixes #1010

Signed-off-by: Artem Shaporenko artem.shaporenko@intel.com
maksenox added a commit that referenced this issue Dec 20, 2018
@maksenox
Revert "Silently Disable MFE for VDEnc Fixes #1010"
c036f6e 
This reverts commit ba4518d.
@dvrogozh
Copy link
Contributor

@vilichev : fyi, if commit has line like "Fixes #1010", github will automatically close the mentioned issue one patch will be merged...

fzhar pushed a commit to fzhar/MediaSDK that referenced this issue Dec 26, 2018
@fzhar
Silently Disable MFE for VDEnc
a4f4149 
Fixes Intel-Media-SDK#1010
jonrecker pushed a commit to jonrecker/MediaSDK that referenced this issue Feb 19, 2019
@jonrecker
Silently Disable MFE for VDEnc
fd7ff8b 
Fixes Intel-Media-SDK#1010
jonrecker pushed a commit to jonrecker/MediaSDK that referenced this issue Feb 19, 2019
@jonrecker
Silently Disable MFE for VDEnc
f6eaf93 
Fixes Intel-Media-SDK#1010
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@dvrogozh @fulinjie @lizhong1008 @onabiull @artem-shaporenko