Poor quality with all hardware-based codecs

[bool3max]

When using -c libx264rgb (software encoding) I get a pretty crisp image, regardless of other settings.

However, using -c h264_vaapi or -c hevc_vaapi, the image is always pretty blurry in certain parts (pretty much as shown in #130). I have also tested the proprietary AMDGPU amf decoder (-c h264_amf) and it produces the same results.

I've tried using various encoder options (-p b=100M, -p quality=1, -p qp=1, etc...), but none of them seem to affect the blurriness of the end result much.

Screenshot of video recording:

Actual screenshot:

The blurriness is quite noticeable.

[soreau]

Maybe see #130

[bool3max]

I did and I referenced it in my issue. The solution stated over there is to switch to a (I'm assuming) software-based, non hardware accelerated encoder, libx264rgb, and I mentioned in my issue that I have in fact tried that, and that it works well.

I'm wondering if it's possible to achieve the same result using a hardware encoder.

[soreau]

This says "The libx264rgb encoder is the same as libx264, except it accepts packed RGB pixel formats as input instead of YUV." According to this, you have to pass -preset veryslow -crf 0 but "lossless output files will likely be huge, and most non-FFmpeg based players will not be able to decode lossless. Therefore, if compatibility or file size are an issue, you should not use lossless."

[bool3max]

To be honest I have no idea what you're trying to say as I'm not too well-versed when it comes to codecs, nor video terminology in general. libx264rgb and libx264 are software encoders, as in, they make no use of the GPU, right? For me libx264rgb produces good results, but h264_vaapi and hevc_vaapi don't. How does lossless H264 come into play there? (I tried -c h264_vaapi -p crf=0 -p preset=versylow but it made no difference). Are you saying that there's no way to achieve crisp recordings with _vaapi codecs and that my only option is lossless h264?

[soreau]

Yes you are right, libx264rgb is software and I should be referring to the vaapi docs which states "hardware generally wants the nv12 layout" so apparently most hardware only supports lossy yuv420 format, though I couldn't find any further direct information on this.

[ammen99]

@bool3max You may want to try the filters functionality (#133) some people have reported that VAAPI is broken on that branch but it appears to work for me at least. So you could try and say what happens on your machine.

To skip the conversion to yuv420 (which is lossy as @soreau pointed out), you could try some filters like -F hwupload,scale_vaapi=format=nv12 but replace nv12 with another supported format. Maybe yuv444p or similar.

[bool3max]

Unfortunately, on that branch wf-recorder -f video.mp4 -c h264_vaapi -d /dev/dri/renderD128 -a -t fails with:

selected region 0 0 0 0
[Source @ 0x7f99281274c0] sws_param option is deprecated and ignored
Ignoring request to force yuv420p, because it is not supported by the codec
Using video filter: hwupload,scale_vaapi=format=nv12
[hevc_vaapi @ 0x7f9928001b40] No quality level set; using default (25).
[hevc_vaapi @ 0x7f9928001b40] Driver does not support some wanted packed headers (wanted 0xd, found 0x1).
[hevc_vaapi @ 0x7f9928001b40] colour_primaries does not match inferred value: 0, but should be 2.
[hevc_vaapi @ 0x7f9928001b40] Failed to write unit 1 (type 33).
[hevc_vaapi @ 0x7f9928001b40] Failed to write packed header.
[hevc_vaapi @ 0x7f9928001b40] Failed to write sequence header for extradata: -1094995529.
avcodec_open2 failed: Invalid data found when processing input

And if I try to use the filters you mentioned: wf-recorder -f video.mp4 -c h264_vaapi -d /dev/dri/renderD128 -a -t -F hwupload,scale_vaapi=format=yuv444p:

selected region 0 0 0 0
[Source @ 0x7f8f381315c0] sws_param option is deprecated and ignored
Ignoring request to force yuv420p, because it is not supported by the codec
Using video filter: hwupload,scale_vaapi=format=yuv444p
[Parsed_scale_vaapi_1 @ 0x7f8f38133100] Hardware does not support output format yuv444p.
[Parsed_scale_vaapi_1 @ 0x7f8f38133100] Failed to configure output pad on Parsed_scale_vaapi_1
Failed to configure graph filter:                 

[ammen99]

Unfortunately, on that branch wf-recorder -f video.mp4 -c h264_vaapi -d /dev/dri/renderD128 -a -t fails with:

selected region 0 0 0 0
[Source @ 0x7f99281274c0] sws_param option is deprecated and ignored
Ignoring request to force yuv420p, because it is not supported by the codec
Using video filter: hwupload,scale_vaapi=format=nv12
[hevc_vaapi @ 0x7f9928001b40] No quality level set; using default (25).
[hevc_vaapi @ 0x7f9928001b40] Driver does not support some wanted packed headers (wanted 0xd, found 0x1).
[hevc_vaapi @ 0x7f9928001b40] colour_primaries does not match inferred value: 0, but should be 2.
[hevc_vaapi @ 0x7f9928001b40] Failed to write unit 1 (type 33).
[hevc_vaapi @ 0x7f9928001b40] Failed to write packed header.
[hevc_vaapi @ 0x7f9928001b40] Failed to write sequence header for extradata: -1094995529.
avcodec_open2 failed: Invalid data found when processing input

Interesting, it does work for me, the exact same command. Can you retry with -l to get more debug messages?

And if I try to use the filters you mentioned: wf-recorder -f video.mp4 -c h264_vaapi -d /dev/dri/renderD128 -a -t -F hwupload,scale_vaapi=format=yuv444p:

selected region 0 0 0 0
[Source @ 0x7f8f381315c0] sws_param option is deprecated and ignored
Ignoring request to force yuv420p, because it is not supported by the codec
Using video filter: hwupload,scale_vaapi=format=yuv444p
[Parsed_scale_vaapi_1 @ 0x7f8f38133100] Hardware does not support output format yuv444p.
[Parsed_scale_vaapi_1 @ 0x7f8f38133100] Failed to configure output pad on Parsed_scale_vaapi_1
Failed to configure graph filter:                 

This is not so bad, it just means that yuv444p is not supported (but there are other formats which could still be used), I think we can experiment to find the right format after we figure out why the filters are not working at all (even with the default configuration).

[bool3max]

Can you retry with -l to get more debug messages?

wf-recorder -f video.mp4 -c h264_vaapi -d /dev/dri/renderD128 -a -t -l:

selected region 0 0 0 0
[AVHWDeviceContext @ 0x7ff92c00cbc0] libva: VA-API version 1.11.0
[AVHWDeviceContext @ 0x7ff92c00cbc0] libva: Trying to open /usr/lib/dri/radeonsi_drv_video.so
[AVHWDeviceContext @ 0x7ff92c00cbc0] libva: Found init function __vaDriverInit_1_11
[AVHWDeviceContext @ 0x7ff92c00cbc0] libva: va_openDriver() returns 0
[AVHWDeviceContext @ 0x7ff92c00cbc0] Initialised VAAPI connection: version 1.11
[AVHWDeviceContext @ 0x7ff92c00cbc0] Format 0x3231564e -> nv12.
[AVHWDeviceContext @ 0x7ff92c00cbc0] Format 0x30313050 -> p010le.
[AVHWDeviceContext @ 0x7ff92c00cbc0] Format 0x36313050 -> unknown.
[AVHWDeviceContext @ 0x7ff92c00cbc0] Format 0x30323449 -> yuv420p.
[AVHWDeviceContext @ 0x7ff92c00cbc0] Format 0x32315659 -> yuv420p.
[AVHWDeviceContext @ 0x7ff92c00cbc0] Format 0x56595559 -> unknown.
[AVHWDeviceContext @ 0x7ff92c00cbc0] Format 0x32595559 -> yuyv422.
[AVHWDeviceContext @ 0x7ff92c00cbc0] Format 0x59565955 -> uyvy422.
[AVHWDeviceContext @ 0x7ff92c00cbc0] Format 0x41524742 -> bgra.
[AVHWDeviceContext @ 0x7ff92c00cbc0] Format 0x41424752 -> rgba.
[AVHWDeviceContext @ 0x7ff92c00cbc0] Format 0x58524742 -> bgr0.
[AVHWDeviceContext @ 0x7ff92c00cbc0] Format 0x58424752 -> rgb0.
[AVHWDeviceContext @ 0x7ff92c00cbc0] VAAPI driver: Mesa Gallium driver 21.0.3 for Radeon RX 580 Series (POLARIS10, DRM 3.40.0, 5.11.16-arch1-1, LLVM 11.1.0).
[AVHWDeviceContext @ 0x7ff92c00cbc0] Driver not found in known nonstandard list, using standard behaviour.
detected 4 logical cores
[Source @ 0x7ff92c1667c0] Setting 'video_size' to value '2560x1080'
[Source @ 0x7ff92c1667c0] Setting 'pix_fmt' to value '123'
[Source @ 0x7ff92c1667c0] Setting 'time_base' to value '1/1000000'
[Source @ 0x7ff92c1667c0] Setting 'pixel_aspect' to value '1/1'
[Source @ 0x7ff92c1667c0] Setting 'sws_param' to value 'flags=fast_bilinear'
[Source @ 0x7ff92c1667c0] w:2560 h:1080 pixfmt:bgr0 tb:1/1000000 fr:0/1 sar:1/1
[Source @ 0x7ff92c1667c0] sws_param option is deprecated and ignored
Ignoring request to force yuv420p, because it is not supported by the codec
Using video filter: hwupload,scale_vaapi=format=nv12
[Parsed_scale_vaapi_1 @ 0x7ff92c111200] Setting 'format' to value 'nv12'
[AVFilterGraph @ 0x7ff92c165b40] query_formats: 4 queried, 3 merged, 0 already done, 0 delayed
[hwupload @ 0x7ff92c155640] Surface format is bgr0.
[AVHWFramesContext @ 0x7ff92c108200] Created surface 0x1.
[AVHWFramesContext @ 0x7ff92c108200] Direct mapping possible.
[AVHWFramesContext @ 0x7ff92c0fed00] Created surface 0x3.
[AVHWFramesContext @ 0x7ff92c0fed00] Direct mapping disabled: deriving image does not work: 1 (operation failed).
[AVHWFramesContext @ 0x7ff92c0fed00] Created surface 0x4.
[AVHWFramesContext @ 0x7ff92c0fed00] Created surface 0x5.
[AVHWFramesContext @ 0x7ff92c0fed00] Created surface 0x6.
[AVHWFramesContext @ 0x7ff92c0fed00] Created surface 0x7.
[AVHWFramesContext @ 0x7ff92c0fed00] Created surface 0x8.
[AVHWFramesContext @ 0x7ff92c0fed00] Created surface 0x9.
[AVHWFramesContext @ 0x7ff92c0fed00] Created surface 0xa.
[AVHWFramesContext @ 0x7ff92c0fed00] Created surface 0xb.
[AVHWFramesContext @ 0x7ff92c0fed00] Created surface 0xc.
################################################################################
+----------+
|  Source  |default--[2560x1080 1:1 bgr0]--Parsed_hwupload_0:default
| (buffer) |
+----------+

                                                                +--------------+
Parsed_scale_vaapi_1:default--[2560x1080 1:1 vaapi_vld]--default|     Sink     |
                                                                | (buffersink) |
                                                                +--------------+

                                             +-------------------+
Source:default--[2560x1080 1:1 bgr0]--default| Parsed_hwupload_0 |default--[2560x1080 1:1 vaapi_vld]--Parsed_scale_vaapi_1:default
                                             |    (hwupload)     |
                                             +-------------------+

                                                             +----------------------+
Parsed_hwupload_0:default--[2560x1080 1:1 vaapi_vld]--default| Parsed_scale_vaapi_1 |default--[2560x1080 1:1 vaapi_vld]--Sink:default
                                                             |    (scale_vaapi)     |
                                                             +----------------------+


################################################################################
[h264_vaapi @ 0x7ff92c001b40] Input surface format is nv12.
[h264_vaapi @ 0x7ff92c001b40] Using VAAPI profile VAProfileH264High (7).
[h264_vaapi @ 0x7ff92c001b40] Using VAAPI entrypoint VAEntrypointEncSlice (6).
[h264_vaapi @ 0x7ff92c001b40] Using VAAPI render target format YUV420 (0x1).
[h264_vaapi @ 0x7ff92c001b40] Driver supports RC modes CQP, CBR, VBR.
[h264_vaapi @ 0x7ff92c001b40] Driver does not support ICQ RC mode.
[h264_vaapi @ 0x7ff92c001b40] No quality level set; using default (20).
[h264_vaapi @ 0x7ff92c001b40] RC mode: CQP.
[h264_vaapi @ 0x7ff92c001b40] RC quality: 20.
[h264_vaapi @ 0x7ff92c001b40] RC framerate: 65535/1 (65535.00 fps).
[h264_vaapi @ 0x7ff92c001b40] Using intra and P-frames (supported references: 1 / 0).
[h264_vaapi @ 0x7ff92c001b40] Driver does not support some wanted packed headers (wanted 0xd, found 0x1).
[h264_vaapi @ 0x7ff92c001b40] Using nv12 as format of reconstructed frames.
[AVHWFramesContext @ 0x7ff92c123800] Created surface 0xf.
[AVHWFramesContext @ 0x7ff92c123800] Direct mapping disabled: deriving image does not work: 1 (operation failed).
[h264_vaapi @ 0x7ff92c001b40] Using fixed QP = 20 / 20 / 24 for IDR- / P- / B-frames.
[h264_vaapi @ 0x7ff92c001b40] Using level 5.
[h264_vaapi @ 0x7ff92c001b40] colour_primaries does not match inferred value: 0, but should be 2.
[h264_vaapi @ 0x7ff92c001b40] Failed to write unit 0 (type 7).
[h264_vaapi @ 0x7ff92c001b40] Failed to write packed header.
[h264_vaapi @ 0x7ff92c001b40] Failed to write sequence header for extradata: -1094995529.
avcodec_open2 failed: Invalid data found when processing input

[ammen99]

@bool3max what version of ffmpeg are you using?

[bool3max]

4.3.2

[ammen99]

Not sure how ffmpeg branching works, but this could be caused by https://trac.ffmpeg.org/ticket/8959

I am using an older version, 4.2.4. Can you try either a newer or an older version?

[ammen99]

Maybe a good test would be trying the command in the ffmpeg bug report and seeing whether it works on your computer.

[bool3max]

I am unsure about how to test that particular ffmpeg command out as it expects some sort of input and output.