HW Encoding with Intel arc gpu is scrambled

[femiaf13]

Is there an existing issue for this?

• I have searched the existing issues

Is your issue described in the documentation?

• I have read the documentation

Is your issue present in the nightly release?

• This issue is present in the nightly release

Describe the Bug

When using Sunshine with vaapi the stream looks like the attached picture regardless of moonlight client, but here is the android client with extra stats showing.

Expected Behavior

A stream without all the artifacting

Additional Context

When using software encoding everything works fine for both remote desktop and gaming.

As a note: gaming works, hw encoding and decoding works with handbrake/ffmpeg and vlc. I have checked every way I could to ensure it wasn't a problem elsewhere first.

Host Operating System

Linux

Operating System Version

Fedora 38

Architecture

64 bit

Sunshine commit or version

0.18.4 nightly

Package

Linux - rpm

GPU Type

Intel

GPU Model

Arc A380

GPU Driver/Mesa Version

23.0.0

Capture Method (Linux Only)

KMS

Config

It's empty

Apps

No response

Relevant log output

[2023:03:14:15:12:36]: Info: Found encoder vaapi: [h264_vaapi, hevc_vaapi]
[2023:03:14:15:12:36]: Info: Adding avahi service Sunshine
[2023:03:14:15:12:36]: Info: Configuration UI available at [https://localhost:47990]
[2023:03:14:15:12:37]: Info: Avahi service Sunshine successfully established.
[2023:03:14:15:12:48]: Info: Executing [Desktop]
[2023:03:14:15:12:48]: Info: CLIENT CONNECTED
[2023:03:14:15:12:48]: Info: Screencasting with KMS
[2023:03:14:15:12:48]: Info: Found monitor for DRM screencasting
libva info: VA-API version 1.17.0
libva info: Trying to open /usr/lib64/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_17
libva info: va_openDriver() returns 0
[2023:03:14:15:12:48]: Info: SDR color coding [Rec. 709]
libva info: VA-API version 1.17.0
[2023:03:14:15:12:48]: Info: Color range: [MPEG]
libva info: Trying to open /usr/lib64/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_17
libva info: va_openDriver() returns 0
[2023:03:14:15:12:48]: Info: Setting default sink to: [sink-sunshine-stereo]
[2023:03:14:15:12:48]: Info: Found default monitor by name: sink-sunshine-stereo.monitor
[2023:03:14:15:12:57]: Info: CLIENT DISCONNECTED

[kode54]

I have this issue too, on an A770 LE 16GB. Maybe it's sending RGB to the codec and it only supports NV12?

[hashFactory]

I also have this issue with an A770 16GB on Manjaro Plasma. Capture method was KMS on X11.

To try to figure out the issue I reproduced an ffmpeg command that most closely matched Sunshine's capture method and can confirm that the output is identical to the glitched Sunshine capture if I omit the pixel conversion to nv12.

Scrambled capture command:
ffmpeg -hwaccel vaapi -vaapi_device /dev/dri/renderD129 -crtc_id 0 -plane_id 0 -device /dev/dri/card1 -framerate 60 -f kmsgrab -i - -vf 'hwmap=derive_device=vaapi' -c:v hevc_vaapi -y -r 60 -f mpegts -omit_video_pes_length 1 udp://192.168.1.48:8000

Working capture command:
ffmpeg -hwaccel vaapi -vaapi_device /dev/dri/renderD129 -crtc_id 0 -plane_id 0 -device /dev/dri/card1 -framerate 60 -f kmsgrab -i - -vf 'hwmap=derive_device=vaapi,scale_vaapi=format=nv12' -c:v hevc_vaapi -y -r 60 -f mpegts -omit_video_pes_length 1 udp://192.168.1.48:8000

Client command:
ffplay -fflags nobuffer -flags low_delay -probesize 32 -analyzeduration 1 -strict experimental -framedrop -f mpegts -vf 'setpts=0' udp://192.168.1.86:8000

Not sure where in Sunshine the pixel format gets decided though.

[cgutman]

Can you file a bug on Intel's media driver? It should be pretty simple for them to replicate since you've got an ffmpeg command that reproduces the issue.

https://github.com/intel/media-driver/issues

[kode54]

This appears to be a relevant issue and has been known for multiple years now:

intel/media-driver#401

Basically the media driver doesn't accept or output any RGB formats on gen10+.

There's a completely different issue apparently with using the media driver to postprocess and later convert RGB to NV12, as well, though it may not be related at all:

intel/media-driver#644

Also a multiple years issue, still open.