help users configure GStreamer

[totaam]

As per #3750 (comment), #3872, #3706 and #3843.
We need a tool to help users.
This tool can test various pipelines and the user should be able to save the ones that work and offer good performance.

One major difficulty is going to be runtime management of these pipelines.
Xpra normally adjusts framerate, quality and speed to the bandwidth conditions but this is much harder to do when we don't really know what the pipeline is made of without introspecting it..

[belotv]

To follow up on the referenced ticket, I confirm that Gstreamer VAAPI is failing on AMDGPU (using 5825U with Renoir/Barcelo Graphics) on any distribution I tested (Ubuntu 22.04, 22.04, 23.10 and Fedora 38). Issue occurs with both open source and "pro" version of the driver, on kernel 6.2, 6.4 and 6.5 version.

FFMPEG is working fine to convert video with VAAPI using command line so it is unfortunate it is not available for XPRA (anyway even with previous version it did not work with xpra because it was detected as "h264+mp4" rather than just "h264" so it was not being used by XPRA).

[ehfd]

@belotv This is evidently related to:
https://discussion.fedoraproject.org/t/proprietary-video-codecs-are-no-longer-hardware-accelerated-by-default-on-amd-gpus-since-fedora-37/73723
https://baronhk.wordpress.com/2022/10/05/the-facts-about-video-codec-acceleration-on-fedora-37/
https://news.itsfoss.com/fedora-drops-vaapi-codec/
https://discussion.fedoraproject.org/t/why-is-fedora-38-missing-the-essential-gpu-driver-component-to-have-hw-acceleration-enabled-by-default/87945/3

But why it doesn't work even in AMDGPU Pro drivers is unknown.

[belotv]

No, I am aware that codecs are not included in Fedora anymore, I enabled all codecs by installing from rpmfusion (and even tried compiling mesa/gstreamer manually). Codecs are properly detected by vainfo, and can be used for video encoding using ffmpeg VAAPi hardware acceleration successfully (5 times quicker than CPU encoding, so it definitely works).

Running "xpra video" causes a ring vcn_enc0 timeout and reset the GPU same as #3843.

It seems that xpra is using some gstreamer parameters that amdgpu vaapi does not like.

[totaam]

@ehfd / @belotv One of the key things that we do different in xpra is that we try to tune the video encoders dynamically to adapt to the content, client performance and to the bandwidth available.
This is more difficult with gstreamer elements than with the native codecs we normally use, but this makes a massive difference to performance and perceived quality.
It's quite possible that our default settings are wrong for some encoders, though I would argue that locking up the system is not helpful!

[belotv]

Yes not very practical to reset the GPU when an encoding fails, however I guess they had no other option as they cannot guarantee the operations after the ring has failed.

I may try to debug that over the next 2 months if I have time (but it is unlikely I will).

Unrelated but you added amf encoding for Gstreamer for Linux only, which does not make sense 😊. The gstreamer implementation of AMF is Windows only as it relies on d3d11 (https://github.com/GStreamer/gstreamer/blob/57bfbf51b2ca8a82d04070a1445ece3ee14077e0/subprojects/gst-plugins-bad/sys/amfcodec/meson.build#L10)

[ehfd]

va is the way forward for AMD GPUs in Linux.

[totaam]

libva should be good, it just doesn't work reliably enough.
The point of this ticket is to tell users quite clearly that even trying to run the pipeline might completely crash their system (as it did with mine again just a couple of days ago) and we can enable what doesn't crash.
That said, it might crash at some later point due to a library / kernel update.. but at least we will try to give them a way to enable it safely.

[ehfd]

va is the way forward for AMD GPUs in Linux.

Ah, I meant vah264enc, vaav1enc, and all other GStreamer plugins in the family.

vaapih264enc is set to be deprecated.

[totaam]

xpra configure gstreamer landing screen:

[stdedos]

So ... given

... can I just disable gstreamer?

Amazing benefits don't warrant instability 😕

[totaam]

Amazing benefits don't warrant instability 😕

No one is making you run this tool.

[stdedos]

No, ofc not 😅

Should we then ... disable gstreamer when connecting?
I'm not sure how this and

2023-11-11 22:15:40,676 Error setting up the pipeline:
2023-11-11 22:15:40,676  gst_parse_error: could not link decoder to sink (3)
2023-11-11 22:15:40,676  GStreamer pipeline for:
2023-11-11 22:15:40,676   appsrc name=src emit-signals=1 block=0 is-live=1 do-timestamp=1 stream-type=0 format=2 caps=video/x-h264,width=128,height=128,profile=(string)main,stream-format=(string)byte-stream,alignment=(string)au ! \
2023-11-11 22:15:40,677   d3d11h264dec name=decoder ! \
2023-11-11 22:15:40,677   appsink name=sink emit-signals=1 max-buffers=10 drop=false sync=false async=true qos=false caps=video/x-raw,width=128,height=128,format=(string)I420
2023-11-11 22:15:40,677 Error creating context h264 128x128 YUV420P
2023-11-11 22:15:40,677 gstreamer: h264 decoding failed: failed to setup gstreamer pipeline

that interface with each other 😕

[Vkhark]

Have you been able to fix that error with setting up the Pipeline. I am having the same issue.

[totaam]

@Vkhark + @stdedos these are harmless, we're checking if the d3d11h264dec element can be used.
Nothing to worry about.

[stdedos]

Encoding

Should I add those in the Wiki / Docs under a developer diary or something?

Seems a lot of compiled information 😅

[totaam]

Should I add those in the Wiki / Docs under a developer diary or something?

Not yet!
This is just a recap of where we are at so that I can try to come up with a plan.
Once the changes are implemented, a lot of this information will be obsolete.

[totaam]

Worth remembering that not having python3-gstreamer1 installed can manifest itself as:

TypeError: unknown type GstValueList

Sent to stderr... And missing encoder information in get_encoder_info
Not obvious, is it!

[totaam]

As of the latest commit, here's how I test desktop or monitor mode with gstreamer hardware accel (Intel only):

XPRA_STREAM_MODE=gstreamer xpra start  --bind-tcp=0.0.0.0:10000  --start=xterm :10 -d compress,gstreamer

Then attach:

xpra attach --no-mmap --encoding=stream

[ehfd]

@totaam Do you have VA-API accelerated jpeg encode in your non-GStreamer backend?

[totaam]

@ehfd no vaapi, nvjpeg instead.
Could do it via gstreamer I guess, but it's a pain to start a pipeline for what is one-shot encodes (jpeg).

[ehfd]

It could be possible to use libva to do it outside of GStreamer just like how you do it with nvjpeg.

[totaam]

libva is a generic api, which makes it a nightmare to code properly for. I've tried.

[totaam]

v6 will have this GUI disabled for now and we automatically enabled encoding=stream and gstreamer streaming mode when appropriate: 509af1a

[totaam]

I am disabling GStreamer again following server crashes I have seen at random, and other reports - ie from IRC:

Xorg-for-Xpra-43: ../../../include/privates.h:121: dixGetPrivateAddr: Assertion `key->initialized' failed.
(EE)
(EE) Backtrace:
(EE) 0: /usr/libexec/Xorg-for-Xpra-43 (xorg_backtrace+0x89) [0x56094a14d739]
(EE) 1: /usr/libexec/Xorg-for-Xpra-43 (0x560949f77000+0x1df269) [0x56094a156269]
(EE) 2: /lib64/libc.so.6 (0x7f0274800000+0x3e6f0) [0x7f027483e6f0]
(EE) 3: /lib64/libc.so.6 (0x7f0274800000+0x8b94c) [0x7f027488b94c]
(EE) 4: /lib64/libc.so.6 (raise+0x16) [0x7f027483e646]
(EE) 5: /lib64/libc.so.6 (abort+0xd3) [0x7f02748287f3]
(EE) 6: /lib64/libc.so.6 (0x7f0274800000+0x2871b) [0x7f027482871b]
(EE) 7: /lib64/libc.so.6 (0x7f0274800000+0x37386) [0x7f0274837386]
(EE) 8: /usr/libexec/Xorg-for-Xpra-43 (0x560949f77000+0x18d405) [0x56094a104405]
(EE) 9: /usr/libexec/Xorg-for-Xpra-43 (0x560949f77000+0x18d8cc) [0x56094a1048cc]
(EE) 10: /usr/libexec/Xorg-for-Xpra-43 (0x560949f77000+0x196b1b) [0x56094a10db1b]
(EE) 11: /usr/libexec/Xorg-for-Xpra-43 (0x560949f77000+0x4caf7) [0x560949fc3af7]
(EE) 12: /lib64/libc.so.6 (0x7f0274800000+0x29590) [0x7f0274829590]
(EE) 13: /lib64/libc.so.6 (__libc_start_main+0x80) [0x7f0274829640]
(EE) 14: /usr/libexec/Xorg-for-Xpra-43 (_start+0x25) [0x560949fc3ea5]
(EE)
(EE)
Fatal server error:
(EE) Caught signal 6 (Aborted). Server aborting
(EE)
(EE)

This was solved by --video-encoders=all,-gstreamer --video-decoders=all,-gstreamer.