Displaylink support

[Algram]

Hello,

I wanted to ask if displaylink is something that this project wants to support in the future. I've composed a list of resources to make it easier to work on this issue:

Why support displaylink?

We all know that the protocol itself is probably going away at some point in the future. Right now though, there are a lot of companies that have docking stations that ONLY work with displaylink. Supporting the protocol will allow wlroots users access to those adapters.

Current problems (afaik)

• According to sway this should already be supported by wlroots (Sway not working with DisplayLink devices sway#4176 (comment)). I think this is not the case, please correct me if I am wrong
• One user reported that they got it working in sway for a short periord with the displaylink package in the AUR (Wayland support DisplayLink/evdi#58 (comment), https://aur.archlinux.org/packages/displaylink/). It stopped working after an update though and since then it is dead.

How to

There is the evdi (https://github.com/DisplayLink/evdi) kernel module that already works ok. Every wayland compositor needs to implement it though. Mutter has done that already (DisplayLink/evdi#58 (comment)), so maybe looking at their implementation helps moving this forward in wlroots.

---

wlroots has migrated to gitlab.freedesktop.org. This issue has been moved to:

https://gitlab.freedesktop.org/wlroots/wlroots/-/issues/1823

[ascent12]

Yeah, this is something I eventually want to support. I even bought a displaylink device a while ago for this purpose.
It has been a low priority while some rather drastic backend changes are being made, which should eventually lead to us being able to support devices without their own rendering capabilities, like displaylink ones.

Another comment I made a while ago related to this: #1278 (comment)

[Algram]

Alright thanks for your answer! I think I already know the answer, but let me ask anyway: Is there such a thing as a quick and dirty workaround to get at least display output working?

Otherwise I will unfortunately have to switch back to bspwm since my workplace only provides displaylink stations.

Thanks a ton!

[dstoe]

I came here since I tried to get started with sway and my display-link port-replicator. Update from my today's debugging session:

• Sway aborts with Bus Error on initializing the monitor attached via a port-replicator with display-link. Logging output of sway --verbose --debug:

[ ... ]
2020-05-29 09:50:41 - [backend/drm/drm.c:695] Modesetting 'DVI-I-1' with '1920x1080@60000 mHz'
2020-05-29 09:50:41 - [backend/drm/drm.c:591] Initializing renderer on connector 'DVI-I-1'
Bus Error

• Nothing special in the kernel log
• Backtrace hints to this issue which is unrelated to wlroots. Last call in wlroots codebase is eglSwapBuffers in egl.c#L445. Line numbers in the stacktrace may be broken since I added some log statements for debugging.

#0  futex_wait_cancelable (private=<optimized out>, expected=0, futex_word=0x55c6269436f8) at ../sysdeps/nptl/futex-internal.h:183
#1  __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x55c6269436a8, cond=0x55c6269436d0) at pthread_cond_wait.c:508
#2  __pthread_cond_wait (cond=0x55c6269436d0, mutex=0x55c6269436a8) at pthread_cond_wait.c:638
#3  0x00007fb2e9b50a9b in  () at /usr/lib/x86_64-linux-gnu/dri/kms_swrast_dri.so
#4  0x00007fb2e9b5abed in  () at /usr/lib/x86_64-linux-gnu/dri/kms_swrast_dri.so
#5  0x00007fb2e9b5b747 in  () at /usr/lib/x86_64-linux-gnu/dri/kms_swrast_dri.so
#6  0x00007fb2e9558e0b in  () at /usr/lib/x86_64-linux-gnu/dri/kms_swrast_dri.so
#7  0x00007fb2e954ce59 in  () at /usr/lib/x86_64-linux-gnu/dri/kms_swrast_dri.so
#8  0x00007fb2ead01d22 in  () at /lib/x86_64-linux-gnu/libEGL_mesa.so.0
#9  0x00007fb2eacfaa9a in  () at /lib/x86_64-linux-gnu/libEGL_mesa.so.0
#10 0x00007fb2eacf3023 in  () at /lib/x86_64-linux-gnu/libEGL_mesa.so.0
#11 0x00007fb2eeaf48e7 in wlr_egl_swap_buffers (egl=0x55c6269028e8, surface=0x55c627479240, damage=0x0) at ../subprojects/wlroots/render/egl.c:446
#12 0x00007fb2eeb02f3f in drm_fb_acquire (fb=0x55c6269051d8, drm=0x55c6269027f0, mgpu=0x55c6269051b8) at ../subprojects/wlroots/backend/drm/renderer.c:422
#13 0x00007fb2eeafa4b7 in set_plane_props (atom=0x7ffd2874c010, drm=0x55c6269027f0, plane=0x55c626905190, crtc_id=32, x=0, y=0)
    at ../subprojects/wlroots/backend/drm/atomic.c:141
#14 0x00007fb2eeafaa0a in atomic_crtc_commit (drm=0x55c6269027f0, conn=0x55c6273f0510, flags=1025) at ../subprojects/wlroots/backend/drm/atomic.c:232
#15 0x00007fb2eeafcb4f in drm_crtc_commit (conn=0x55c6273f0510, flags=1) at ../subprojects/wlroots/backend/drm/drm.c:340
#16 0x00007fb2eeafcd02 in drm_crtc_page_flip (conn=0x55c6273f0510) at ../subprojects/wlroots/backend/drm/drm.c:365
#17 0x00007fb2eeafda5b in drm_connector_pageflip_renderer (conn=0x55c6273f0510) at ../subprojects/wlroots/backend/drm/drm.c:690
#18 0x00007fb2eeafdc6d in drm_connector_init_renderer (conn=0x55c6273f0510, mode=0x55c6273fcf60) at ../subprojects/wlroots/backend/drm/drm.c:734
#19 0x00007fb2eeafe076 in drm_connector_set_mode (conn=0x55c6273f0510, wlr_mode=0x55c6273fcf60) at ../subprojects/wlroots/backend/drm/drm.c:830
#20 0x00007fb2eeafd63d in drm_connector_commit (output=0x55c6273f0510) at ../subprojects/wlroots/backend/drm/drm.c:581
#21 0x00007fb2eeb4715d in wlr_output_commit (output=0x55c6273f0510) at ../subprojects/wlroots/types/wlr_output.c:585
#22 0x000055c625b91a42 in apply_output_config (oc=0x0, output=0x55c6273e22c0) at ../sway/config/output.c:419
#23 0x000055c625b752cb in handle_new_output (listener=0x55c625be30c0 <server+96>, data=0x55c6273f0510) at ../sway/desktop/output.c:951
#24 0x00007fb2eeb58660 in wlr_signal_emit_safe (signal=0x55c6260d22f8, data=0x55c6273f0510) at ../subprojects/wlroots/util/signal.c:29
#25 0x00007fb2eeb0a118 in new_output_reemit (listener=0x55c627090088, data=0x55c6273f0510) at ../subprojects/wlroots/backend/multi/backend.c:146
#26 0x00007fb2eeb58660 in wlr_signal_emit_safe (signal=0x55c626902818, data=0x55c6273f0510) at ../subprojects/wlroots/util/signal.c:29
#27 0x00007fb2eeb008ef in scan_drm_connectors (drm=0x55c6269027f0) at ../subprojects/wlroots/backend/drm/drm.c:1477
#28 0x00007fb2eeafad00 in backend_start (backend=0x55c6269027f0) at ../subprojects/wlroots/backend/drm/backend.c:26
#29 0x00007fb2eeaf9360 in wlr_backend_start (backend=0x55c6269027f0) at ../subprojects/wlroots/backend/backend.c:36
#30 0x00007fb2eeb09d19 in multi_backend_start (wlr_backend=0x55c6260d22d0) at ../subprojects/wlroots/backend/multi/backend.c:31
#31 0x00007fb2eeaf9360 in wlr_backend_start (backend=0x55c6260d22d0) at ../subprojects/wlroots/backend/backend.c:36
#32 0x000055c625b6fa7e in server_start (server=0x55c625be3060 <server>) at ../sway/server.c:218
#33 0x000055c625b6f036 in main (argc=3, argv=0x7ffd2874c7a8) at ../sway/main.c:394

• sway starts with internal and external monitor enabled if LP_NUM_THREADS=1 as suggested in the issue mentioned above: llvmpipe crashes using kms_swrast_dri.so.
• My system: fresh install of Ubuntu 20.04. Kernel 5.4.0-33-generic

[pati-ni]

I may have a Displaylink device to spare. Is this still under development? Will this be of any use?

[emersion]

It depends which generation of displaylink this is. The more recent generations have a proprietary driver.

[pati-ni]

It is an HP USB-C Universal Dock. Currently, it only works after I spin up gnome 3 with evdi 1.7 module and the proprietary Displaylink v5.3.1.34-4.

Not sure about the protocol version though. I found no way to flush any firmware to this thing which is odd.

[emersion]

Ah, so probably it's one of the newer ones. I'm not really interested in helping with that, but feel free to send pull requests.

[progandy]

There are some options to use/debug the evdi driver without displaylink. Most of the non-displaylink use cases can be covered with compositor support like create_output in sway and wf-recorder / wayvnc, though.

https://github.com/xytovl/evdi-hello
https://github.com/dgarfias/screx

[ascent12]

Even if evdi is "open source", it effectively only exists to push a proprietary driver. Any kind of special support for it should not be in wlroots.

[dstoe]

Spent some time digging into this. I'm not sure to how far this is an evdi issue. The mentioned issue "evdi_dri.so not present" is totally fine, since evdi is an output-only device with no rendering capabilities.

My observations

• First the good story: sway starts perfectly fine if the copy from the primary to the secondary drm device was disabled (writing this message from sway using the displaylink station 😍 ). Instead, simply re-interpret the buffer from the rendering-device as if it was usable on the output device, too. Hope you forgive this dirty unsafe hacking for testing 🙈 This works with my i915 rendering node which also uses the CPU-RAM. I hacked this into wlroots/backend/drm/renderer.c::drm_plane_lock_surface:

	// [...]
	struct wlr_buffer *local_buf;
	// Change the following check to always use the `else`-path even if a parent-device is used.
	if (drm->parent && 0) {
		// Perform a copy across GPUs
		local_buf = drm_surface_blit(&plane->mgpu_surf, buf);
		if (!local_buf) {
			wlr_log(WLR_ERROR, "Failed to blit buffer across GPUs");
			return false;
		}
	} else {
		local_buf = wlr_buffer_lock(buf);
	}

• The copy-path also works if the LP_NUM_THREADS was set to 1 forcing the software-rasterizer from mesa to work single-threaded. Then, however, the rendering is unusable slow.
• With a multi-threaded software-rasterizer, there seems to be a race-condition which makes the llvmpipe component of mesa crash with a bus-error (see referenced issue in my comment from May 29th).

Open Questions

• How to programmatically check that the output and rendering device operate on the same memory using the same address-space? Any advice for a clean implementation for that case from someone more experienced?
• Why does the drm_surface_blit fail? Isn't it using dma-buf which should work anyway at a slightly higher cost of performing a copy?

[emersion]

How to programmatically check that the output and rendering device operate on the same memory using the same address-space? Any advice for a clean implementation for that case from someone more experienced?

It's a little tricky. Maybe the kernel solution described here can help, but maybe not: https://lists.freedesktop.org/archives/dri-devel/2021-April/303692.html

As of today, all we're left with are heuristics.

We can discover whether a DRM device has rendering capabilities by checking if it has a render node. If it doesn't have a render node, there's a trap: maybe a completely different DRM device can be used to render to buffers allocated on the first device (split render/display SoC).

We can try directly displaying a buffer coming from a different device with KMS. We need to perform an atomic test-only commit to make sure that works. We already support this for client buffers (see drm_connector_test). We could support it for buffers coming from the primary device.

Maybe this will work, but maybe this will be sub-optimal. We can also try to do a CPU copy of the buffer. We don't quite yet have the architecture for this, I suspect we'll need something like DRM dumb buffers. Ref #2700.

Also ref #1347, which is about improving the whole multi-GPU copy situation.

Why does the drm_surface_blit fail? Isn't it using dma-buf which should work anyway at a slightly higher cost of performing a copy?

Yeah, it should work. But from what you wrote it seems like this is just a llvmpipe bug when multi-threading is enabled? And it seems like it works fine when disabling multi-threading?

[pati-ni]

Spent some time digging into this. I'm not sure to how far this is an evdi issue. The mentioned issue "evdi_dri.so not present" is totally fine, since evdi is an output-only device with no rendering capabilities.

My observations

• First the good story: sway starts perfectly fine if the copy from the primary to the secondary drm device was disabled (writing this message from sway using the displaylink station heart_eyes ). Instead, simply re-interpret the buffer from the rendering-device as if it was usable on the output device, too. Hope you forgive this dirty unsafe hacking for testing see_no_evil This works with my i915 rendering node which also uses the CPU-RAM. I hacked this into wlroots/backend/drm/renderer.c::drm_plane_lock_surface:

	// [...]
	struct wlr_buffer *local_buf;
	// Change the following check to always use the `else`-path even if a parent-device is used.
	if (drm->parent && 0) {
		// Perform a copy across GPUs
		local_buf = drm_surface_blit(&plane->mgpu_surf, buf);
		if (!local_buf) {
			wlr_log(WLR_ERROR, "Failed to blit buffer across GPUs");
			return false;
		}
	} else {
		local_buf = wlr_buffer_lock(buf);
	}

• The copy-path also works if the LP_NUM_THREADS was set to 1 forcing the software-rasterizer from mesa to work single-threaded. Then, however, the rendering is unusable slow.
• With a multi-threaded software-rasterizer, there seems to be a race-condition which makes the llvmpipe component of mesa crash with a bus-error (see referenced issue in my comment from May 29th).

Open Questions

• How to programmatically check that the output and rendering device operate on the same memory using the same address-space? Any advice for a clean implementation for that case from someone more experienced?
• Why does the drm_surface_blit fail? Isn't it using dma-buf which should work anyway at a slightly higher cost of performing a copy?

Unfortuantely, @dstoe 's hack did not work for me. At the time of writing the code modified to get displaylink up and running has been moved to backend/drm/drm.c::drm_connector_set_pending_fb

@dstoe Can you give us a bit information about the model of your displaylink docking station and/or setup you tried it with? I can say that not all usb type C docking stations are created equally. Sway works out of the box with a Lenovo thinkpad type-C docking station while it fails with a Dell one. I am suspecting that the Lenovo thinkpad one is doing displayport over the type-c usb but I am not sure on this one.

[spaceone]

This hit me, too. For google searchability here are the symtoms:

swaynag: swaynag/swaynag.c:390: swaynag_setup: Assertion `swaynag->compositor && swaynag->layer_shell && swaynag->shm' failed.

Aborted (core dumped)

[dstoe]

@dstoe Can you give us a bit information about the model of your displaylink docking station and/or setup you tried it with?

My tests were with a Fujitsu port replicator pr8.1. I didn't find any details on which displaylink chip version is included. The manual only states: "Displaylink Certified".

I can say that not all usb type C docking stations are created equally. Sway works out of the box with a Lenovo thinkpad type-C docking station while it fails with a Dell one. I am suspecting that the Lenovo thinkpad one is doing displayport over the type-c usb but I am not sure on this one.

It looks to me as if the Lenovo replicator is not using the DisplayLink technology so maybe the video-signal is directly transmitted over usb-c where the scan-out (conversion to an appropriate video signal) happens in the laptop-graphics card as usual. In the DisplayLink technology only the rendering to the frame-buffer happens on the laptop-graphics card. The frame-buffer is then transmitted to the replicator so it's the replicators duty to do the scan-out. This splitting of the rendering and scan-out to different devices is here the involved part why it doesn't work out of the box (although it should). The hack I posted only works if the laptop has a graphics-card which does the rendering to the CPU-RAM. With the hack the source frame-buffer is assumed to be readable by the output-device driver instead of performing an appropriate cross-device copy (which would e.g. be necessary if it was a discrete graphics card). It's also relying on the dangerous hope that the source frame-buffer would not be freed until the DisplayLink-driver would have transmitted it to the replicator.

All of this is my very rough knowledge of what I learned during debugging and reading on the issue. Very likely that I'm not using the correct terms or that I'm wrong.