KDE Plasma: the compositor side for spatial displays on Linux

[danielcamposramos]

Hi @dfattal and the DisplayXR team,

I'm Daniel Campos Ramos, an electrical engineer in Brazil.
I build Sparky Stereo OS, a stereo 3D edition of SparkyLinux with KDE Plasma, where the compositor itself understands stereo.
I read your Linux roadmap with real interest, especially the drag phase-lock work and the note that a client cannot hook Mutter's window moves.

On Plasma the compositor can do that part.
Our KWin takes stereo surfaces that programs declare as full side by side, left eye first, and converts them for each output: 3D TVs over HDMI, and red-cyan glasses on any screen.
A spatial display would be one more output of that kind, and since KWin moves every window itself, it can place a window's origin on the lens phase before the frame is drawn, for X11 and Wayland applications alike.

What we can offer: the KWin side for DisplayXR on Plasma, tested on our bench (virtual 3D outputs in KWin and in VKMS), packaged in our edition, and offered upstream to KDE.
What would help us: knowing whether the srSDK Linux weave can be used by open-source integrators, and a way to test on a real panel.

Our desktop's stereo design in one page: https://invent.kde.org/danielcamposramos/kwin/-/blob/stereo3d/STEREO3D.md
This message was written with AI assistance for clarity.

[dfattal]

Hi Daniel, thank you. STEREO3D.md is a clear read, and you've found the right seam.

On GNOME a client can't take part in the compositor's move, so we built the compositor side as a GNOME Shell extension. It publishes window geometry, and during a drag it corrects each step onto a table of phase-clean positions that the app supplies. The interface is org.displayxr.WindowGeometry1 on the session bus; the spec is docs/specs/runtime/wayland-window-geometry.md (§7–9: drop snap, drag lattice, move sync). KWin moves every window natively, so it can do this job more cleanly than an extension can.

One principle shapes how it fits together (ADR-033): geometry crosses the boundary, phase never does. On a tracked autostereoscopic panel the interlace depends on per-unit calibration and on where the viewer is, so it stays inside the display vendor's weaver. The compositor never computes or sees the lens lattice. It receives a per-drag table of allowed displacements, built from the vendor's own snap answer, and applies it. So the weave isn't a compositor filter. What KWin can own is placement, geometry, and getting the woven surface to the screen 1:1.

What would help most:

1. A KWin implementation of the same interface the GNOME extension serves: window geometry, the drag lattice (SetDragLattice / DragLatticeNeeded), move sync and capture exclusion. DisplayXR apps would then work on Plasma unchanged. The interface identity is fixed by the spec; a KWin version would be packaged as an alternative to the GNOME one.
2. Generalizing the runtime's geometry consumer beyond its Mutter-specific assumptions (layout mode, scale) to KWin's per-output fractional scale.
3. A guarantee that a woven surface reaches scanout unscaled. Today the runtime detects resampling and falls back to 2D; a compositor guarantee would make that a contract.
4. Your virtual-output/VKMS bench as a Plasma CI lane for us.

If this works on two compositors, it becomes a candidate for a proper Wayland protocol instead of D-Bus, and KDE would be the natural place to start that.

On your questions: the DisplayXR runtime is open source (BSL-1.0), and so is the Leia plug-in (Apache-2.0). Each vendor's plug-in builds against that vendor's SDK under its terms, and weaving needs the vendor's runtime on the device. Leia's Linux runtime isn't publicly released yet; I'll let you know when it is. You don't need it to start. The plug-in's default build uses a stub weaver, and sim_display has an interlaced output mode (SIM_DISPLAY_OUTPUT=interlaced) that makes resampling and origin errors visible on an ordinary monitor. For real hardware, the simplest path is a Samsung Odyssey 3D.

A thought for later: KWin could hand its composed full side-by-side frame for a 3D output to the runtime as one full-screen session, with the vendor weaving it. That would make "DisplayXR" one of your output filters, covering any display with a vendor plug-in. Does that fit your design?

If you open an issue in displayxr-runtime for item 1, we'll review it with you.

[danielcamposramos]

Thank you, David.

The boundary is agreed: geometry crosses, phase never does.
That is already how our KWin works.
Programs declare full side by side, and each output's conversion is a filter that never reaches back into the program.

Your later thought fits exactly.
A DisplayXR output would be one more of those filters.
KWin hands the composed full side-by-side frame of that output to the runtime as one full-screen session, and the vendor weaves it.
KWin never sees the lens lattice.

Item 1: we will build a KWin publisher of org.displayxr.WindowGeometry1 with the identity unchanged: the bus name, the three interfaces and the schema.
In KWin the drag correction can sit where KWin already snaps a moving window to screen edges and other windows.
So a drag position lands on your table before it is applied, instead of being corrected after.
The issue, with our questions: #1928

Item 2: we will measure KWin's logical to device mapping at fractional scales on our bench, as you did for Mutter, and send the consumer change as a pull request with the numbers.

Item 3: KWin can tell, per window, whether it is composed 1:1: an integer device origin, no scale, no effect transforming it.
We will propose that as an additive field, so the fallback becomes a contract.

Item 4: our bench is yours, and its code is already online.
The VKMS bench: https://github.com/Sparky-OS/sparky-stereo-os/tree/main/tools/vkms-stereo-bench
Start with its README and ./bench doctor.
run-in-docker.sh runs every phase without installing anything on the host.
The step-by-step guide: https://github.com/Sparky-OS/sparky-stereo-os/blob/main/docs/test-stereo-without-a-3d-tv.md
The compositor phase needs Stereo KWin.
Its source is our KDE fork, branch stereo3d, which carries its stereo protocols in-tree: https://invent.kde.org/danielcamposramos/kwin/-/tree/stereo3d
kscreen-doctor sets the 3D modes through our libkscreen and kscreen forks, branch stereo3d-6.7, beside it.
To spare you that build, we will publish a ready root file system with Stereo KWin for the bench.
No 3D HDMI screen at hand is fine too: Stereo KWin's anaglyph output shows the same full side-by-side picture on any ordinary monitor with red and cyan glasses, chosen per output in Display Configuration.
Several monitors can each run a different 3D output at the same time.
Mirroring one screen to several outputs, each in its own 3D output, should work the same way, though we have not tested that case yet.
Update: Just tested, with our drivers, and across two GPUs.
One copied screen on two Sony 3D televisions, each in its own output:

• KDL-46HX855 on the AMD integrated GPU (amdgpu): frame packing at 30 Hz.
• KDL-46EX725 on an NVIDIA RTX 3060 (NVIDIA's open kernel modules): 2D at 60 Hz first, then top and bottom at 60 Hz.

Both showed 3D, mirrored across the two GPUs.

We will test with sim_display's interlaced output, and look at the Samsung Odyssey 3D for real hardware.
And yes: once this works on two compositors, a Wayland protocol is the right home, and KDE is where we would start it.