Output mirroring and complex layout configurations

[ddevault]

First step is to make these two statements true:

• Outputs always render the tiled views in their own sway_container and never anyone else's
• Floating views are rendered with respect to the output layout, and can show on multiple outputs

Then, we should start to introduce two new kinds of configuration to each output config:

• Layout configuration
• Mirroring configuration

A layout configuration stores how much of the output's space is allocated to its sway_container:

output HDMI-A-1 layout 100x100@10,10

Bonus: could subdivide the output into multiple layouts

output HDMI-A-1 res 1920x1080 layout 960x1080@0,0 layout 960x1080@960,0 subdivides a 1080p output into two halves, which each get their own workspace (useful for ultrawides, for example)

If no layout is specified, the default layout is used - WxH@0,0

No layout at all can also be specified via layout none, in which case no workspace is allocated.

Separately, you can specify a mirroring configuration via mirror [source output] [at X,Y], which will draw tiling windows from the source output onto the configured output at the target position. You can specify this several times.

Some example configurations:

Normal configuration

output HDMI-A-1 res 1920x1080
output DP-1 res 1920x1080 pos 1920,0

Basic mirroring

output HDMI-A-1 res 1920x1080
output DP-1 layout none mirror HDMI-A-1

Disjoint resolution mirroring

output DP-1 res 1280x720
output HDMI-A-1 res 1920x1080 layout none mirror DP-1

Disjoint resolution mirroring with floating views

Note: this is the same config as the last example

output DP-1 res 1280x720
output HDMI-A-1 res 1920x1080 layout none mirror DP-1

Disjoint resolution mirroring with multiple layouts

output DP-1 res 1280x720
output HDMI-A-1 res 1920x1080 mirror DP-1

Disjoint resolution mirroring with multiple layouts and double mirroring

output DP-1 res 1280x720 mirror HDMI-A-1
output HDMI-A-1 res 1920x1080 mirror DP-1

Partial mirroring plus partial layout

output HDMI-A-1 res 1920x1080
output DP-1 res 1920x1080 mirror HDMI-A-1 at -1800,0 layout 1800x1080@120,0

Unanswered questions:

• z-ordering config

[acrisci]

wlr-output-layout has no support for z-ordering so we'll have to address it there.

[ddevault]

I don't think that's in scope for wlr-output-layout, we'll probably want to store that info on sway's structures

[onorua]

do you have a roadmap for this issue already?

[emersion]

We never have ETAs.

[schauveau]

This message is for people following this ticket and looking for a workaround for the current lack of screen mirroring for instance to use an overhead projector.

The trick is that the mouse and the floating windows are rendered according to their global coordinates.
Also sway does not complain when outputs have overlapping geometries so you can do something like

swaymsg output  eDP-1 pos 0 0 
swaymsg output  HDMI-A-2 pos 0 0 

At first that does not seem to do anything except that the mouse is now displayed on both screens.
What is interesting is that all floating windows are now also rendered on both screens.

I am pretty sure that during my first experiments fullscreen windows were rendered on both screens but unfortunately I cannot reproduce that anymore (another trick needed)?

That setup is not without issues. The main one is that sway is tracking the mouse coordinates to between switch between the output workspaces. Consequently, unexpected and mostly unnoticeable switches between the 2 overlapping workspaces can happen during common actions using the mouse (e.g. moving, resizing, leaving or entering floating windows).

[minus7]

I'd prefer a looser coupling between layouts and outputs. Instead of having workspaces on outputs, you have them on layouts. Layouts can have an arbitrary position and size (but probably should not intersect). Outputs can also be arbitrarily positioned. They act as viewports into the layout plane instead of directly having workspaces. To make an output into a viewport (i.e. have no workspaces itself), the output configuration line gets the 'viewport' keyword.
If an output happens to view a region where some layouts lie it'll be a viewport for them. Dunno how DPI scaling of outputs/viewports could be handled.

Example:

layout 100x100 pos 0,0
layout 100x100 pos 100,0
layout 100x100 pos 0,100
layout 100x100 pos 100,100
output DP-1 res 100x100 pos 50,50 viewport

The black rectangles are layouts, the red one is a output/viewport. In this (unrealistic) example you'd see a corner of each of the four layouts on output DP-1.

[Tormen]

// Mirrorring
+1 for a config allowing you to mirror any output ... ideally be able to switch with keyboard shortcut to mirror-mode, because my default should be:

// Stacking on top
My setup is 99.9% of all cases: Laptop screen + 1 external screen / Beamer.

@ddevault : With a notebook in-front of a desktop monitor, I would love to extend the external screen above my laptop screen. And have a generic config, that by default stacks the external screen on-top of the laptop screen.

It seems to me that with the current sway-output syntax this should not be possible, because the 0|0 coordinate is in the top left corner and thus I would need to use the height of the external screen in the sway config... but this height can vary depending on screen model... and I am looking for a way to use the "*" to match any external screen.

[Tormen]

What I am using in the mean-time is a small script cycles through the available screens and disables all others. Like this you can at least easily move everything to an output (e.g. external screen or projector). Maybe that can be an idea for Thomas as well. Tormen

…

On Thu, 25 Apr 2019 01:24:18 -0700 Thomas Kobber Panum ***@***.***> wrote: I hope mirroring comes soon, that is the missing feature that forced me back to X11 :-(

[MichaelAquilina]

Would this issue also cover handling different output configurations with different combinations of connected displays?

e.g. if I am using just my laptop screen, then that should be turned on
if I have connected my laptop to my external monitor, then the external display should be on and the laptop screen should be off.

There is currently no way to declare this (from what I can tell) in sway configuration.

[emersion]

There are no plans to support such thing in sway. You can use a helper client instead, such as kanshi.

[MichaelAquilina]

Was not aware of kanshis existance. Thanks for the heads up @emersion :)

[torwag]

Is there any progress on this? Honestly speaking, not being able to switch between dual and mirror mode is a major problem for most laptop users to the point that they can't use sway at all.

[J0nnyMak0]

Is there any progress on this? Honestly speaking, not being able to switch between dual and mirror mode is a major problem for most laptop users to the point that they can't use sway at all.

I'm not sure about not being able to use sway at all, but I agree it would be nice to at least have mirroring work. It is a bit of a bummer to have to switch back to i3 for conferences when I have to project the screen for audiences.

Would it be better to split mirroring off as a sub-feature and implement instead of waiting for this whole feature set to be done together?

[m-khvoinitsky]

Some hint for conferences and presentations: I was able to make OBS Studio (with wlrobs) to display a window with real time contents of another display (big screen/projector) which I can view on laptop display which is quite enough for presentations. I suppose there should be more lightweight solution but I haven't got enough time to dig into it.

[emersion]

Would it be better to split mirroring off as a sub-feature and implement instead of waiting for this whole feature set to be done together?

Yes. The plan was never to implement everything in this issue at once. Incremental pull requests are welcome.

[emersion]

swaywm/wlroots#3067 might come in handy.

[ShinobiX9X]

Is there any way to mirror display to, for example, a projector?
I would actually need that next week.
Is there another app that could make this work?

[Synthetica9]

@ShinobiX9X wl-mirror does the trick quite well.

[syco]

I'd prefer a looser coupling between layouts and outputs. Instead of having workspaces on outputs, you have them on layouts. Layouts can have an arbitrary position and size (but probably should not intersect). Outputs can also be arbitrarily positioned. They act as viewports into the layout plane instead of directly having workspaces. To make an output into a viewport (i.e. have no workspaces itself), the output configuration line gets the 'viewport' keyword. If an output happens to view a region where some layouts lie it'll be a viewport for them. Dunno how DPI scaling of outputs/viewports could be handled.

Example:

layout 100x100 pos 0,0
layout 100x100 pos 100,0
layout 100x100 pos 0,100
layout 100x100 pos 100,100
output DP-1 res 100x100 pos 50,50 viewport

The black rectangles are layouts, the red one is a output/viewport. In this (unrealistic) example you'd see a corner of each of the four layouts on output DP-1.

Did this got anywhere? I have a real world example where this would be useful.
Ultrawide monitor split into 3 virtual monitors. Can that be achieved in any way?

[minus7]

That's exactly what I had in mind when I wrote that. A 21:9 monitor works okay without that,, though on a 32:9 I briefly had it was really necessary. @emersion did write a patch that does in the other direction (mentioned above), but other than that nothing has happened. I'd still be interested in seeing it happen though, since I think the concept of decoupling is useful in general.

[dustymabe]

On a different note: Complex Layout configurations

Ultrawide and 4K screens are becoming more popular, and many people that had multiple screens previously are switching or thinking about switching to a single larger monitor. In such situations complex layout configurations like splitting a single monitor into two or more logical workspaces is a MUST. It's pretty much unusable otherwise.

Note: I hope this comment doesn't get categorized as off-topic. My goal is to be helpful to future people trying to search for this feature.

I've searched for this several times but this is the first time I've found an issue discussing this feature. Every time in the past I've tried to search for fake-outputs which is what the feature is called in i3. I've held off on switching to sway for a while because I'd like to use that same fake-outputs functionality (I have 42" TV as a monitor, split virtually into 3 spaces).

[dR3b]

Wouldn't it make sense to split this issue? The "mirror" feature is probably very easy to implement right?

[salkin-mada]

Wouldn't it make sense to split this issue? The "mirror" feature is probably very easy to implement or?

Just to reiterate from the thread: https://github.com/Ferdi265/wl-mirror
I think this is light years better than "classic" mirroring. So maybe the question could be: Wouldn't it make sense to move this into sway (though I am all for the add-on/package/plugin structure)

[josch]

Because other comments suggesting wl-mirror have received some down-votes and because I myself had no clue why wl-mirrors is "light years better" than "classic" mirroring or vnc based solutions, I tried it out myself and though that maybe what I found out could be useful to others that look for a mirroring solution.

I'm running sway on an ARM Cortex A53 at 1.5 GHz. This means my platform is very slow compared to modern intel machines. Browsing websites that heavily make use of JavaScript is no fun, for example. So I was hesitant to try and use wl-mirror.

But what I found was: wl-mirror does not even use 1% of my CPU even while recursively mirror-ing itself. Even when I'm playing a video at 1080p I do not observe any CPU spikes nor any perceivable lag before all the nested copies get updated. How is this possible? To quote the author from https://bugs.debian.org/1031284:

wl-mirror by default uses the wlr-export-dmabuf protocol extension to receive frames from the compositor. The compositor hands wl-mirror a dmabuf handle, which is a handle to frame data already in VRAM
on the GPU. Importing it into EGL and rendering to the screen therefore does not include copying it to the CPU and back, which is why the CPU overhead does not depend that much on screen resolution and
activity in the image.

I like the UNIX philosophy of doing one thing and doing it well. Since wl-mirror already is doing a perfect job with no lag and nearly no CPU usage, I do not see why sway would need to have that feature added as well.

[dR3b]

Because it is not user friendly. A normal Sway user knows nothing about wl-mirror!

[vassilit]

I do not see why sway would need to have that feature added as well.

Because this feature is not only about mirroring, and wl-mirror cannot split an ultrawide monitor in two workspaces.

[josch]

Because it is not user friendly. A normal Sway user knows nothing about wl-mirror!

Then sway could document or recommend it better so that it's easier to find for sway users.

Sway doesn't have to do everything. The systems we use are not monolithic but a collection of different tools doing different things. I also do not argue that the cat tool should add the ability to filter the data it prints just because I've never heard of the grep utility.

Because this feature is not only about mirroring, and wl-mirror cannot split an ultrawide monitor in two workspaces.

True. This problem (as stated above already) is that this issue is about two issues. My comment about wl-mirror only addressed the "output mirroring" part and not the "split ultrawide mirror in two workspaces" part.

[dR3b]

Then sway could document or recommend it better so that it's easier to find for sway users.

I agree. Even better: directly as wrapper command: "swaymsg output DP-1 mirror wl-mirror".

[schauveau]

I did not down-vote wl-mirror but I do not think that the main issue is the CPU usage. IMHO, this is that wl-mirror is using its own workspace. I tried to use it last year for a simple presentation and that was not an enjoyable experience. It is very easy to become confused while switching workspaces if you accidentally select the workspace running wl-mirror (by moving the mouse or with the keyboard). The last thing I want during a presentation is to start fighting against my desktop.

The problem I am describing here probably has some solutions. I did not look into it but it should be possible to change my sway config so that the mirror output and workspace cannot be (easily) switched to. A possible and simple solution could be a sway command to make a workspace unfocusable.

[dustymabe]

Given the current state and trajectory of X11 I'm wondering if there's anything an outsider (current non-contributor to this repository) can do to help this along? I'm specifically looking for the functionality described in #1666 (comment) so that I can switch from i3 to sway.

Is there a bug (feature) bounty ($$) we could start that would help?

In a previous lifetime where I was more adventurous and had copious amounts of free time I would have attempted to make the contribution myself, but that's not my reality today.

[Antiarchitect]

Due to the lack of this feature I had to buy two monitors instead of the ultra wide one and use black electrical tape to hold them together (not on the table). Maybe I should buy better onkron but anyway this feature is very desirable :)

[kryzar]

Is there a bug (feature) bounty ($$) we could start that would help?

I strongly second this. I do not have the time or skills to try to implement this feature for such a used project, but I would be happy to contribute by throwing in a few bucks. If different people contribute, we may attract developers to implement the feature, and make everybody happy!

[kryzar]

Is there a bug (feature) bounty ($$) we could start that would help?

I strongly second this. I do not have the time or skills to try to implement this feature for such a used project, but I would be happy to contribute by throwing in a few bucks. If different people contribute, we may attract developers to implement the feature, and make everybody happy!

I give a talk tomorrow which reminds me that I still won't have output mirroring. This is becoming seriously embarrassing. Can we at least get some approval from the dev. team to launch a bounty? It would be better and more right to do that with you guys on board; we are all aware that the result will be better if you are involved and have control over the implementation. But it has been one year since @dustymabe's first mention of a bounty, and three months since mine. Sway is probably the most widely used tiling WM on Wayland, and this feature (independently from its difficulty to implement) would be considered basic in any DE...

With love,
Kryzar

[emersion]

Sorry, we don't do bounties.

[kryzar]

Thanks for answering. May I ask why? What would be the way to have the feature implemented? Many want it, few can do it with quality. Also, does that mean that you would refuse a PR if it originated from a bounty?

[emersion]

Bounties come with a bunch of drawbacks and potentially toxic consequences, for instance competing contributors or frustrated contributors if a patch needs many review iterations.

[kryzar]

That's why we suggest to do that with you on board (if well organized and with clear rules, those problems are easily manageable; the consequences you mention are very general and do not apply to all situations). The best would even be to pay you with the funding from bounty; I'm sure many would be willing to do so.

However, if you really insist on not organizing that, which we respect, could you answer to the two other questions?

What would be the way to have the feature implemented?

Also, does that mean that you would refuse a PR if it originated from a bounty?

I am well aware that this is FOSS and you don't owe anyone anything. However, the WM is a big part of the desktop experience of many people, and I think those are entitled of at least knowing if the feature is on the roadmap. If you say that you do not intend to implement the feature for the next X years, that's fine, but I believe people would likely want to know, and change their config if necessary.

Thank you very much for your time and consideration (and Sway!).

[dustymabe]

The best would even be to pay you with the funding from bounty

Indeed. My use of the term "bug bounty" may have not been clear. I definitely don't want users/contributors competing for the bounty. More clear would be either paying @emersion (or some other known contributor) directly for the work somehow. Use of a bug bounty like platform could enable us to transfer those funds.