rendering: scaling one display affects rendering on the other

[Saviq]

With a couple displays, I have a setup like so:

layouts:
  default:
    cards:
    - card-id: 0
      eDP-1:
        position: [1920, 0]
        scale: 1.25
      DisplayPort-1:

So the left-most display is unscaled, at 0,0.

If I change the scale, I can clearly see it badly affecting my unscaled display, suggesting that the applications get told to scale at the other display's scale, and then get scaled back.

---

The reason for this is we are not tracking the outputs the surface is on. All we do is send an "enter" for every output when the window is created.

https://github.com/MirServer/mir/blob/bc6864bc7b2575b445355dcd63a2b3525be68458/src/server/frontend_wayland/window_wl_surface_role.cpp#L472

[Saviq]

Since the problem doesn't show on screenshots, here's a photo of the display taken at scale: 1:

And then, at scale: 10:

You can see how the text gets messed up.

[AlanGriffiths]

This is how it is supposed to work:

It is intended that scaling aware clients track the
current output of a surface, and if it is on a scaled
output it should use wl_surface.set_buffer_scale with
the scale of the output. That way the compositor can
avoid scaling the surface, and the client can supply
a higher detail image.

So, we need to check we're giving the client appropriate information to scale the buffer

[AlanGriffiths]

One clarification: does this happen with the scales set on startup? Or only if changed dynamically?

[Saviq]

One clarification: does this happen with the scales set on startup? Or only if changed dynamically?

On startup, too.

[Saviq]

So, we need to check we're giving the client appropriate information to scale the buffer

Here's a log from WAYLAND_DEBUG=1 qterminal, starting on a scale: 1 display and then moved to a scale: 4 one:

qterminal.txt

This looks suspect, then:

$ grep set_buffer_scale qterminal.txt        
[1438493.523]  -> wl_surface@23.set_buffer_scale(1)
[1438496.586]  -> wl_surface@23.set_buffer_scale(4)
[1441307.225]  -> wl_surface@33.set_buffer_scale(1)
[1441323.424]  -> wl_surface@33.set_buffer_scale(1)
[1441351.490]  -> wl_surface@33.set_buffer_scale(1)
[1445547.278]  -> wl_surface@33.set_buffer_scale(1)

[AlanGriffiths]

Looks as though we tell the surface it is on both monitors:

$ grep -e scale -e wl_surface@23.enter -e wl_surface@23.leave ~/Downloads/qterminal.txt 
[1438440.443] wl_output@14.scale(4)
[1438440.506] wl_output@16.scale(1)
[1438493.523]  -> wl_surface@23.set_buffer_scale(1)
[1438496.567] wl_surface@23.enter(wl_output@14)
[1438496.586]  -> wl_surface@23.set_buffer_scale(4)
[1438496.588] wl_surface@23.enter(wl_output@16)
[1441307.225]  -> wl_surface@33.set_buffer_scale(1)
[1441323.424]  -> wl_surface@33.set_buffer_scale(1)
[1441351.490]  -> wl_surface@33.set_buffer_scale(1)
[1445547.278]  -> wl_surface@33.set_buffer_scale(1)

[Saviq]

Looks as though we tell the surface it is on both monitors:

That'd do it.

Compared to GNOME, moving between a scale: 1 and scale: 1.25 displays:

[2654316.190] wl_surface@32.enter(wl_output@18)
[2654316.199]  -> wl_surface@32.set_buffer_scale(1)
[2654335.118]  -> wl_surface@32.set_buffer_scale(1)
[2655447.422]  -> wl_surface@32.set_buffer_scale(1)
[2655747.367]  -> wl_surface@32.set_buffer_scale(1)
[2656375.139] wl_surface@23.enter(wl_output@6)
[2656383.329] wl_surface@32.enter(wl_output@6)
[2656424.332]  -> wl_surface@23.set_buffer_scale(2)
[2657160.916]  -> wl_surface@32.set_buffer_scale(1)
[2657160.928]  -> wl_surface@32.set_buffer_scale(1)
[2658812.059] wl_surface@23.enter(wl_output@18)
[2658835.304] wl_surface@32.enter(wl_output@18)
[2658879.578]  -> wl_surface@23.set_buffer_scale(1)
[2659195.546]  -> wl_surface@32.set_buffer_scale(1)

[AlanGriffiths]

This looks suspicious:

    // TODO: send enter/leave when the surface actually enters and leaves outputs

[Saviq]

Duplicate of #342

[hbatagelo]

The images show what appear to be texture minification artifacts. The downsampling filter currently in use is a simple bilinear filter (OpenGL's GL_LINEAR), which is prone to this kind of aliasing. I reproduced the effect on a QEMU machine and took some screenshots with a zoomed-in detail for better comparison:

The problem also occurs when multiple outputs display cloned content at different scales. If the client uses the highest scale to set the buffer's resolution, the compositor downsamples the buffer when the surface is rendered on the lower-scale outputs. The issue affects server-side decorations for the same reason.

I ran some experiments using different types of texture sampling to try to improve the image quality for the case of a buffer being created for scale 10 and rendered at scale 1. Of course, it is unlikely that someone will use cloned outputs with such large differences in scale, but the artifacts are visible even if one output has a scale of 4 while another is unscaled.

The best results seem to be achieved by enabling mipmapping with trilinear filtering and a LOD bias of -1:

The rationale is that mipmapping prefilters the texture, while the negative bias forces the sampler to use the next higher LOD to avoid a loss of sharpness. Other biases also work, but -1 seems to be the sweet spot for clearer text.

I also tried supersampling (using the GL_OES_standard_derivatives extension) with different patterns (2x2, 2x2 rotated grid, quincunx) combined with biased trilinear filtering, but the text is blurrier when compared to the biased mipmapping:

In summary, if downsampling artifacts become a problem, biased trilinear filtering could be used. It is faster than supersampling and produces better results.

[Saviq]

@hbatagelo thanks for the dig! Yes,we could scale better. But we shouldn't be scaling at all in this case - Alan's already found this:

https://github.com/MirServer/mir/blob/33bb16ee57626b2c31499970b0fda6277202c742/src/server/frontend_wayland/window_wl_surface_role.cpp#L461

We need to be telling the apps which output they're on, so they render to the correct scale in the first place.

Your stuff will still be useful for when apps can't scale to the target scale for whatever reason. We may even want to use different filters depending on the factor by which we need to scale (e.g. 2x → 1.5x, or 1x → 2x etc.)