Transparency in X11 and Wayland #2513

elParaguayo · 2021-06-06T20:36:17Z

This PR adds 32-bit colour depth for X11 which allows transparent windows (e.g. the bar). This PR aims to address some existing issues (#900, #1320, #1722, #1934 and probably some others) but will need more work before it can be merged. I'm uploading it now in case people want to test it.

It does look pretty good though:

There's a discussion about getting transparency to work here: #2508

This is based on the work of frostidaho from a number of years ago - so big kudos to them for doing most of the legwork.

I'm assuming the system tray will be an issue so I'll look at that next...

Wayland would also need to be looked at separately.

EDIT: systray may be ok:

EDIT 2: To do list

ramnes · 2021-06-07T08:18:06Z

Nice. Can you please share the to do list? Maybe you could get some help. :)

elParaguayo · 2021-06-07T08:29:04Z

Hopefully it's just a case of testing now. I thought that the systray may be a problematic because that seemed to be an issue when frostidaho originally worked on this. It may be working now by my sheer dumb luck!

m-col · 2021-06-07T09:57:49Z

The Wayland Windows are all rendered as 32bit so should work fine without much changing.

Do we need to make the depth configurable? Perhaps we could try to use 32bit but get_depth_and_visual could fallback to the root visual if a 32bit depth isn't available?

Could you elaborate on the custom colormap stuff and the issues with RecordingSurface? Just so I understand what's going on :)

m-col · 2021-06-07T10:08:05Z

Just seen #2508 (comment) which helped. What does cairocffi.OPERATOR_CLEAR do? Is it the case the RecordingSurface is incompatible with using alpha?

elParaguayo · 2021-06-07T10:11:16Z

The Wayland Windows are all rendered as 32bit so should work fine without much changing.

Well, it didn't work when I tried it but I don't know if I've got Wayland working or not (got a solid magenta background when I started it).

Do we need to make the depth configurable? Perhaps we could try to use 32bit but get_depth_and_visual could fallback to the root visual if a 32bit depth isn't available?

Currently, you can set the depth for the bar but, other than that, the changes default to requesting 32 bit windows for internal windows. It should fallback to root visual if 32 bit isn't available.

Could you elaborate on the custom colormap stuff and the issues with RecordingSurface? Just so I understand what's going on :)

Custom colormap was shamelessly copied from frostidaho's work (#1059 (comment)).

As for the issue with the RecordingSurface, this post on our discussions page should explain the issue. In short, the clear operator is not applied to the XCBSurface. My "solution" was to write the clear direct to the XCBSurface.

qtile/libqtile/backend/x11/drawer.py

Lines 189 to 194 in d0f47dd

    
           if getattr(self, "_xcb_surface", None) is not None: 
        
               ctx = cairocffi.Context(self._xcb_surface) 
        
               ctx.save() 
        
               ctx.set_operator(cairocffi.OPERATOR_CLEAR) 
        
               ctx.paint() 
        
               ctx.restore()

elParaguayo · 2021-06-07T10:16:40Z

Ah - our posts overlapped.

This page is good at explaining the operators but, in short, OPERATOR_CLEAR removes content from the area being painted. The alternative is to use OPERATOR_SOURCE which doesn't do any blending and just puts the new source onto the surface but that also didn't work. The default operator is OPERATOR_OVER which does blending and that's the behaviour that I was seeing.

I haven't tested more but it seems that RecordingSurfaces aren't handling changing operators correctly. However, if we wrote directly to an XCBSurface we wouldn't have this issue.

In case it's not obvious, there's no issue when using opaque surfaces because OPERATOR_OVER has no alpha to blend so you just get the updated content each time.

m-col · 2021-06-07T11:21:14Z

Well, it didn't work when I tried it but I don't know if I've got Wayland working or not (got a solid magenta background when I started it).

The magenta is actually currently the default "you have no background set" colour, which I was using to help with damage tracking because it's so high contrast :p I should probably change that to black like the X server.... As for Internal, they themselves are represented by a 32 bit texture to which pixels are copied but Drawer will need updating in the same way as the x11 Drawer, and if some parts overlap then they can be implemented in base.py as is currently the case.

Custom colormap was shamelessly copied from frostidaho's work (#1059 (comment)).

OK thanks, think I'm following now.

I haven't tested more but it seems that RecordingSurfaces aren't handling changing operators correctly. However, if we wrote directly to an XCBSurface we wouldn't have this issue.

IIRC RecordingSurfaces were introduced so that the underlying pixmaps can be resized (i.e. destroyed and then replaced with different one) while keeping the drawing operations around and usable across pixmaps. Maybe we could find an alternative solution for that, otherwise we'll have to figure out a way to accomplish using alpha while continuing to use RecordingSurfaces.

elParaguayo · 2021-06-07T11:29:50Z

The magenta is actually currently the default "you have no background set" colour, which I was using to help with damage tracking because it's so high contrast :p I should probably change that to black like the X server.... As for Internal, they themselves are represented by a 32 bit texture to which pixels are copied but Drawer will need updating in the same way as the x11 Drawer, and if some parts overlap then they can be implemented in base.py as is currently the case.

Good to know. I would like to inlcude Wayland in this PR if I can (will need to rename it...) and it sounds like it might be easy to do so I'll add to the list at the top.

IIRC RecordingSurfaces were introduced so that the underlying pixmaps can be resized (i.e. destroyed and then replaced with different one) while keeping the drawing operations around and usable across pixmaps. Maybe we could find an alternative solution for that, otherwise we'll have to figure out a way to accomplish using alpha while continuing to use RecordingSurfaces.

As far as I can tell, it's only going to be an issue when we need to clear the background which should be solved by my updated clear code. If that's the case, then I don't see a need to necessarily move away from using a RecordingSurface. For the rest of the time, if you're drawing semi transparent content over the background in the Drawer then the RecordingSurface should be fine as it will be using OPERATOR_OVER which is exactly the behaviour we'd want at that point.

ramnes · 2021-06-07T14:09:30Z

Hopefully it's just a case of testing now.

You probably want to take a look at @m-col's work on #1697 for this, the tests should be quite similar I think?

elParaguayo · 2021-06-07T14:12:10Z

Will do. Although I was thinking of more "hands-on" tests and seeing if things still display properly. A test script won't necessarily do that.

elParaguayo · 2021-06-08T18:33:14Z

Good news is that transparency is working in wayland (much less effort than x11).

However, I'm getting some interesting differences between x11 and wayland with same config.

More digging to do!

m-col · 2021-06-08T19:56:32Z

Very cool. What differences are you seeing?

elParaguayo · 2021-06-08T20:21:56Z

If you look at the left-hand side you can see that the layout icon is transparent and shows the bar colour in x11 but, in wayland, you see the wallpaper instead. x11 is correct here.

However, also in x11, the group box has lines underneath the inactive groups whereas wayland doesn't. wayland is correct on this.

EDIT: solved the first one, just the second to go...

EDIT 2: second one is an issue with the widget itself. If the line colour isn't set for inactive groups then it draws the line with the background colour. However, where you've got semi-opaque backgrounds, drawing this colour on top of itself makes it darker because the draw blends in overlay mode.

X11:

Wayland:

m-col · 2021-06-09T09:52:27Z

Nice, nice. re to do list entry Check multiple systray widgets, what's the worry there? I always found that having multiple systrays was buggy so I put one only on my first screen.

elParaguayo · 2021-06-09T09:58:38Z

To be very honest: that's a typo.

I was thinking about checking multiple widgets (as in I want to check whether transparency threw out any issues in individual widgets, like the GroupBox in my earlier post). I was also thinking about the system tray as that had been flagged earlier as not working with transparency. My brain just squashed those ideas together!

m-col · 2021-06-09T10:05:11Z

Lol fair. The systray is an interesting case because the pixels are provided by clients. Should transparency even apply to its icons?

elParaguayo · 2021-06-09T10:11:29Z

Systray is one of the widgets I'm least familiar with but I plan to look at it later today as it's crashing on wayland at the moment but I suspect that's a quick fix. Once it's running I can then test the transparency point.

In terms of your question, I would hope that the icons are in 32 bit windows so transparency is available to them. That seems to be the case with one of my earlier screenshots showing vlc in the tray but I haven't dug in to the code to see how it all fits together.

m-col · 2021-06-09T10:21:35Z

Systray is one of the widgets I'm least familiar with but I plan to look at it later today as it's crashing on wayland at the moment but I suspect that's a quick fix.

I could be wrong, but my understanding of the systray is that it is inherently tied to X (https://specifications.freedesktop.org/systemtray-spec/systemtray-spec-0.2.html), and therefore will never work under Wayland. For this reason I haven't looked at it at all as part of the wayland work. However, that is not to say that a system tray cannot be used under Wayland, it would just use a different protocol, e.g. https://www.freedesktop.org/wiki/Specifications/StatusNotifierItem/ which is what Swaybar uses.

elParaguayo · 2021-06-09T10:35:33Z

Good to know. So my quick fix will be to remove it from my wayland config!

Can look at making a wayland systray another time.

m-col · 2021-06-09T10:44:17Z

Good to know. So my `quick fix` will be to remove it from my wayland config!

:D yep!

…

On 210609 03:35, elParaguayo wrote: Good to know. So my `quick fix` will be to remove it from my wayland config! Can look at making a wayland systray another time. -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #2513 (comment)

elParaguayo · 2021-06-10T05:29:48Z

Interesting bug in SysTray with these changes. First icon displays fine but subsequent icons copy the drawing from the first icon and so on.

On Master, it should look like this:

But on this branch, it looks like this:

Need to dig in to the SysTray code and see how the drawers work there.

elParaguayo · 2021-06-11T08:20:06Z

I think the bug has something to do with this error in the logs:

2021-06-11 08:23:23,950 ERROR libqtile core.py:_xpoll():L329 Got an exception in poll loop
Traceback (most recent call last):
  File "/home/elp/Documents/scripts/github/qtile-dev/qtile/libqtile/backend/x11/core.py", line 282, in _xpoll
    event = self.conn.conn.poll_for_event()
  File "/home/elp/Documents/scripts/github/qtile-dev/qtile/venv/lib/python3.9/site-packages/xcffib/__init__.py", line 571, in wrapper
    return f(*args)
  File "/home/elp/Documents/scripts/github/qtile-dev/qtile/venv/lib/python3.9/site-packages/xcffib/__init__.py", line 612, in poll_for_event
    return self.hoist_event(e)
  File "/home/elp/Documents/scripts/github/qtile-dev/qtile/venv/lib/python3.9/site-packages/xcffib/__init__.py", line 686, in hoist_event
    return self._process_error(ffi.cast("xcb_generic_error_t *", e))
  File "/home/elp/Documents/scripts/github/qtile-dev/qtile/venv/lib/python3.9/site-packages/xcffib/__init__.py", line 649, in _process_error
    raise error(buf)
xcffib.xproto.MatchError

I don't get this when the bar's window depth is set to 24 bit so I'm assuming that there's some mismatch between the bar and the SysTray in terms of the depth or visual but I'm having difficulties tracing it down.

m-col · 2021-06-11T09:09:07Z

Anything interesting here? https://github.com/polybar/polybar/blob/master/src/x11/tray_manager.cpp

elParaguayo · 2021-06-11T10:05:31Z

Certainly some interesting bits in there but there's also this:

TODO: 32-bit visual
_NET_SYSTEM_TRAY_VISUAL visual_id VISUALID/32

The property should be set by the tray manager to indicate the preferred visual for icon windows.

To avoid ambiguity about the colormap to use this visual must either be the default visual for the screen 
or it must be a TrueColor visual. If this property is set to a visual with an alpha channel, the tray manager 
must use the Composite extension to composite the icon against the background using PictOpOver.

m-col · 2021-06-11T10:43:47Z

How will that affect what you're doing?

elParaguayo · 2021-06-19T08:51:08Z

Almost got the Mirror working as above. It works fine on wayland but has a weird bug on x11: The background is getting cleared but the contents is not immediately drawn. So, if you mirror a Clock the text appears after 1 second (i.e. after the first refresh) but if you mirror a static TextBox the text is never drawn.

Can't immediately see where I've gone wrong and why it's working on wayland...

@m-col would be good if you could take a look at some point.

EDIT: looks like I need to rebase onto master but will do this once we've fixed this bug.

m-col · 2021-06-19T13:06:29Z

docs/manual/config/screens.rst

+    In X11 backends, transparency will be disabled in a bar if the ``background``
+    color is fully opaque.
+
+


Maybe add a note about Systray?

I added a SysTray comment in its own docstring so it's visible on the Widgets page.

m-col · 2021-07-03T16:19:01Z

The only issue I have is that the very first draw of mirror widgets (or, at least, mirrors of the CPUGraph widget) appear as black and then subsequent draws show what they are meant to. It's a small issue, but fixing it would make this "complete" IMO. If fixing that is not straight forward, I'd rather this PR be in the next rlease as is than not at all.

elParaguayo · 2021-07-03T17:03:58Z

Great - let me take a look at that.

EDIT: Interesting. It's Wayland only...

EDIT 2: Think I've fixed it. Just need to rebase.

This PR adds 32-bit colour depth for X11 which allows transparent windows (e.g. the bar). This is based on the work of frostidaho from a number of years ago. The PR also makes some changes to the Wayland backend to enable rendering of transparent windows.

This PR fixes rendering issues in widgets related to the use of transparent backgrounds. SysTray: Only works on fully transparent backgrounds. GroupBox and TaskList: fixes issues when background is semi-transparent.

The SysTray widget will not display correctly with a solid background in a 32-bit window. We therefore set the bar's window depth to 24-bit if the user requests a fully opaque bar.

m-col · 2021-07-03T18:25:05Z

libqtile/backend/wayland/drawer.py

+        rect_changed = current_rect != self.previous_rect
+
+        # Check if draw has content (would be False for completely transparent drawer)
+        ink_changed = any(not math.isclose(0.0, i) for i in self.surface.ink_extents())


It's interesting that ink_extents doesn't take into account transparent painted things. I'm guessing you checked this before doing it this way? I'd have expected them to be handled in the same way as things without 100% transparency.

Correct. Anything with 100% transparency doesn't register with ink_extents(). However, I think that makes sense as, if it's fully transparent, there's nothing to draw.

m-col

Great! Everything looks good to me, so if there is nothing left you had in mind to change then I guess this is ready for merge 🎉

elParaguayo · 2021-07-03T19:35:59Z

It's light on tests (just checking some additional utilities) but, other than that, there's nothing more that I think needs to be added here.

elParaguayo force-pushed the transparency branch 2 times, most recently from 62308ce to d0f47dd Compare June 6, 2021 20:38

This comment has been minimized.

Sign in to view

elParaguayo changed the title ~~Add 32-bit windows for X11~~ Transparency in X11 and Wayland Jun 8, 2021

IgnisFlamme mentioned this pull request Jun 9, 2021

Bar transparency settings is causing all widgets to become transparent #2516

Closed

elParaguayo force-pushed the transparency branch 3 times, most recently from 624d934 to a169709 Compare June 19, 2021 09:12

This comment has been minimized.

Sign in to view

elParaguayo force-pushed the transparency branch 2 times, most recently from b2500ab to 0afe403 Compare June 19, 2021 13:05

m-col reviewed Jun 19, 2021

View reviewed changes

elParaguayo force-pushed the transparency branch 4 times, most recently from ddd0569 to 5007ddf Compare June 19, 2021 17:25

ramnes mentioned this pull request Jun 22, 2021

Bar Background Opacity #1722

Closed

This was referenced Jun 27, 2021

Add widget decorations #2489

Closed

Update widget keyboard grabbing and prompt widget to be backend-agnostic #2553

Merged

elParaguayo added 4 commits July 3, 2021 18:41

Transparency: widget fixes

2a6771c

This PR fixes rendering issues in widgets related to the use of transparent backgrounds. SysTray: Only works on fully transparent backgrounds. GroupBox and TaskList: fixes issues when background is semi-transparent.

Transparency: default bar colour depth

440527c

The SysTray widget will not display correctly with a solid background in a 32-bit window. We therefore set the bar's window depth to 24-bit if the user requests a fully opaque bar.

Transparency: Fix mirror widget render issue

150187e

elParaguayo force-pushed the transparency branch from 5007ddf to 150187e Compare July 3, 2021 17:41

m-col reviewed Jul 3, 2021

View reviewed changes

m-col approved these changes Jul 3, 2021

View reviewed changes

m-col merged commit b2b1ed8 into qtile:master Jul 4, 2021

This was referenced Jul 4, 2021

Incorrect behaviour of alpha channel in Bar background #900

Closed

Bar with transparency redraw issue #1059

Closed

elParaguayo deleted the transparency branch July 11, 2021 14:49

dequis mentioned this pull request Jan 22, 2022

Widget backgrounds with transparency broke #3234

Closed

		In X11 backends, transparency will be disabled in a bar if the ``background``
		color is fully opaque.

Transparency in X11 and Wayland #2513

Transparency in X11 and Wayland #2513

Conversation

elParaguayo commented Jun 6, 2021 • edited Loading

ramnes commented Jun 7, 2021

elParaguayo commented Jun 7, 2021

m-col commented Jun 7, 2021

m-col commented Jun 7, 2021

elParaguayo commented Jun 7, 2021 • edited Loading

elParaguayo commented Jun 7, 2021 • edited Loading

m-col commented Jun 7, 2021

elParaguayo commented Jun 7, 2021

This comment has been minimized.

ramnes commented Jun 7, 2021

elParaguayo commented Jun 7, 2021

elParaguayo commented Jun 8, 2021

m-col commented Jun 8, 2021

elParaguayo commented Jun 8, 2021 • edited Loading

m-col commented Jun 9, 2021

elParaguayo commented Jun 9, 2021

m-col commented Jun 9, 2021

elParaguayo commented Jun 9, 2021

m-col commented Jun 9, 2021

elParaguayo commented Jun 9, 2021

m-col commented Jun 9, 2021 via email

elParaguayo commented Jun 10, 2021

elParaguayo commented Jun 11, 2021 • edited Loading

m-col commented Jun 11, 2021

elParaguayo commented Jun 11, 2021

m-col commented Jun 11, 2021

elParaguayo commented Jun 19, 2021 • edited Loading

This comment has been minimized.

m-col Jun 19, 2021

Choose a reason for hiding this comment

elParaguayo Jun 19, 2021

Choose a reason for hiding this comment

m-col commented Jul 3, 2021

elParaguayo commented Jul 3, 2021 • edited Loading

m-col Jul 3, 2021

Choose a reason for hiding this comment

elParaguayo Jul 3, 2021

Choose a reason for hiding this comment

m-col left a comment

Choose a reason for hiding this comment

elParaguayo commented Jul 3, 2021

elParaguayo commented Jun 6, 2021 •

edited

Loading

elParaguayo commented Jun 7, 2021 •

edited

Loading

elParaguayo commented Jun 7, 2021 •

edited

Loading

elParaguayo commented Jun 8, 2021 •

edited

Loading

elParaguayo commented Jun 11, 2021 •

edited

Loading

elParaguayo commented Jun 19, 2021 •

edited

Loading

elParaguayo commented Jul 3, 2021 •

edited

Loading