Corner Radius Fix for GLX

[pijulius]

This should fix the corner radius in the glx backend to be almost exactly the same as xrender.
Not that good at math myself but I think the problem lies behind the circle already being the right size just the center of it was positioned off a bit and that's why it looked a bit "rectanglish".

Please see attached screenshots.

[yshui]

hmmm, thanks for the patch, but i want to understand this problem better.

what corner radius did you use to get these screenshots?

[pijulius]

hi @yshui , for sure! I used:
corner-radius = 10

and tested with corner radius 1 too and indeed looks fine with that too.

[yshui]

hmm, i wonder if the off by 1 error is actually here:

		if (rect_distance > 0.0f) { // <----
			c = (1.0f - clamp(rect_distance, 0.0f, 1.0f)) * rim_color;
		} else {
			float factor = clamp(rect_distance + border_width, 0.0f, 1.0f);
			c = (1.0f - factor) * c + factor * border_color;
		}

maybe it should be

		if (rect_distance > -0.5f) { // <----
			c = (0.5f - clamp(rect_distance, -0.5f, 0.5f)) * rim_color;
		} else {
			float factor = clamp(rect_distance + border_width, 0.0f, 1.0f);
			c = (1.0f - factor) * c + factor * border_color;
		}

?

[pijulius]

Hmmm, I tried that change but it makes shadows disappear, ok, not disappear but to see them I need to set shadow-opacity to 0.9 or so, so it affects the shadow in big way. The -1 I added doesn't seem to affect the shadow at all.

[yshui]

So my current thought is this: texcoord is at center of pixels (I think this is the case for fragment shaders), so rect_distance == 0 means the pixel center is right at the edge of the circle, which means it should be color at approx. 50%. But we are currently coloring it 100%

[pijulius]

FYI: have now fixed rounded corners for the mask too, please see:
Fix GLX corner radis for the mask too

Also added this new feature:
Add possibility for frame opacity to include menubar/toolbar and so on

not sure what you think about it, if you like it at all can work on formatting and other backends too but for now to showcase and for myself this was enough to achieve what I wanted.

[frebib]

Are we likely to see this, or something like it land in master at some point @yshui?

[absolutelynothelix]

@frebib, if you're looking for smoother rounded corners you can also try my squircles branch, see #609 (comment) (no eta of landing in the next branch, i guess it has some demand but there are open questions about how it should be implemented in production and missing implementation for the xrender backend).

[yshui]

Yes, I want to have this fixed in next. Just haven't gotten around to it yet.

[yshui]

@pijulius hi, can you rebase your branch on top of next? thanks.

[pijulius]

hi @yshui not sure what you mean by that, I have updated my repo to the next and all my changes are now up to date with your repo. If there is something I need to change please let me know and will get onto it asap.

Please note, there are only 3 main changes in my repo:

1. Desktop Switching Animation support using _NET_CURRENT_DESKTOP atom
2. Corner radius "fix" (not sure if it's correct but so far haven't noticed any problems at all and do look waaay better)
3. Possibility for frame opacity to include menubar/toolbar and so on
   Corner Radius Fix for GLX #1261 (comment)
   968112a

Please feel free to use any of these in the core as you wish, I don't mind if you don't pull but add the changes you find useful manually as the only thing that is important is the end result :)

[yshui]

@pijulius hi, you updated your branch by merging next into it. here what we prefer is rebasing the branch on top of next, because that gives us a cleaner git history.

[pijulius]

hi @yshui not sure what you did as I see you added the corner radius fix of your own but unfortunately there are some problems:

1. if you set corner-radius = 0 all the windows will remain fully transparent and so when you start picom you won't see anything at all on your screen
2. Shadows have disappeared, not sure what is causing this but you don't have any shadows at all now

Please note: this is all using current next brunch directly from your repo and using the picom.sample.conf from it too so no modifications at all nor any custom config settings.

Also tested my fork of your repo and that works just fine with the above tests cases (using your own picom.sample.conf and changing as noted above) so it must be something within the latest 14 commits:
pijulius/picom@next...yshui:picom:next

[pijulius]

have tracked it down to these changes:
0585e31

if you undo these changes everything seems to be working just fine.

[pijulius]

hi @yshui
Don't want to bother you with this, you really do great work with picom, love all the animations, the scripts, the stability of it and so on but re this case not sure why you are so against this quick fix.

It only adds two small numbers, nothing more, just two -1 and that's it and that makes it work in any situation without having to add any special case for 0 or non zero values and it also matches xrender corner radius too.

Your latest fix here:
6b858b8

brakes again other things, please see attached screenshots and also as you can see it doesn't make the bottom borders any different at all.

This small popup window showing the position showcases the additional border around borderless windows

Really hope you reconsider.
Thank you again for all your hard work!

[yshui]

why you are so against this quick fix.

because it's technically wrong to subtract from inner_size since that moves the center of the circle. obviously my solution is poorly thought out too, i should have put more thought into it, so i apologize.

[yshui]

@pijulius try https://github.com/yshui/picom/tree/rounded-corner-anti-alias, i think this is the best option i can come up with.

[pijulius]

@pijulius try https://github.com/yshui/picom/tree/rounded-corner-anti-alias, i think this is the best option i can come up with.

@yshui this is looking good! thank you!

ps: "since that moves the center of the circle." exactly what was trying to achieve :)