Setting background_opacity to 0.0 does not make the area around characters transparent

[Luflosi]

I was trying to implement a transparent background blur on macOS using NSVisualEffectView. For this to work, kitty needs to have a transparent background. When setting background_opacity 0.0 in kitty.conf the text looks very bad:


This is roughly how I would expect it to look:

Am I missing something or is this a limitation of the current text rendering implementation?

[kovidgoyal]

I dont have time to look at the code right now, but I suspect that is because the text anti-aliasing does not work with background opacity of 0. I suggest instead using a low but non-zero value.

[Luflosi]

The text anti-aliasing seems to get worse as the background_opacity approaches 0. I stacked five kitty instances with one window each on top of each other. Every kitty instance only has background_opacity set in kitty.conf. The value of background_opacity of the lowest (leftmost) window is 1.0 and the background is completely black. Every other window has at least some transparency, through which the black color of the lowest window is visible.

As you may be able to see every window except the one with a value of 0.0 has at least some anti-aliasing going on although the text seems to expand in thickness as the opacity approaches 0 and an opacity value of 0.01 looks almost as bad as a value of 0:
0.01:

0.0:

[kovidgoyal]

I suspect the problem is that anti-aliasing works by mixing the foreground and background colors using a mixing function that is different than the mixing function used by the OS to blend pixels in the window buffer. I can' t think of a good way around that.

[kovidgoyal]

Yeah looking at lines 104- of cell_fragment.glsl we see that alpha blending of foreground and background colors happens using a custom pre-multiplied blend function. Probably you'd need to adjust that to compensate for whatever blend function the OS uses. Seems pretty fragile to me.

[Luflosi]

After playing around with the code a little, I found that making this change:

diff --git a/kitty/cell_fragment.glsl b/kitty/cell_fragment.glsl
index 689e2334..d7b31926 100644
--- a/kitty/cell_fragment.glsl
+++ b/kitty/cell_fragment.glsl
@@ -102,7 +102,7 @@ void main() {
     // SIMPLE
 #ifdef TRANSPARENT
     final_color = alpha_blend_premul(fg.rgb, fg.a, background * bg_alpha, bg_alpha);
-    final_color = vec4(final_color.rgb / final_color.a, final_color.a);
+    final_color = vec4(final_color.rgb, final_color.a);
 #else
     // since background alpha is 1.0, it is effectively pre-multiplied
     final_color = vec4(premul_blend(fg.rgb, fg.a, background), 1.0f);

makes anti-aliasing work again, although I don't understand the rendering process enough to know why.
The following screenshot is of kitty with a transparency of 0.0:

Although there are some weird side effects (that also occur without that change AFAICT; the following screenshots are without the modification), like selection not being properly shown, the filler area between where all the cells are and the window border being a slightly non-transparent white-ish color or black borders around everything when the window doesn't have focus:



But maybe that's for another GitHub issue.

[kovidgoyal]

does that actually work with values other than zero?

[Luflosi]

With a value of 0.5:

0.9:

0.95:

0.98:

1.0:

[kovidgoyal]

Hmm odd, ok I'll take a look at it in more detail when I have a minute.