0.29.1 Font rendering with transparent background has black pixels around edges of characters

[i-lost-my-bagel]

Describe the bug
On kitty 0.29.1 and later with a transparent background my font is having a slight black outline around portions of the characters. I'm using Cascadia Code as my font. With text_composition_strategy not set to legacy the issue isn't nearly as bad but it is still noticeable.

To Reproduce

1. Set background opacity to less than 100 percent

Screenshots

kitty 0.28.1

kitty 0.29.2

kitty 0.29.2 without text_composition_strategy: legacy. You can see the black pixels around the bottom right of the resolution.

Environment details

kitty 0.29.2 created by Kovid Goyal
Linux lily-legion-5-pro 6.1.39-3-lts #1 SMP PREEMPT_DYNAMIC Wed, 02 Aug 2023 10:12:55 +0000 x86_64
Arch Linux 6.1.39-3-lts (/dev/tty)

DISTRIB_ID="Arch"
DISTRIB_RELEASE="rolling"
DISTRIB_DESCRIPTION="Arch Linux"
Running under: X11
Frozen: False
Paths:
  kitty: /usr/bin/kitty
  base dir: /usr/lib/kitty
  extensions dir: /usr/lib/kitty/kitty
  system shell: /bin/bash
Loaded config files:
  /home/lily/.config/kitty/kitty.conf

Config options different from defaults:
background_opacity        0.65
bell_path                 /home/lily/Music/bell.wav
cursor_shape              2
enable_audio_bell         False
font_family               Cascadia Code
font_size                 10.0
hide_window_decorations   1
text_composition_strategy legacy
window_padding_width      FloatEdges(left=16.0, top=16.0, right=16.0, bottom=16.0)
Colors:
	color1                    #5bcdfa   
	color10                   #4d88b0   
	color11                   #6c9bb5   
	color12                   #c28692   
	color13                   #f598a6   
	color14                   #dedede   
	color2                    #51b9df   
	color3                    #48a4c6   
	color4                    #f5a9b8   
	color5                    #f5a9b8   
	color6                    #f5f5f5   
	color8                    #000000   
	color9                    #55b4ce   

Important environment variables seen by the kitty process:
	PATH                                /usr/local/bin:/usr/bin:/usr/local/sbin:/home/lily/.dotnet/tools:/var/lib/flatpak/exports/bin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl
	LANG                                en_US.UTF-8
	SHELL                               /bin/bash
	DISPLAY                             :0
	USER                                lily
	XDG_SEAT                            seat0
	XDG_SESSION_DESKTOP                 i3
	XDG_SESSION_TYPE                    x11
	XDG_CURRENT_DESKTOP                 i3
	XDG_SESSION_CLASS                   user
	XDG_VTNR                            2
	XDG_SESSION_ID                      3
	XDG_RUNTIME_DIR                     /run/user/1000
	XDG_DATA_DIRS                       /home/lily/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share/:/usr/share/

[kovidgoyal]

@m4rw3r ping

[Nohac]

Having the same issue, in some cases the text isn't even readable.
This is how I expect it to look (kitty 0.28.1):

This is how it looks in 0.29.2 (same background):

Lighter background (unreadable):

[kovidgoyal]

Incidentally while we wait for @m4rw3r to respond, you should post what compositor you are using. To me it looks like the compositor is not doing an srgb compose unlike all other compositors out there.

[Nohac]

I use picom as compositor, here's the full system spec if that helps:

 ~ inxi -GSC -xx
System:
  Host: tektite Kernel: 6.5.2-zen1-1-zen arch: x86_64 bits: 64 compiler: gcc
    v: 13.2.1 Desktop: i3 v: 4.22 dm: startx Distro: Arch Linux
CPU:
  Info: 12-core model: AMD Ryzen 9 5900X bits: 64 type: MT MCP arch: Zen 3+
    rev: 0 cache: L1: 768 KiB L2: 6 MiB L3: 64 MiB
  Flags: avx avx2 ht lm nx pae sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3 svm
Graphics:
  Device-1: AMD Navi 22 [Radeon RX 6700/6700 XT/6750 XT / 6800M/6850M XT]
    vendor: Micro-Star MSI driver: amdgpu v: kernel arch: RDNA-2 pcie:
    speed: 16 GT/s lanes: 16 ports: active: HDMI-A-1 empty: DP-1,DP-2,DP-3
    bus-ID: 2f:00.0 chip-ID: 1002:73df
  Display: server: X.org v: 1.21.1.8 with: Xwayland v: 23.2.0
    compositor: Picom v: git-b700a driver: X: loaded: amdgpu dri: radeonsi
    gpu: amdgpu display-ID: :0 screens: 1
  Screen-1: 0 s-res: 1920x1080
  Monitor-1: HDMI-A-1 mapped: HDMI-A-0 model: BenQ G2420HD res: 1920x1080
    dpi: 92 diag: 609mm (24")
  API: OpenGL v: 4.6 Mesa 23.1.7-arch1.1 renderer: AMD Radeon RX 6700 XT
    (navi22 LLVM 16.0.6 DRM 3.54 6.5.2-zen1-1-zen) direct-render: Yes

[m4rw3r]

Using commit 863adb3, I managed to replicate with the following config:

font_family SF Mono
font_size 12
background_opacity 0.9
background #555
foreground #999

If we skip transparency (background_opacity), then the pixels are normal:

The cause seems to be the conditional in the alpha-adjustment for compositor:

float adjust_alpha_for_incorrect_blending_by_compositor(float text_fg_alpha, float final_alpha) {
    // Adjust the transparent alpha-channel to account for incorrect
    // gamma-blending performed by the compositor (true for at least wlroots, picom)
    // We have a linear alpha channel apply the sRGB curve to it once again to compensate
    // for the incorrect blending in the compositor.
    // We apply the correction only if there was actual text at this pixel, so as to not make
    // background_opacity non-linear
    // See https://github.com/kovidgoyal/kitty/issues/6209 for discussion.
    const float threshold = 0.000001;
    // linear2srgb(final_alpha) if text_fg_alpha >= threshold else final_alpha
    return mix(final_alpha, linear2srgb(final_alpha), step(threshold, text_fg_alpha));
}

The threshold here is applied unevenly when pixels are close to transparent which causes issues. If we apply the linear2srgb unconditionally on the alpha the issue disappears:

Though this reintroduces the issue of background blending.

Going to see if I can find a better solution for the background gamma.

[m4rw3r]

A quick solution is to change the adjust_alpha_for_incorrect_blending_by_compositor and remove the step-function:

float adjust_alpha_for_incorrect_blending_by_compositor(float text_fg_alpha, float final_alpha) {
    return mix(final_alpha, linear2srgb(final_alpha), text_fg_alpha);
}

This way we would gradually apply the alpha over semi-transparent parts of the text, though it might still cause some issues with some color/transparency-combinations.

[kovidgoyal]

You mean remove the threshold? Doesnt that cause the issue with background blending?

[m4rw3r]

It will cause some minor issues, like the semi-transparent part of a glyph having its hue slightly off due to blending. So just plain background we have no adjustment, but semi-transparent glyph-pixels we add some adjustment proportional to the glyph transparency instead of all of the adjustment which will cause some very sudden differences in alpha.

[kovidgoyal]

The threshold is there so that linear2srgb is not applied to pixels that dont have any fg color. Without it we get back to the non-linear alpha problem from #6209. But reducing the threshold by a couple of orders of magnitude might work as well.

[m4rw3r]

That would still cause the original problem, just on fewer pixels.

Note that the code in #6218 did not include the mix(alpha, adjusted_alpha, fg_alpha) , it applied the adjustment on every pixel. In the comment above the mix is still present. Only the step-function is removed to apply the adjustment on the text-alpha smoothly.

[kovidgoyal]

Hmm I dont really have any intuition as to why removing the step function would be better, you are essentially interpolating the alpha between the original alpha and the srgbed alpha based on the amount of fg admixture. So you are proposing the linear2srgb correction be stronger for pixels with more foreground admixture.

But if I learned one thing about graphics, its that it doesnt matter if what you do makes sense, only that it looks better than before. So I am not averse to applying your suggested fix, it doesnt seem to make things worse in any of my test cases but then I have relatively poor eyesight.