Inline images display in the wrong places after tab bar disappears

[koterpillar]

Describe the bug

After opening and closing a new tab (making the tab bar appear and disappear), images output with c and r parameters are appearing in an incorrect place.

To Reproduce
Steps to reproduce the behavior:

1. Fill the terminal with some data so the input line is at the bottom.
2. echo -n 1; printf "\e_Gf=100,t=f,a=T,c=10,r=1,q=1;$(echo -n "desktop/square.png" | base64)\e\\"; echo 2 (square.png)
3. Open a new tab
4. Close a new tab

Screenshots

Before tabs:

After tabs:

Environment details

kitty 0.28.1 (07bcc5ba61) created by Kovid Goyal
Linux atlassian-5570 6.2.15-300.fc38.x86_64 #1 SMP PREEMPT_DYNAMIC Thu May 11 17:37:39 UTC 2023 x86_64
S
Kernel 6.2.15-300.fc38.x86_64 on an x86_64 (/dev/tty)

Running under: Wayland
Frozen: True
Paths:
  kitty: /home/akotlyarov/.local/mybox/kovidgoyal--kitty/bin/kitty
  base dir: /home/akotlyarov/.local/mybox/kovidgoyal--kitty/lib/kitty
  extensions dir: /home/akotlyarov/.local/mybox/kovidgoyal--kitty/lib/kitty-extensions
  system shell: /bin/zsh
Loaded config files:
  /home/akotlyarov/.config/kitty/kitty.conf

Config options different from defaults:
font_family             Fira Code
font_size               10.0
macos_titlebar_color    3772833794
shell_integration       frozenset({'no-title'})
symbol_map:
	U+e0a0 - U+e0a3 → PowerlineSymbols
	U+e0b0 - U+e0b3 → PowerlineSymbols
tab_bar_edge            1
tab_bar_style           slant
update_check_interval   0.0
wayland_titlebar_color  3772833794
window_margin_width     FloatEdges(left=2.0, top=2.0, right=2.0, bottom=2.0)
Added shortcuts:
	ctrl+page_down →  next_tab
	ctrl+page_up →  previous_tab
	ctrl+t →  new_tab_with_cwd
Changed shortcuts:
	kitty_mod+0 →  change_font_size all 0
	kitty_mod+page_down →  move_tab_forward
	kitty_mod+page_up →  move_tab_backward
Colors:
	active_tab_background   #ffffff   
	background              #ffffff   
	color1                  #b70000   
	color10                 #00ad00   
	color11                 #9a9a00   
	color12                 #8787ff   
	color13                 #ff28ff   
	color14                 #00a5a5   
	color15                 #c6c6c6   
	color2                  #006900   
	color3                  #5d5d00   
	color4                  #3131ff   
	color5                  #a000a0   
	color6                  #006464   
	color7                  #949494   
	color8                  #5a5a5a   
	color9                  #ff5d5d   
	cursor                  #630063   
	cursor_text_color       #ffffff   
	foreground              #000000   
	inactive_tab_background #c0c0c0   
	tab_bar_background      #e0e0e0   
	tab_bar_margin_color    #e0e0e0   

Important environment variables seen by the kitty process:
	PATH                                /home/akotlyarov/.local/bin:/home/akotlyarov/.jenv/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin
	LANG                                en_AU.UTF-8
	EDITOR                              nvim
	SHELL                               /bin/zsh
	DISPLAY                             :0
	WAYLAND_DISPLAY                     wayland-0
	USER                                akotlyarov
	XDG_DATA_DIRS                       /home/akotlyarov/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share/:/usr/share/
	XDG_RUNTIME_DIR                     /run/user/1000
	XDG_CURRENT_DESKTOP                 GNOME
	XDG_MENU_PREFIX                     gnome-
	XDG_SESSION_CLASS                   user
	XDG_SESSION_DESKTOP                 gnome
	XDG_SESSION_TYPE                    wayland

Additional context

Reproducible with --config NONE except I had to use a white image for a black background.

[kovidgoyal]

Opening the tab bar will resize the window which will cause images to be
moved around. c and r only set the initial position of an image, they do
not persist after a resize or other changes to the screen.

[kovidgoyal]

Although, on second thoughts, generally speaking for a tab bar appearance there should be only a vertical resize, so no reflow so the images should scroll along with the text.

[koterpillar]

• Manually resizing the window vertically is similarly broken.
• Horizontal resize stretches the images with the window while resizing (which with c and r is an error IMO) but resets back once the resizing is stopped.

If any of this is by design I would really love a way to pin images to character positions without actively redrawing them on resize.

[kovidgoyal]

You definitely cannot pin images to char positions. But simple
scrolling/veritcal resizing and horizontal resizing without reflaow
involved should not change image position wrt to surrounding text.

[koterpillar]

From Terminal graphics protocol:

The graphics should also scroll with the text, automatically.

The image is rendered at the current cursor position, from the upper left corner of the current cell.

Given that the primary purpose of a terminal emulator, to me, is to display characters in cells, it seems fitting for images to be anchored to the cells they were originally drawn at. This means:

• Vertical resize and scrolling never changes image positions relative to the surrounding text.
• Image dimensions (in pixels) don't change with any resizing.

Reflow behavior doesn't matter for my selfish use case, but I would like images with e.g. c=1 and r=1 behave as normal characters would (and c=2 as double-width characters, etc.)

Regardless of the decision, best to note resizing in the protocol specification to avoid surprises :-)

[kovidgoyal]

There is no guarantee in the protocol about resizes. Images should however scroll with text, regardless of the use of c/r or anything else. All placement fields simply set the initial position of the image, they have no effect on how the image is moved/resized/reflowed subsequently.

The image not scrolling with the text on a vertical resize is the reason I re-opened this issue. I will look at it when I have a moment.

[AnonymouX47]

Hello!

Just came across this (on my way to report something else) and thought there were a few things worthy of note.

---

It seems to me as though c and r are being mistaken as related to positioning. Last I checked, they were only to specify the image size. Explicitly:

c: The number of columns to display the image over
r: The number of rows to display the image over

---

For a fact, vertical resize mostly breaks image positioning (relative to surrounding text) on Kitty.

---

• Horizontal resize stretches the images with the window while resizing (which with c and r is an error IMO) but resets back once the resizing is stopped.

Yes. It does stretch but like you said, only while resizing, which isn't necessarily a bug (or non-compliance with the spec) but the behaviour could definitely be improved upon.

The only case in which I can say horizontal resize actually breaks image positioning is when a reflow occurs.

---

Regardless of the decision, best to note resizing in the protocol specification to avoid surprises :-)

A little rewording: If it's possible to update the spec, it'll be good to specify (at least, to a reasonable extent ¹) behaviour upon resize (possibly also while resizing) and reflow, as it pertains to image positioning.

---

Finally, I should note that both Wezterm and Konsole handle image [re-]positioning as expected upon both resize and reflow.

As much as some of these behaviours do not break the spec (since it doesn't specify the expected behaviour), it'll be good to have these desired behaviours on Kitty also.

Thank you.

Footnotes

1. For instance, behaviour when the terminal window is zoomed (font resize) is quite a slippery slope since an image placement may be sized by pixels or cells (amongst other concerned factors). ↩

[AnonymouX47]

Also, I think it'll be good to reword the title of this issue.

[kovidgoyal]

Specifying how images behave under resize/reflow is a whole can of worms, I am not going to open anytime soon. I have described some of the problems in another issue. This issue exists solely until I get a chance to investigate the vertical resize bug.

[AnonymouX47]

Thanks! Please, how soon will a nightly with this fix be ready?

[kovidgoyal]

Nightlies are built once every 24 hrs

[AnonymouX47]

Sorry, should've done this sometime ago.

I've tested and confirmed that images are now re-positioned correctly, along with text

• after vertical resize
• upon text reflow after horizontal resize

The stretching is still there but that's not within the scope of this issue and like i said earlier, it's not necessarily a problem.

Thanks.