Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kitty eats one page worth of scrollback on clear / Ctrl+L, then scrolling up #1113

Closed
ronjouch opened this issue Nov 1, 2018 · 33 comments
Closed

Comments

@ronjouch
Copy link
Contributor

ronjouch commented Nov 1, 2018

One GIF speaks a thousand words. Below is:

  • On top, gnome-terminal running fish
  • On bottom, kitty running the same fish with the same config

I'm running ls -la /bin then clear in both, then I scroll up using the mouse wheel. See how kitty fails to display lines after umount (it "eats" everything from uname to zsh5). Problem happens with both mouse wheel and kitty_mod+page_up.

kitty-eats-one-page-scrollback-on-ctrl-l

Am I missing anything? (Fish / Kitty misconfiguration?)

  • Fish 2.7.1-1638-gdfcf140e
  • Kitty HEAD @ 1f3550b
  • kitty.conf keys potentially relevant: scrollback_lines 90000, shell fish

Thanks for Kitty.

@egmontkob
Copy link

FYI: gnome-terminal and konsole have nonstandard behavior on "clear", see gnome-terminal/vte 506438 and konsole 384218 for details.

@ronjouch
Copy link
Contributor Author

ronjouch commented Nov 2, 2018

"FYI: gnome-terminal and konsole have nonstandard behavior on "clear", see gnome-terminal/vte 506438 and konsole 384218 for details."

@egmontkob thanks for the info! But the bugs you link to got me confused: does that mean Kitty's behavior is intentional / "xterm-compatible"? And if it is, do we agree it's undesirable? By undesirable, I mean that I use Ctrl+L (or clear, which I used in the bug report because it's visible on the GIF) to get a clean slate, e.g. to run unit tests. But that certainly does not mean I want to scrap a partial, arbitrary chunk of scrollback 😄. I do want to preserve this scrollback, e.g. to peek at the previous unit test results. Right?

A succinct sentence for what I expect from Ctrl+L is: give me a clean terminal prompt (by scrolling by a pageful, just enough to have my prompt in the top line), and don't touch the scrollback contents.

Is there a workaround (a config, or maybe a Ctrl+L- overriding map, maybe?) to have this behavior? Am I the only one to feel the current behavior is a bug?

@ronjouch ronjouch changed the title Kitty eats one page worth of scrollback on scrolling up Kitty eats one page worth of scrollback on clear / Ctrl+L and scrolling up Nov 2, 2018
@ronjouch ronjouch changed the title Kitty eats one page worth of scrollback on clear / Ctrl+L and scrolling up Kitty eats one page worth of scrollback on clear / Ctrl+L, then scrolling up Nov 2, 2018
@egmontkob
Copy link

the bugs you link to got me confused

If the situation wasn't totally confusing, terminal emulators would have probably fixed it by now so that you wouldn't have reported this bug. :)

does that mean Kitty's behavior is intentional / "xterm-compatible"? And if it is, do we agree it's undesirable?

Yes and yes, the latter in the sense that in my personal opinion, the "other" behavior makes more sense.

But one of the problems is: It shouldn't be the terminal emulators altering their behavior to please the applications running inside. It should be the other way around: apps/utilities being aware of what terminal emulators do, and build a reasonable thing on top of that.

And then if some people believe that the emulator's behavior is unreasonable and change it, or change it to please applications, that's where total messes like this situation arise.

(Disclaimer: I'm not Kitty's author. Let's see what he says.)

@kovidgoyal
Copy link
Owner

Sorry I am going to be busy for a while, so it will be some time before I can comment.

@egmontkob
Copy link

What I accidentally left out of my previous post:

The cleanest way out of this mess would probably be to get bash change its Ctrl+L handling to emit a screenful of newlines first (to effectively scroll out the contets in xterm, kitty, ...) and then change vte and konsole to the standard behavior. I have no clue if this has ever been brought up with bash's developer, and if so, what his response was.

@ronjouch
Copy link
Contributor Author

ronjouch commented Nov 2, 2018

"The cleanest way out of this mess would probably be to get bash change its Ctrl+L handling to emit a screenful of newlines first (to effectively scroll out the contets in xterm, kitty, ...) and then change vte and konsole to the standard behavior. I have no clue if this has ever been brought up with bash's developer, and if so, what his response was."

@egmontkob but then you need to convince each shell to do this change 🙂. For example, I use fish.

@egmontkob
Copy link

but then you need to convince each shell to do this change

Got any better idea? :)

@kovidgoyal
Copy link
Owner

Looks to me like kitty is working as it is supposed to. Ctrl+L sends the clear escape code and the standard is very clear about how the clear escape code should work, it must clear the contents of the current screen, not first scroll them up and then clear. I dont know why vte and konsole chose to interpret clear as scroll screen contents+clear instead of just clear.

The solution, IMO is simple. Add another escape code that means "scroll screen contents". Then have your shell map ctrl+l to generate that escape code + clear screen instead of just clear screen.

And since this solution will take a while to be implemented in terminal emulators and shells, here is a workaround you can do in kitty today to get the same behavior:

In kitty.conf

map ctrl+l all kitty +runpy "from kitty.utils import *; cols = screen_size_function()().cols; print('\n' * cols, end='\x0c')"

I haven't fully tested the solution above, so it might need a bit of adjustment, but it works by basically printing screen_height newlines and then the char to clear the screen. If for some reason it does not work, let me know and I will debug it.

@egmontkob if you are interested in implementing such an extra escape code in vte or a similar solution, feel free to continue the discussion. I will be happy to co-operate in coming up with a spec for it.

@kovidgoyal
Copy link
Owner

And just to note, the solution above is not quite optimal, since if the screen is partially filled it will insert extra blank lines into the scrollback. For the optimal solution, we need a new escape code.

@kovidgoyal
Copy link
Owner

And i have implemented the optimal solution, whch can be implemented via a mapping in kitty.conf.

If there is interest in creating a standardised escape code for this, as I discussed previously, I'll be happy to participate.

@egmontkob
Copy link

I'm not against a standardized (whatever that means) new escape sequence, but how would it solve the current pretty complex problem set?

If you convince shell authors to emit a new escape sequence (subject to feature detection which is another hell), you might as well convince them to emit a screenful of newline characters which is a much simpler and more compatible route to take.

I've taken a quick look at bash, I assume it's lib/readline/display.c _rl_clear_screen() clearing the screen which uses a variable initialized from the cl capability, so why doesn't it clear the scrollback according to terminfo definition? Does readline use its own bundled termcap (there's one in its source tree), or what? (I have no time now to further investigate.)

It's a totally messy situation we are in right now. It's not a new escape sequence that we need, but a complete roadmap to fixing this (which may or may not include a new escape sequence as one of the steps).

@kovidgoyal
Copy link
Owner

you cant emit a screenfull of newlines, since there could be less than a screenful of contents, which means you'd end up with empty lines in the scrollback. Anyway, I've already solved this issue for kitty, for people that want clear to add screen contents to the scrollback.

@egmontkob
Copy link

This is what happens now in VTE and friends. A screenful of empty lines is still better than losing contents (I'm speaking of the user experience here, not whichever technical implementation thereof).

The ideal solution, of course, would be a "smart" escape sequence that scrolls out just as much as required. (How much exactly, what would it depend on? The cursor positioin? If so then what about editing multiline commands (the cursor can be over any of its lines) and pressing Ctrl+L then? Or the onscreen contents? Then how to define "unused lines" vs. "content lines that happen to be empty"?)

The real question is however: if there's such a new escape sequence introduced, would shells really pick it up? How would they know if the emulator supports that (or at least silently ignores, that is, safe to emit)? Can you see the end of the tunnel in this direction, with a higher likelihood than in the direction of shells emitting newlines?

@maximbaz
Copy link
Contributor

maximbaz commented Nov 3, 2018

Honestly, as a user I always assumed that that's what Ctrl+L does, it puts the current screen into the scrollback. I've never noticed that part of the scrollback is gone after pressing Ctrl+L, that is until this issue was created 🙂

So for me the new config map ctrl+l combine : clear_terminal scroll active : send_text normal,application \x0c is exactly what I intuitively always expected from Ctrl+L, definitely adding that to my kitty.conf.

Maybe worth showing this exact mapping in docs @kovidgoyal, I found it in your commit message 🙂

@kovidgoyal
Copy link
Owner

@egmontkob yes the escape code would be smart, either based on the cursor position or if you really want to be fancy checking for empty lines from the bottom. See the implementation I just committed for kitty which is based on cursor position. I dont really care if shells pick up the escape code or not. As a terminal developer, providing the escape code is my job, what the rest of the ecosystem does is up to them. In designing the escape code I would simply pick one that is ignored by most existing terminals, then shell can simply send it followed by the codes to clear the screen. So they would continue to work just as tooday in terminals that dont support it and would work as expected in terminals that do. No need for detection.

@maximbaz feel free to send a PR with the addition to the docs. Personally, I've always expected the clear command to not scroll contents into the scrollback :)

@ronjouch
Copy link
Contributor Author

ronjouch commented Nov 3, 2018

Thanks Kovid for the fast fix. Added to my config file and tested in a fresh build, works as expected 👍.

"feel free to send a PR with the addition to the docs. Personally, I've always expected the clear command to not scroll contents into the scrollback :)"

@maximbaz rather than adding the command to the docs (or in addition to it), the default configuration file (with commented configuration examples) would be a natural place.

@maximbaz
Copy link
Contributor

maximbaz commented Nov 3, 2018

rather than adding the command to the docs (or in addition to it), the default configuration file (with commented configuration examples) would be a natural place.

It's the same place, docs and config file are both generated from the same python file 🙂 Created #1121.

elasticdog added a commit to elasticdog/dotfiles that referenced this issue Nov 29, 2018
This was what I had attempted to do previously, but could not get
working.

See: kovidgoyal/kitty#1113
@maxnoe
Copy link
Contributor

maxnoe commented Jun 5, 2021

Sorry to come back to this, but the workaround that is now also in the docs breaks tmux.

When using ctrl-L in tmux, somehow also the status bar gets scrolled..., without that setting, it works fine but I'd like to keep my history in the scrollbuffer when doing ctrl-l

@kovidgoyal
Copy link
Owner

Yes, obviously, there is no way for a terminal to scroll screen contents when using a multiplexer, since the terminal has no knowledge of the semantics of what is on the screen. Only the mutiplexer can know that. Part of the reason I have always considered terminal multiplexers to be a horrible hack.

@trygveaa
Copy link
Sponsor Contributor

trygveaa commented Jun 7, 2021

One thing with the current solution that's somewhat annoying is that since the screen contents is cleared by the clear_terminal command, the screen contents will blink when you press ctrl-l. This is particularly visible in vim.

I think a solution for this (which I think would also fix @maxnoe's problem) is to have a command that copies the screen contents into the scrollback so the screen contents isn't cleared, and combine that with sending '\x0c'. Would that be possible?

@trygveaa
Copy link
Sponsor Contributor

trygveaa commented Jun 7, 2021

Actually, what alacritty and gnome-terminal seems to do is to scroll the screen contents into the scrollback before clearing when erase in display is called with 2/all. Having some way to do this, instead of scrolling the screen contents into the scrollback every time ctrl+l is pressed would probably be the best solution.

Edit: I realize now this is what you're talking about above, and that you don't want to change what this escape code does. So maybe my suggestion in the previous comment would be better, since that doesn't change any escape codes.

@kovidgoyal
Copy link
Owner

kovidgoyal commented Jun 7, 2021 via email

@kovidgoyal
Copy link
Owner

kovidgoyal commented Jun 7, 2021 via email

@trygveaa
Copy link
Sponsor Contributor

trygveaa commented Jun 7, 2021

yes, but that is not what the standard specifies for this escape code.

Right, I realized this a bit later (see the edit).

I don't follow, isn't ctrl-l supposed to clear the screen?? Are you saying that it doesnt do so in vte/alacritty?

Ctrl-l doesn't always clear the screen as far as I can see. For instance when you press it in vim, vim doesn't send erase in display. However, since the clear_terminal command clears the screen, this causes the contents to blink and jump before and while it's drawn again.

When erase in display is sent, vte/alacritty does clear the screen, though they somehow manage to avoid the screen contents blinking when this is the same content that's drawn again by the shell after clearing. While if I use the clear_terminal + \x0c solution in kitty the shell contents that's drawn again in the same place blinks.

That's exactly what the current solution does. It scrolls the terminal till the cursor position, then sends \x0c As I understand it the issue with tmux is that tmux intercepts the \x0c and changes its behavior to only clear the contents of the currently focused tmux pane, not the whole terminal window. In other words it doesnt use the clear screen escape code. Thereby preserving the position of the scrollbar/other windows. There is no way for the terminal emulator to do that, since it know nothing about how tmux has divided the screen.

The current solution scrolls the terminal, which means that the screen won't have any content left. I was thinking that if we instead had a command that scrolled the screen contents into the backlog, while still keeping the screen contents present we could solve the issues with applications not expecting the screen to be cleared. I.e. it would duplicate the screen contents into the scrollback, and keep the actual screen contents the way it is.

@kovidgoyal
Copy link
Owner

kovidgoyal commented Jun 7, 2021

And for completeness, the correct solution to this problem is to have a setting in the shell that remaps ctrl+l to do what you want, aka scroll and then clear. I dont know if shells have such functionality or not but it is trivial to implement in a standalone program, which you can then bind to ctrl+l in the shells rc file. The program would simply output number of lines on the screen newlines (which is trivial to get using tput lines) and then sends the escape code for clearing the screen. Here is a five line python script that does it:

    import array, fcntl, sys, termios
    buf = array.array('H', [0, 0, 0, 0])
    fcntl.ioctl(sys.stdout, termios.TIOCGWINSZ, buf)
    print('\n' * buf[0], end='\x1b[H\x1b[2J', flush=True)

@kovidgoyal
Copy link
Owner

kovidgoyal commented Jun 7, 2021 via email

@trygveaa
Copy link
Sponsor Contributor

trygveaa commented Jun 7, 2021

Here is a five line python script that does it

Thanks! That seems to work great (after adding [ after the two \x1bs).

If anyone else want's to use this in zsh, I bound it like this:

scroll_and_clear () { path_to_script.py; zle redisplay }
zle -N scroll_and_clear
bindkey '^l' scroll_and_clear

@kovidgoyal
Copy link
Owner

It should be possible to implement it in pure zsh using tput lines. I leave that as an exercise to the reader.

@trygveaa
Copy link
Sponsor Contributor

trygveaa commented Jun 7, 2021

Right, that would of course be better. Here is a pure zsh solution:

scroll_and_clear() {
  printf '\n%.0s' {1..$(tput lines)}
  printf '\e[H\e[2J'
  zle redisplay
}
zle -N scroll_and_clear
bindkey '^l' scroll_and_clear

@maxnoe
Copy link
Contributor

maxnoe commented Jun 7, 2021

Awesome, @trygveaa works like a charm. I only changed it to:

scroll_and_clear() {
  printf '\n%.0s' {2..$(tput lines)}   # ignore one line (the prompt)
  printf '\e[H\e[2J'
  zle redisplay
}
zle -N scroll_and_clear
bindkey '^l' scroll_and_clear

This makes ctrl-l idempotent / a no-op if the screen is already clear.

@trygveaa
Copy link
Sponsor Contributor

trygveaa commented Jun 7, 2021

@maxnoe: Ah, that's a nice idea.

I figured I should use the same function zsh calls by default when you press ctrl-l, instead of sending the escape codes directly. So here is an improved version:

scroll-and-clear-screen() {
  printf '\n%.0s' {1..$LINES}
  zle clear-screen
}
zle -N scroll-and-clear-screen
bindkey '^l' scroll-and-clear-screen

And here is it with @maxnoe's change, and changed to also work if you have multiple lines in the prompt:

scroll-and-clear-screen() {
  local i=1
  while read; do ((i++)); done <<< $PS1
  printf '\n%.0s' {$i..$LINES}
  zle clear-screen
}
zle -N scroll-and-clear-screen
bindkey '^l' scroll-and-clear-screen

@kovidgoyal
Copy link
Owner

Does zsh keep $LINES up-to-date when the terminal is resized?

@trygveaa
Copy link
Sponsor Contributor

trygveaa commented Jun 7, 2021

Yes, it does for me.

yoogottamk added a commit to yoogottamk/dotfiles that referenced this issue Jun 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants