Add scrollback_fill_enlarged_window option

[elebow]

This is the beginning of a change to optionally fill the buffer with lines from the history when enlarging a window.

References #1206, #1972, #2398, #2531

This PR is unfinished, and it may take me some time to do so because I am unfamiliar with the codebase and out-of-practice with C. Any advice on the remaining work would be appreciated.

[kovidgoyal]

There are two files you will need to understand, screen.c and history.c. These maintain respectively the screen and history buffers. Currently there are functions to push a line from the top of the screen buffer to the bottom of the history buffer. You need to implement the reverse, namely pull a line from the bottom of the history buffer to the top of the screen buffer. It should not be difficult once you understand how the buffers work.

Then you need to spec how you want this to actually work. Should the number of blank lines at the bottom of the screen remain the same? Or should resizing the window mean that the cursor is moved to the last line? If only "new space" is filled does that mean this is triggerred only when the new window is taller than the old, not wider? If the window grows by n lines does that mean only at most N lines are filled?

Then you have to write tests to check that your code behaves as per the spec.

Feel free to ask if you are having difficulty anywhere. And when you have a spec, do please discuss it before spending time writing code.

[elebow]

Thank you for the advice.

Here is my thinking for the spec.

Increasing height

Lines move into the history by being "pushed up" beyond the top edge, so they should come out of history by being "pulled down" from the top edge. This supports the thought model that the screen buffer and history buffer are a single buffer, with only the lowest portion being visible.

The bottom edge of the screen is the origin, so the position of the cursor relative to the bottom edge should not change when increasing a window's height. The number of blank lines at the bottom should not be affected.

Increasing width

If rewrapping causes blank rows to become available, the space should be filled by "pulling down" from the window's top edge.

Case behaviors

• Height increased, width unchanged → pull down lines to fill new space at the top
• Height increased, width increased → rewrap, pull down
• Height increased, width decreased → rewrap, pull down if possible
• Height unchanged, width increased → rewrap, pull down if possible
• Height decreased, width increased → rewrap, pull down if possible

[kovidgoyal]

Sounds ok to me, you will also have to deal with pulling down graphics from the graphics manager. Since those can scroll up as well.

[elebow]

@kovidgoyal

I have pushed a partial implementation.

Other than the graphics manager which I have not looked at yet, I am having trouble figuring out how to add lines to the top of the screen's linebuffer in linebuf_add_line_to_top().

[kovidgoyal]

See the INDEX_UP macro, you need to do a similar thing, except instead of calling linebuf_index you call linebug_reverse_index

[elebow]

I see that HistoryBuf uses a ring buffer, and we can insert content by writing at location (self->start_of_data + self->count) % self->ynum.

But LineBuf seems to use some other data structure that I don't understand. What do line_map and scratch mean? Is there a straightforward way to write new content to a specific line number in the LineBuf?

[kovidgoyal]

On Thu, Mar 11, 2021 at 10:15:18PM -0800, Eddie Lebow wrote: I see that HistoryBuf uses a ring buffer, and we can insert content by writing at location `(self->start_of_data + self->count) % self->ynum`. But LineBuf seems to use some other data structure that I don't understand. What do `line_map` and `scratch` mean? Is there a straightforward way to write new content to a specific line number in the LineBuf?

See linebuf_insert_lines(). That allows you to insert n lines at any y. The new lines will be blank so you will need to copy in the actual line content. See copyline_to for the operation of copying a line from LineBuf to a Lineobject. You will need to write a similar function that copies a Line object to the LineBuf. line_map maps line numbers to actual locations in the underlying memory buffer. scratch is simply scratch space used internally for various operations.

[elebow]

I believe this PR is ready for review.

The tests for MacOS are failing, but I'm not sure why. None of the affected code seems like it would be platform-specific. I don't have convenient access to a real MacOS system to test further.

[elebow]

Also, I put forth an alternative name for the option, scrollback_window_anchor_bottom. Since the linebuffer and historybuffer appear to the user as a single buffer, it might be confusing to speak of "filling".

[kovidgoyal]

I think maybe the option should be named fill_expanded_window_from_scrollback it makes it pretty clear what it does.

[kovidgoyal]

Use tuple not list for default as lists are mutable

[kovidgoyal]

Use tuple not list for default

[kovidgoyal]

does not handle count == 0

[kovidgoyal]

should use index_type rather than unsigned int

[kovidgoyal]

This has unneccessary copies of the line object. Instead rewrite pop_line to take a pointer to the line object and return a bool indicating failure when count is 0. And there is a builtin line object in the LinueBuf class you can use, self->line

[kovidgoyal]

this doesnt handle the case of there being more free space because of the window becoming wider.

[elebow]

Is that not handled by the next line?

                          + (linebuf_continued_lines_count(self->main_linebuf) - linebuf_continued_lines_count(n));

[kovidgoyal]

Depends on what you mean by work. That counts continued lines both above and below the cursor, so if thre are continued lines below it will change cursor position w.r.t the bottom of the screen.

[elebow]

Would something like this address that issue?

diff --git a/kitty/line-buf.c b/kitty/line-buf.c
index 7ac76e9b..b0372436 100644
--- a/kitty/line-buf.c
+++ b/kitty/line-buf.c
@@ -327,9 +327,9 @@ is_continued(LineBuf *self, PyObject *val) {
 }
 
 unsigned int
-linebuf_continued_lines_count(LineBuf *self) {
+linebuf_continued_lines_count(LineBuf *self, index_type stop_at_line) {
     unsigned int count = 0;
-    for (unsigned int i = 0; i < self->ynum; i++)
+    for (unsigned int i = 0; i < self->ynum && i < stop_at_line; i++)
       if (self->line_attrs[i] & CONTINUED_MASK) count++;
 
     return count;
diff --git a/kitty/screen.c b/kitty/screen.c
index bb2a0b13..e94ce538 100644
--- a/kitty/screen.c
+++ b/kitty/screen.c
@@ -249,7 +249,7 @@ screen_resize(Screen *self, unsigned int lines, unsigned int columns) {
 
     if (OPT(scrollback_fill_enlarged_window)) {
       int lines_to_fill = (lines - self->main_linebuf->ynum) \
-                          + (linebuf_continued_lines_count(self->main_linebuf) - linebuf_continued_lines_count(n));
+                          + (linebuf_continued_lines_count(self->main_linebuf, y) - linebuf_continued_lines_count(n, y));
 
       while (lines_to_fill > 0) {
         if (self->historybuf->count <= 0) break;

[elebow]

Pushed changes.

[kovidgoyal]

Here is a fix for add_line_to_top()

diff --git a/kitty/line-buf.c b/kitty/line-buf.c
index 5afba116..7783c8f0 100644
--- a/kitty/line-buf.c
+++ b/kitty/line-buf.c
@@ -407,10 +407,9 @@ delete_lines(LineBuf *self, PyObject *args) {
 
 void
 linebuf_add_line_to_top(LineBuf *self, Line *line) {
-    Line dest;
-    init_line(self, &dest, self->line_map[0]);
-    copy_line(line, &dest);
-    linebuf_mark_line_dirty(self, 0);
+    init_line(self, self->line, self->line_map[0]);
+    copy_line(line, self->line);
+    self->line_attrs[0] = TEXT_DIRTY_MASK | (line->continued ? CONTINUED_MASK : 0);
 }

There is no need to use a separate Line object on stack, and also the continued flag was not being copied.

[kovidgoyal]

yes, that should be fine.

…

On Tue, Mar 16, 2021 at 10:13:07PM -0700, Eddie Lebow wrote: @elebow commented on this pull request. > @@ -236,6 +245,28 @@ screen_resize(Screen *self, unsigned int lines, unsigned int columns) { index_type x = self->cursor->x, y = self->cursor->y; LineBuf *n = realloc_lb(self->main_linebuf, lines, columns, &num_content_lines_before, &num_content_lines_after, self->historybuf, &x, &y, &self->as_ansi_buf); if (n == NULL) return false; + + + if (OPT(scrollback_fill_enlarged_window)) { + int lines_to_fill = (lines - self->main_linebuf->ynum) \ Would something like this address that issue? ```diff diff --git a/kitty/line-buf.c b/kitty/line-buf.c index 7ac76e9b..b0372436 100644 --- a/kitty/line-buf.c +++ b/kitty/line-buf.c @@ -327,9 +327,9 @@ is_continued(LineBuf *self, PyObject *val) { } unsigned int -linebuf_continued_lines_count(LineBuf *self) { +linebuf_continued_lines_count(LineBuf *self, index_type stop_at_line) { unsigned int count = 0; - for (unsigned int i = 0; i < self->ynum; i++) + for (unsigned int i = 0; i < self->ynum && i < stop_at_line; i++) if (self->line_attrs[i] & CONTINUED_MASK) count++; return count; diff --git a/kitty/screen.c b/kitty/screen.c index bb2a0b13..e94ce538 100644 --- a/kitty/screen.c +++ b/kitty/screen.c @@ -249,7 +249,7 @@ screen_resize(Screen *self, unsigned int lines, unsigned int columns) { if (OPT(scrollback_fill_enlarged_window)) { int lines_to_fill = (lines - self->main_linebuf->ynum) \ - + (linebuf_continued_lines_count(self->main_linebuf) - linebuf_continued_lines_count(n)); + + (linebuf_continued_lines_count(self->main_linebuf, y) - linebuf_continued_lines_count(n, y)); while (lines_to_fill > 0) { if (self->historybuf->count <= 0) break; ``` -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #3371 (comment)

--

_____________________________________ Dr. Kovid Goyal https://www.kovidgoyal.net https://calibre-ebook.com

_____________________________________

[elebow]

Pushed changes.

[trygveaa]

Thanks for implementing this! I experience an issue though. When I increase the height of the window, scrollback is pulled from above, but instead of the previously visible scrollback being pushed down, it's overwritten instead, so the last output from the command is gone. See the attached video.

kitty-scrollback-resize-2021-03-17_15.53.11.mp4

[kovidgoyal]

I havent looked at it in detail but see if my recent changes to do the filling after cursor position is finalized work.

[trygveaa]

The same thing still happens with the current master.

Maybe it's easier to see what's happening if I don't return to the shell. The cursor remains at the original row from the top, it isn't moved down with the lines, see video:

kitty-scrollback-resize-2021-03-17_18.17.59.mp4

Also, I see that the cursor does move a bit sideways for some reason.

[elebow]

I see that same behavior with current master (4a996c1).

git-bisect says the bad behavior was introduced with 0719861.

[elebow]

This seems to solve it.

diff --git a/kitty/screen.c b/kitty/screen.c
index 473e34b7..094cdbcf 100644
--- a/kitty/screen.c
+++ b/kitty/screen.c
@@ -330,6 +330,7 @@ screen_resize(Screen *self, unsigned int lines, unsigned int columns) {
             if (!historybuf_pop_line(self->historybuf, self->alt_linebuf->line)) break;
             INDEX_DOWN;
             linebuf_copy_line_to(self->main_linebuf, self->alt_linebuf->line, 0);
+            self->cursor->y++;
         }
     }
     return true;

[kovidgoyal]

Should be fixed by my latest commit

[trygveaa]

Thanks, now it works great!