Skip to content

Commit 9ed2a76

Browse files
committed
Detect cap rotation via first-row hash to keep scrollback fresh
Follow-up correctness fix on top of the scrollback-in-buffer commit, in a separate commit so it can be reverted independently. When libghostty's scrollback hits its byte cap and starts evicting the oldest rows in lockstep with new ones being pushed, the row at scrollback index 0 changes underneath us. The normal delta-detection in redraw() tracks `total_rows` deltas, but those don't capture content rotation — and worse, the existing trim path (delta < 0) removes our top rows under the assumption they match the rows libghostty just evicted, which isn't true after rotation has shifted the content. User-visible symptom: after sustained streaming past the cap, isearch / consult-line over the buffer's scrollback returns rows that no longer exist in libghostty. Fix: - Add `wrote_since_redraw: bool` and `first_scrollback_row_hash: u64` to the Terminal struct. - vtWrite sets `wrote_since_redraw = true`. The end of redraw clears it. resize() also clears `first_scrollback_row_hash` because reflow invalidates the row content. - Add `computeFirstScrollbackRowHash` helper: scrolls libghostty's viewport to the top, reads the first row's first ~16 cells, mixes them into an FNV-1a 64-bit hash, restores the viewport. Six libghostty calls — cheap, gated to only run when rotation is suspected (writes happened + we have scrollback + we have a stored hash). - At the start of redraw, before the existing delta sync, run the rotation check. If the stored hash differs from the freshly sampled hash, libghostty's scrollback has rotated underneath us: eraseBuffer, set scrollback_in_buffer = 0, force a full redraw. The delta-sync below will then see libghostty_sb - 0 = libghostty_sb and refetch everything fresh via insertScrollbackRange. - After the delta-sync, update the stored hash whenever we have scrollback so the next redraw has a fresh baseline. Why hash-the-row instead of comparing counts: libghostty's total_rows is allowed to plateau OR shrink when the cap is hit (it depends on page allocation and eviction semantics). Counter comparison alone misses the case where total_rows is steady at the cap with content rotating, and is wrong in the case where total_rows shrinks while content has actually rotated. Sampling the first row's content is the only signal that always tracks rotation correctly. Test (test/ghostel-test.el): - ghostel-test-scrollback-rotation-rebuild — write 5000 EARLY rows into a tiny-cap terminal (libghostty saturates at ~920 rows) + redraw, then write 5000 LATE rows without an intervening redraw, then redraw. The second redraw must detect rotation and rebuild so the buffer no longer contains any "early-" markers and shows the most recent late- rows. A previous version of this commit also included a "batched multi-row insert" optimization in insertScrollbackRange that collapsed N per-row env.insert calls into roughly N/page_rows calls. Empirically it only bought ~2-8% on the streaming bench because libghostty's vt_write parsing dominates redraw cost in that workload, not Elisp FFI. The added complexity (~140 lines: new RowMeta struct, new flushScrollbackChunk helper, cumulative char_offset arithmetic, chunk overflow handling, oversized-row fallback) wasn't worth the small gain. The batched-insert patch is preserved at .claude/batched-insert.patch and can be re-applied with `git apply .claude/batched-insert.patch` if the streaming hot path becomes more important later.
1 parent 34645e2 commit 9ed2a76

3 files changed

Lines changed: 160 additions & 1 deletion

File tree

src/render.zig

Lines changed: 97 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -396,6 +396,56 @@ fn isRowPrompt(term: *Terminal) bool {
396396
return semantic != 0;
397397
}
398398

399+
/// Hash the first ~16 cells of libghostty's first scrollback row using
400+
/// FNV-1a. Returns 0 if there is no scrollback or if anything fails.
401+
///
402+
/// Used to detect rotation: when libghostty's scrollback is plateaued at
403+
/// its byte cap, sustained writes evict the oldest row in lockstep with
404+
/// new rows being pushed, so `total_rows` doesn't change and the normal
405+
/// delta-detection sees no work to do. Sampling the first scrollback
406+
/// row's content lets us detect that the row at index 0 has changed
407+
/// underneath us.
408+
///
409+
/// Scrolls libghostty's viewport to the top to read the row, then
410+
/// restores the previous viewport offset. Cheap (~6 libghostty calls);
411+
/// gated by the caller to only run when rotation is suspected.
412+
fn computeFirstScrollbackRowHash(term: *Terminal) u64 {
413+
const sb = term.getScrollbar() orelse return 0;
414+
const saved_offset = sb.offset;
415+
416+
term.scrollViewport(gt.SCROLL_TOP, 0);
417+
defer {
418+
term.scrollViewport(gt.SCROLL_TOP, 0);
419+
if (saved_offset > 0) {
420+
term.scrollViewport(gt.SCROLL_DELTA, @intCast(saved_offset));
421+
}
422+
}
423+
424+
if (gt.c.ghostty_render_state_update(term.render_state, term.terminal) != gt.SUCCESS) return 0;
425+
if (gt.c.ghostty_render_state_get(term.render_state, gt.RS_DATA_ROW_ITERATOR, @ptrCast(&term.row_iterator)) != gt.SUCCESS) return 0;
426+
if (!gt.c.ghostty_render_state_row_iterator_next(term.row_iterator)) return 0;
427+
if (gt.c.ghostty_render_state_row_get(term.row_iterator, gt.RS_ROW_DATA_CELLS, @ptrCast(&term.row_cells)) != gt.SUCCESS) return 0;
428+
429+
// FNV-1a 64-bit hash. We mix in the first ~16 cells' first
430+
// codepoints (or a space for empty cells) — enough entropy to
431+
// distinguish rotation states without scanning the whole row.
432+
const fnv_prime: u64 = 0x100000001b3;
433+
var hash: u64 = 0xcbf29ce484222325;
434+
var i: usize = 0;
435+
while (i < 16 and gt.c.ghostty_render_state_row_cells_next(term.row_cells)) : (i += 1) {
436+
var graphemes_len: u32 = 0;
437+
if (gt.c.ghostty_render_state_row_cells_get(term.row_cells, gt.RS_CELLS_DATA_GRAPHEMES_LEN, @ptrCast(&graphemes_len)) != gt.SUCCESS) continue;
438+
if (graphemes_len == 0) {
439+
hash = (hash ^ ' ') *% fnv_prime;
440+
continue;
441+
}
442+
var codepoints: [4]u32 = undefined;
443+
if (gt.c.ghostty_render_state_row_cells_get(term.row_cells, gt.RS_CELLS_DATA_GRAPHEMES_BUF, @ptrCast(&codepoints)) != gt.SUCCESS) continue;
444+
hash = (hash ^ codepoints[0]) *% fnv_prime;
445+
}
446+
return hash;
447+
}
448+
399449
/// Result from buildRowContent: byte length for make_string, char count for properties.
400450
const RowContent = struct {
401451
byte_len: usize,
@@ -671,7 +721,9 @@ fn insertScrollbackRange(
671721
///
672722
/// When `force_full` is true, the viewport region is fully re-rendered
673723
/// instead of using the incremental dirty-row path.
674-
pub fn redraw(env: emacs.Env, term: *Terminal, force_full: bool) void {
724+
pub fn redraw(env: emacs.Env, term: *Terminal, force_full_arg: bool) void {
725+
var force_full = force_full_arg;
726+
675727
// Lock the libghostty viewport to the bottom. Users navigate history
676728
// through Emacs now, so any lingering scroll offset (e.g. from an
677729
// explicit ghostel--scroll) would desync our scrollback tracker.
@@ -693,6 +745,37 @@ pub fn redraw(env: emacs.Env, term: *Terminal, force_full: bool) void {
693745
_ = gt.c.ghostty_render_state_get(term.render_state, gt.RS_DATA_COLOR_FOREGROUND, @ptrCast(&default_fg));
694746
_ = gt.c.ghostty_render_state_get(term.render_state, gt.RS_DATA_COLOR_BACKGROUND, @ptrCast(&default_bg));
695747

748+
// ---- Scrollback rotation detection ------------------------------------
749+
// When libghostty's scrollback is at its byte cap, sustained writes
750+
// evict the oldest rows and push new ones, so the row at scrollback
751+
// index 0 changes underneath us. The normal delta-sync below tracks
752+
// `total_rows` deltas, but those don't capture content rotation —
753+
// if the count is unchanged (or even shrinking) the trim path would
754+
// remove our top rows under the *assumption* they match the rows
755+
// libghostty just evicted, which isn't true after rotation.
756+
//
757+
// Detect rotation by hashing the first scrollback row whenever
758+
// writes have happened since the last redraw and we have scrollback.
759+
// A change means the top row is no longer the row we materialized
760+
// → wipe the buffer and let the delta-sync below re-fetch everything
761+
// fresh from libghostty.
762+
if (term.wrote_since_redraw and term.scrollback_in_buffer > 0 and term.first_scrollback_row_hash != 0) {
763+
const new_hash = computeFirstScrollbackRowHash(term);
764+
// computeFirstScrollbackRowHash scrolled libghostty's viewport to
765+
// sample row 0 and the defer restored the offset, but the render
766+
// state may now be stale — refresh it before continuing.
767+
if (gt.c.ghostty_render_state_update(term.render_state, term.terminal) != gt.SUCCESS) return;
768+
if (new_hash != term.first_scrollback_row_hash) {
769+
// Rotation detected — erase the buffer entirely and force a
770+
// full viewport render. The delta-sync below will then see
771+
// libghostty_sb - 0 = libghostty_sb and refetch everything.
772+
env.eraseBuffer();
773+
term.scrollback_in_buffer = 0;
774+
term.first_scrollback_row_hash = 0;
775+
force_full = true;
776+
}
777+
}
778+
696779
// ---- Scrollback sync ---------------------------------------------------
697780
// libghostty stores scrollback + active screen in a single row space.
698781
// The rows "above" the viewport are scrollback; our invariant is that
@@ -1012,4 +1095,17 @@ pub fn redraw(env: emacs.Env, term: *Terminal, force_full: bool) void {
10121095
if (term.getPwd()) |pwd| {
10131096
_ = env.call1(emacs.sym.@"ghostel--update-directory", env.makeString(pwd));
10141097
}
1098+
1099+
// Update the cached first-scrollback-row hash for the next redraw's
1100+
// rotation check. Always re-sample (cheap) because previous-redraw
1101+
// promotion/insert/trim could have shifted the row at index 0.
1102+
if (term.scrollback_in_buffer > 0) {
1103+
term.first_scrollback_row_hash = computeFirstScrollbackRowHash(term);
1104+
} else {
1105+
term.first_scrollback_row_hash = 0;
1106+
}
1107+
1108+
// Clear the write flag so the next redraw can detect "writes happened
1109+
// since last redraw" for the rotation check.
1110+
term.wrote_since_redraw = false;
10151111
}

src/terminal.zig

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,19 @@ rows: u16,
3636
/// libghostty's scrollback cap.
3737
scrollback_in_buffer: usize = 0,
3838

39+
/// Set by `vtWrite`, cleared at the end of `redraw`. Used to detect that
40+
/// libghostty has been written to since the last redraw — required by
41+
/// the cap-bound stale-scrollback rebuild trigger to distinguish "no
42+
/// activity" from "writes happened but total_rows plateaued".
43+
wrote_since_redraw: bool = false,
44+
45+
/// Hash of the first scrollback row's content, sampled at the end of
46+
/// each redraw that touched scrollback. Used to detect rotation
47+
/// (libghostty evicting the oldest row in lockstep with new ones being
48+
/// pushed) when `total_rows` is plateaued at the cap. Zero means "no
49+
/// scrollback" or "not yet sampled".
50+
first_scrollback_row_hash: u64 = 0,
51+
3952
/// Cached Emacs env pointer — only valid during a callback from Emacs.
4053
env: ?emacs.Env = null,
4154

@@ -187,6 +200,7 @@ pub fn getColorBackground(self: *Self, out: *gt.ColorRgb) bool {
187200
/// Feed VT data from the PTY into the terminal.
188201
pub fn vtWrite(self: *Self, data: []const u8) void {
189202
gt.c.ghostty_terminal_vt_write(self.terminal, data.ptr, data.len);
203+
self.wrote_since_redraw = true;
190204
}
191205

192206
/// Resize the terminal.
@@ -202,6 +216,7 @@ pub fn resize(self: *Self, cols: u16, rows: u16) !void {
202216
self.cols = cols;
203217
self.rows = rows;
204218
self.scrollback_in_buffer = 0;
219+
self.first_scrollback_row_hash = 0;
205220
}
206221

207222
/// Scroll the viewport.

test/ghostel-test.el

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -263,6 +263,54 @@ detection, ghostel-prompt) stay attached."
263263
(should (string-match-p "second-05" content)))))
264264
(kill-buffer buf))))
265265

266+
(ert-deftest ghostel-test-scrollback-rotation-rebuild ()
267+
"Verify cap rotation triggers a rebuild so the buffer reflects libghostty.
268+
The test fills libghostty past its scrollback cap with EARLY markers,
269+
redraws once so the buffer matches the current libghostty state, then
270+
writes a much bigger batch of LATE markers (without an intervening
271+
redraw). When the next redraw runs, libghostty's `total_rows' is
272+
plateaued at the cap so the normal delta-detection sees nothing to do
273+
— the rotation-detect path must kick in, notice the first scrollback
274+
row's hash has changed, erase the buffer, and let the bootstrap fetch
275+
re-sync from libghostty so the buffer reflects the LATE rows."
276+
(let ((buf (generate-new-buffer " *ghostel-test-sb-rotate*")))
277+
(unwind-protect
278+
(with-current-buffer buf
279+
(let* (;; 4 KB cap empirically holds ~920 rows of short content
280+
;; in libghostty's compact storage.
281+
(term (ghostel--new 5 80 (* 4 1024)))
282+
(inhibit-read-only t))
283+
;; Phase 1: write 5000 EARLY rows. libghostty's scrollback
284+
;; saturates at ~920 rows so the surviving rows are
285+
;; early-04080..early-04999 (the most recent 920 of 5000).
286+
(dotimes (i 5000)
287+
(ghostel--write-input term (format "early-%05d\r\n" i)))
288+
(ghostel--redraw term t)
289+
;; After this redraw, buffer's scrollback_in_buffer matches
290+
;; libghostty's count (~920) and contains those high-numbered
291+
;; early rows.
292+
(let ((content (buffer-substring-no-properties (point-min) (point-max))))
293+
(should (string-match-p "early-04999" content)))
294+
;; Phase 2: write 5000 LATE rows WITHOUT redrawing in
295+
;; between. libghostty rotates: every new write evicts an
296+
;; early row and pushes a late row. After 5000 writes, all
297+
;; survivors are late-* (since 5000 > 920 cap).
298+
(dotimes (i 5000)
299+
(ghostel--write-input term (format "late-%05d\r\n" i)))
300+
;; Final redraw: total_rows hasn't changed (libghostty is
301+
;; still at the cap) but the content has fully rotated.
302+
;; Without rotation-detect this would be a no-op and the
303+
;; buffer would still show early-* rows.
304+
(ghostel--redraw term t)
305+
(let ((content (buffer-substring-no-properties (point-min) (point-max))))
306+
;; Late rows must be present (libghostty kept the most
307+
;; recent ones, the rebuild fetched them into the buffer).
308+
(should (string-match-p "late-04999" content))
309+
;; Early rows must NOT be present anywhere — libghostty
310+
;; evicted them AND the rebuild flushed our stale copy.
311+
(should-not (string-match-p "early-" content)))))
312+
(kill-buffer buf))))
313+
266314
;; -----------------------------------------------------------------------
267315
;; Test: clear screen (ghostel-clear)
268316
;; -----------------------------------------------------------------------

0 commit comments

Comments
 (0)