Skip to content

fix(pager): stop CPU pegging and truncation on large piped documents - #978

Merged
benvinegar merged 4 commits into
mainfrom
claude/hunk-cpu-pegging-6gl3gi
Sep 4, 2026
Merged

fix(pager): stop CPU pegging and truncation on large piped documents#978
benvinegar merged 4 commits into
mainfrom
claude/hunk-cpu-pegging-6gl3gi

Conversation

@benvinegar

@benvinegar benvinegar commented Sep 4, 2026

Copy link
Copy Markdown
Member

Fixes two independent bugs on the hunk pager path. The first explains the reported CPU pegging; the second was found while verifying the first, and without it the reported scenario still fails, just quickly instead of slowly.

1. Quadratic ANSI restoration pegged a core

hunk pager pegs a CPU core and grows to gigabytes of resident memory on large color-heavy input, such as the git log --graph --color=always streams LazyGit routes through Git's pager. Reporters saw three concurrent LazyGit branch-log jobs each running at ~100% CPU indefinitely, never emitting output.

sanitizeTerminalText preserved ANSI color by swapping each SGR sequence for a placeholder token, then restoring them one at a time:

for (const [index, sequence] of preservedStyles.entries()) {
  sanitized = sanitized.replaceAll(`\u{f0000}${index}\u{f0001}`, sequence);
}

Each call rescans and reallocates the whole document, making the work quadratic in the sequence count. Real git log --graph output carries a sequence roughly every 18 bytes, so a 3 MB branch log holds ~170k of them.

That also explains why the processes outlived LazyGit's jobs: the loop is synchronous, so the process cannot react to its pipe closing until the loop finishes. The pager already exits promptly on pipe close, SIGHUP, and parent death once it is not stuck.

Both LazyGit-facing paths hit this, since both request preserveAnsiStyle: the captured-pager passthrough in src/main.tsx and pagePlainText in src/core/process/pager.ts.

Fix: restore every placeholder in a single pass. The anti-spoofing guard is unchanged, since input delimiters are still stripped before tokenization, so every surviving token indexes a captured style.

Same 3 MB fixture, same environment, real CLI:

before after
wall clock still running at 240 s, killed 0.4 s
peak resident 475 MB and climbing 163 MB
output produced 0 bytes 3.01 MB

Isolated sanitizer timings showed clean quadratic growth before the fix, from 0.55 s at 64 KB to 17.9 s at 512 KB, extrapolating to roughly ten minutes of CPU at 3 MB.

A differential fuzz over 20,002 generated and real cases confirms byte-identical output against the old implementation, across every option combination and adversarial delimiter input.

2. Piped output was silently truncated at 64 KB

Found while verifying the above, and reproduced on unmodified main: any document read through a pipe stopped at exactly 65536 bytes, one Linux pipe buffer, with exit code 0 and no error. Writing to a file or a terminal delivered everything, which is why the loss stayed hidden. A pipe is exactly how LazyGit and Git's pager contract consume Hunk, so the reported scenario would have produced fast but truncated logs after fix 1 alone.

Headless commands write their document in one call and then process.exit, which drops whatever stdout has not handed over yet. Bun offers nothing to wait on: for a pipe, process.stdout.write returns true and writableLength stays 0 while only one buffer has actually been delivered, and the write callback fires just as early. Draining before exit is not available.

Fix: write straight to the stdout descriptor, resuming partial writes until the consumer has taken every byte. A consumer that stops reading early (| head) ends the run quietly rather than failing, which Bun.write does not.

Verified on a 513 KB document, captured-pager host:

stdout before after
pipe 65536 bytes 525601 bytes
file 525601 bytes 525601 bytes

Tests

  • src/lib/terminalText.test.ts: correct styled output on 30k sequences within a 2 s budget. Verified to fail at 18.8 s against the pre-fix implementation.
  • src/core/process/pager.test.ts: the same input through pagePlainText.
  • src/core/process/stdout.test.ts: partial-write resumption, multi-byte characters split across writes, non-blocking retry, early close, unexpected errno.
  • test/cli/pager-pipe-output.test.ts: black-box CLI check that a piped consumer receives the whole document. Verified to fail against the pre-fix entrypoint.

Typecheck, lint, format, dependency rules, the unit suite, the CLI contract suite, and the PTY suites pass. Remaining failures in this container are pre-existing and environmental: one Jujutsu test needs the jj binary, and two watch-mode tests require Bun 1.3.14 (container has 1.3.11).

Note on --fast

--fast does not help this scenario and never did. It only sets offloadLargeDiff on DiffPane and launchFast in AppHost, both inside the mounted TUI, which non-patch pager input never reaches. Measured on unmodified main with a 208 KB colored log: 3.3 s without it, 3.0 s with it, both truncated to 65536 bytes.

Still buffered

The pager continues to read stdin fully and write once, so peak memory scales with input (about 163 MB for 3 MB). That is bounded and linear now rather than unbounded. Streaming the passthrough path would cut it further and is a reasonable follow-up, but it is a larger change than these two fixes.

🤖 Generated with Claude Code

https://claude.ai/code/session_011eVgGiCsPXU4N8TgsHLy4w

`hunk pager` pegged a CPU core and grew to gigabytes of resident memory on
large color-heavy input, such as the `git log --graph --color=always` streams
LazyGit routes through Git's pager. Three concurrent LazyGit branch-log jobs
each ran at ~100% CPU indefinitely and never emitted output.

`sanitizeTerminalText` swapped every preserved SGR sequence for a placeholder
token, then restored them with one `replaceAll` per sequence. Each of those
rescanned and reallocated the whole document, making the work quadratic in the
sequence count. Real `git log --graph` output carries a sequence roughly every
18 bytes, so a 3 MB branch log holds ~170k of them: minutes of solid CPU and
hundreds of megabytes of churn per process, none of it interruptible, since the
loop is synchronous and cannot react to the host closing the pipe.

Restore every placeholder in a single pass instead. Output is unchanged, and
the spoofing guard still holds because input delimiters are stripped before
tokenization, so each surviving token indexes a captured style.

Measured on the same 3 MB fixture and environment: 240s+ still running with no
output, before; 0.4s and 163 MB peak resident, after. A differential fuzz over
20k generated and real cases confirms byte-identical output.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011eVgGiCsPXU4N8TgsHLy4w
@vercel

vercel Bot commented Sep 4, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
hunk-web Ignored Ignored Preview Sep 4, 2026 4:47pm UTC

Request Review

@greptile-apps

greptile-apps Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Replaces quadratic ANSI placeholder restoration with a single-pass replacement while preserving the existing anti-spoofing behavior.

  • Adds direct regression coverage for dense ANSI input.
  • Adds pager-level coverage confirming styled output is retained promptly.
  • Documents the pager performance fix in a patch changeset.

Confidence Score: 5/5

The PR appears safe to merge with no actionable correctness, security, or repository-rule violations identified.

The new restoration pass is behaviorally equivalent to the prior per-token loop for generated placeholders while removing its quadratic rescanning cost, and the affected sanitizer and pager paths have focused regression coverage.

Important Files Changed

Filename Overview
src/lib/terminalText.ts Restores captured ANSI style placeholders in one linear pass without changing the sanitizer’s accepted style sequences.
src/lib/terminalText.test.ts Adds a dense ANSI regression test covering output parity and the previous performance failure.
src/core/process/pager.test.ts Verifies the plain-text pager preserves and promptly writes ANSI-heavy Git log output.

Reviews (1): Last reviewed commit: "fix(pager): restore ANSI styling in one ..." | Re-trigger Greptile

`hunk pager` truncated its output at 65536 bytes, one Linux pipe buffer,
whenever a host read it through a pipe. That is exactly how LazyGit and Git's
pager contract consume it, so branch logs arrived silently cut off. Writing to
a file or a terminal absorbed the whole document, which is why the loss only
appeared under a pipe.

Headless commands write their document in one call and then `process.exit`,
which drops whatever stdout has not handed over yet. Bun offers nothing to wait
on here: for a pipe, `process.stdout.write` returns true and `writableLength`
stays 0 while only one buffer has actually been delivered, and the write
callback fires just as early, so draining before exit is not available.

Write straight to the stdout descriptor instead, resuming partial writes until
the consumer has taken every byte. A consumer that stops reading early (`|
head`) ends the run quietly rather than failing, which `Bun.write` does not.

Verified on a 513 KB document: 65536 bytes delivered through a pipe before,
the full 525601 after, matching what the same run writes to a file.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011eVgGiCsPXU4N8TgsHLy4w
@benvinegar benvinegar changed the title fix(pager): restore ANSI styling in one pass fix(pager): stop CPU pegging and truncation on large piped documents Sep 4, 2026
@benvinegar
benvinegar merged commit bfbfac8 into main Sep 4, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants