perf(image): image byte category in AnsiByteBreakdown + inline-image encoder bench (G2)#33
Merged
Merged
Conversation
Images are the largest per-frame payload, but AnsiByteBreakdown couldn't see them: the categorizer only understood CSI, so the string escapes that carry images — APC (Kitty ESC_G...ST), DCS (Sixel ESCP...ST), OSC 1337 (iTerm2) — fell through, and their payloads (KB of base64) were swept into `content`, not even `other`. So the #30 encoder fast path shipped unmeasured and an image-byte regression was invisible to every byte-level analysis (incl. the wire gate). - AnsiByteBreakdown gains an `image` category (APC/DCS/OSC-1337 recognized through their ST/BEL terminators; non-image OSC like clipboard 52 / hyperlink 8 correctly stay `other`). Counted as payload (in `total`, like `content`), NOT tunable `overhead`. Inert for image-free output (image=0), so the existing wire-gate axes are unchanged. Six unit tests pin the classification. - image_bench.dart (`fleury benchmark image-bench [--gate]`): image bytes/frame + encode us per protocol (kitty/iterm2), static vs animated, with a --gate on the two invariants that had no byte-level guard — a static image dedups (0 B/frame after the first) and an image-free frame emits 0 (the #30 fast path). Measured: kitty 403 B first / 0 static / 451 animated / 18.8 us; iterm2 414 / 0 / 414 / 6.8 us. Terminal encoder side; the browser/serve image wire (rides G1) and the embed InlineImageOverlay (DOM) are noted v2 follow-ups. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Adds terminal-side coverage for inline-image payloads by teaching AnsiByteBreakdown to recognize image-carrying string escapes (Kitty APC, Sixel DCS, iTerm2 OSC 1337) and introducing a small benchmark/gate to measure and guard image-bytes-per-frame and encode time.
Changes:
- Add an
imagebyte category toAnsiByteBreakdownand unit tests pinning classification (including a non-image OSC negative case). - Introduce
profiling/bin/image_bench.dartplus afleury benchmark image-benchentrypoint to measure image bytes/frame and encode µs, with an optional--gate. - Update perf-pass findings docs to mark G2 (terminal side) as built and reference the new tooling.
Reviewed changes
Copilot reviewed 4 out of 5 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tool/fleury_dev.dart | Adds image-bench command wiring under fleury benchmark …. |
| profiling/bin/image_bench.dart | New inline-image encoder benchmark + optional gate for invariants. |
| packages/fleury/test/rendering/ansi_byte_budget_test.dart | Adds unit tests validating new AnsiByteBreakdown.image classification. |
| packages/fleury/lib/src/rendering/ansi_byte_budget.dart | Implements image category and string-escape consumption for APC/DCS/OSC-1337. |
| docs/implementation/perf-pass-findings.md | Updates G2 status and documents the new measurement/gate. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+125
to
+147
| if (i < n) { | ||
| final intro = data.codeUnitAt(i); | ||
| // String escapes carry inline-image protocol payloads: APC (ESC _, | ||
| // Kitty graphics), DCS (ESC P, Sixel or a tmux image-passthrough | ||
| // envelope), OSC 1337 (ESC ], iTerm2). Consume through the string | ||
| // terminator (ST `ESC \` or BEL) as one run. Non-image OSC (clipboard | ||
| // 52, hyperlink 8) stays 'other'. | ||
| if (intro == 0x5F /* _ APC */ || intro == 0x50 /* P DCS */) { | ||
| i = _consumeStringEscape(data, i + 1, n); | ||
| image += i - start; | ||
| continue; | ||
| } | ||
| if (intro == 0x5D /* ] OSC */) { | ||
| final isImage = _oscIsImage(data, i + 1, n); | ||
| i = _consumeStringEscape(data, i + 1, n); | ||
| if (isImage) { | ||
| image += i - start; | ||
| } else { | ||
| other += i - start; | ||
| } | ||
| continue; | ||
| } | ||
| } |
Comment on lines
+242
to
+247
| const marker = '1337'; | ||
| if (from + marker.length > n) return false; | ||
| for (var i = 0; i < marker.length; i++) { | ||
| if (data.codeUnitAt(from + i) != marker.codeUnitAt(i)) return false; | ||
| } | ||
| return true; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
G2 (terminal side) — image-pipeline coverage. Images are the largest per-frame payload, but they were invisible to every byte-level analysis.
The gap
AnsiByteBreakdown(which the wire gate + scoreboard draw conclusions from) only understood CSI (ESC [). The escapes that carry images are string escapes, not CSI:ESC _ G … STESC P … STESC ] 1337 ; … BEL/STTracing the categorizer: a bare
ESC→other += 1, then the payload bytes aren't ESC, so they were swept into content. So image payloads (KB of base64) were mis-counted as text content — even worse than the "other" the finding assumed. The #30 encoder fast path shipped unmeasured, and an image-byte regression would trip no gate.What this adds
AnsiByteBreakdownimage category — recognizes APC / DCS / OSC-1337 through their ST/BEL terminators. Non-image OSC (clipboard 52, hyperlink 8) correctly staysother. Counted as payload (intotal, likecontent), not tunableoverhead— and inert for image-free output (image=0), so the existing wire-gate axes are unchanged. Six unit tests pin the classification, including the clipboard negative case.image_bench.dart(fleury benchmark image-bench [--gate]) — image bytes/frame + encode µs per protocol, static vs animated, with a--gateon the two invariants that had no byte-level guard: a static image dedups (0 B/frame after the first) and an image-free frame emits 0 (the perf(image): zero-image fast path + cache the kitty placement key #30 fast path).Measured:
Terminal encoder side. The browser/serve image wire rides G1's live harness; the embed
InlineImageOverlayis DOM, not bytes; sixel needs the RGBA sidecar — all noted v2 follow-ups.🤖 Generated with Claude Code