Repository navigation
From Keypress to Pixel
This page follows one plain uppercase A through the current application. The
pane has focus and uses plain text encoding, without active Win32 or Kitty
keyboard negotiation. No palette, search field, copy mode, IME composition, or
key binding consumes the key.
Pressing A does not draw A directly. SonicTerm sends bytes to the child
program. It draws only bytes that return through the pseudo-terminal (PTY).
An interactive shell usually echoes the byte, which makes the round trip look
immediate.
flowchart LR
key["WindowEvent::KeyboardInput<br/>logical A"]
encode["encode_key / encode_logical<br/>UTF-8 0x41"]
inq["bounded PTY input queue"]
writer["sonic-pty-writer"]
child["child process"]
reader["sonic-pty-reader"]
outq["bounded PTY output queue"]
worker["per-pane VT worker"]
parser["Parser::advance_with_replies / Performer"]
grid["Grid cell A<br/>dirty row + revision"]
event["PaneOutput(window_id, pane_id)"]
frame["complete PaneRender frame"]
font["FontStack + GlyphAtlas"]
choice{"presenter"}
wgpu["retained wgpu frame"]
cpu["SoftwareFrame + Windows GDI bridge"]
pixels(["window pixels"])
key --> encode --> inq --> writer --> child
child --> reader --> outq --> worker --> parser --> grid
grid --> event --> frame --> font --> choice
choice -- "wgpu" --> wgpu --> pixels
choice -- "Windows degraded" --> cpu --> pixels
App::do_resumed creates the first native window and renderer, enables IME,
and records the monitor period. Later windows share the first adapter/device/queue
through GpuSharedContext, but own their surfaces and drawing state.
Every presenter requires successful wgpu initialization: Windows CPU/GDI is not
an adapter-free recovery path. Adapter classification and auto/force/off
policy are separate; see Rendering Modes for their exact rules.
flowchart TD
window["native window + wgpu surface"] --> adapter{"compatible adapter?"}
adapter -- "no" --> fail["renderer initialization fails"]
adapter -- "yes" --> classify{"software adapter?"}
classify --> policy{"software_render_mode"}
policy -- "auto" --> auto["degrade = detected"]
policy -- "force" --> force["degrade = true"]
policy -- "off" --> off["degrade = false"]
auto --> platform{"platform + degrade"}
force --> platform
off --> platform
platform -- "Windows + true" --> gdi["CPU BGRA + GDI"]
platform -- "all other cases" --> gpu["wgpu surface"]
The renderer prepares a retained frame, body/footer/title font stacks, separate glyph and image atlases, and row caches. Surface and pacing policy are detailed in Rendering Modes; allocations are listed in Memory.
winit sends WindowEvent::KeyboardInput for presses, repeats, and releases.
SonicTerm retains the complete event: the physical key, layout-resolved logical
key, operating-system-produced text, keypad location, event state, and repeat
marker. For this example the layout has resolved the logical character and text
as uppercase A.
A local input owner may stop the route. Main and child windows share the same
source-WindowId first-press policy:
- quit confirmation;
- command palette;
- active IME composition;
- search;
- READONLY or copy mode;
- configured keymap;
- PTY encoding.
Previously accepted repeats and releases keep their recorded terminal owners before this first-press policy runs. An open search and active composition take precedence over READONLY navigation in every window. Quick-select hint keys stay with the hint overlay instead of becoming application shortcuts. Quit warnings belong to the event's source window, without changing recorded native focus. Unknown, removed and unpromoted warm windows cannot fall through to main input.
IME events use one WindowId-scoped handler before the main/child dispatch split.
The source window's palette consumes composition first; otherwise that window's
IME state supplies committed UTF-8. Its active search owns the commit before
READONLY/copy mode can discard it; without either owner, the source pane receives
it through the existing PTY and broadcast boundaries. Focus in another window
cannot redirect it, and an unknown or removed window does nothing. An open search
retains ownership even if its pane is temporarily missing. Search commits and
search keystrokes share window-scoped handlers that preserve viewport anchoring.
While composition is active, the shared keyboard route suppresses raw input.
Modifier changes and focus cleanup also resolve the source window explicitly:
blur releases its accepted native keys and latched pointer gesture, cancels its
preedit, and reports focus only to its active pane. Focus-in resets its IME caret
throttle without toggling the native input context.
The terminal IME anchor uses the active pane's physical origin, content padding,
and cursor cell with live physical cell metrics. It adds each offset once,
without another DPI multiplication. Each window coalesces identical
(pane id, physical position, physical size) updates; equal-cell focus changes,
zoom, transfer, font/padding changes, and resize still update the native anchor.
Palette and search retain their separate field anchors and reset the terminal
anchor cache before returning input ownership.
Only a press that survives local routing and reaches at least one bounded PTY input queue is recorded as PTY-owned. Its accepted pane set stays fixed for the whole lifecycle: repeats consult it before any palette, search, or keymap owner that opened later, and releases return to it even if focus or broadcast state changed. Native Win32 routes additionally retain their accepted protocol epoch; a protocol transition or reset cancels them, and focus loss drains their synthetic releases while Win32 remains active. A locally consumed or rejected press creates no orphan repeat or release event.
The app selects the encoding protocol from one coherent pane snapshot. On
Windows, requested Win32 input with no active Kitty flags uses the native
metadata carried by that event; it does not reconstruct a key from UTF-8 text.
Other routes call encode_key. In this plain-text example, Key::Character
with no Control or Alt modifier uses the operating-system-produced UTF-8 text
unchanged.
| Property | Value |
|---|---|
| Character | A |
| Code point | U+0041 |
| UTF-8 | 0x41 |
| Decimal byte | 65 |
Modified, keypad, Win32, and Kitty encodings follow the keyboard protocol reference. This example remains UTF-8 0x41.
The app writes the focused source pane exactly once. Broadcast adds peers only
when the focused pane is still the pane that armed broadcast.
BroadcastScope::Tab selects peers in that tab.
BroadcastScope::AllTabs selects peers across tabs and windows. The source is
excluded from the receiver set, and so is every pane whose window is in READONLY
mode.
Each destination crosses this live boundary:
flowchart LR
source["stable PaneId + bytes"] --> write["App::write_to_pane"]
write --> pane["live PaneState / PtyHandle"] --> queue["bounded input queue"]
There is no transient state machine on the native input or broadcast path.
Explicit AppIntent::PtyWrite and AppEffect::PtyWrite enter the same bounded
write boundary with their named pane id. Window-targeted compatibility input
resolves that live window's active pane, never a zero sentinel or guessed
frontmost window, and is dropped while that window is READONLY. A missing target
cannot redirect bytes to another terminal.
PtyHandle::send_input_nonblocking uses try_send:
- queue capacity: 4 messages per pane;
- message limit: 16 MiB;
- rejection cases:
MessageTooLarge,QueueFull,WriterDisconnected.
Every PtyInputError retains the rejected Vec<u8> at the IO boundary. The app
drops those bytes before posting metadata-only UserEvent::PtyInputRejected.
It logs pane identity, the current window, producer-assigned input category,
byte count, reason, and concurrent queue/writer observations, then notifies the
pane's window if it still exists. It does not retry automatically because the
child's input state may change before a later replay.
The dedicated sonic-pty-writer thread removes the owned byte vector, calls
write_all, then attempts a best-effort flush. A failed write stops the
writer. At this point SonicTerm has drawn no A.
The child program receives 0x41 from its PTY side. An ordinary interactive
shell usually has echo enabled, so 0x41 returns as output. Echo belongs to the
child-side terminal behavior, not to SonicTerm.
A raw-mode editor can consume A and send a larger redraw. A password prompt
can send no visible output. A program can send different text. SonicTerm parses
only the bytes that return through the PTY master.
On Unix, portable-pty supplies a native PTY. On Windows it supplies ConPTY.
The shared byte, VT, grid, font, and renderer path starts after this platform
boundary.
sonic-pty-reader reads into a reusable contiguous 64 KiB BytesMut
allocation. It splits filled prefixes into reference-counted Bytes views and
wraps them in PtyOutputChunk. This is reusable flat storage, not a circular
ring data structure. If old views pin the allocation, reserve can allocate
another 64 KiB ring.
The output channel holds 64 chunks. A full channel does not drop output. The reader waits in a blocking select, which lets the operating system's PTY buffers apply back-pressure to the child.
The channel can hold 64 chunks. The reader constructs one more chunk before a
full-channel send blocks. If every chunk pins a distinct 64 KiB ring, the
structural maximum is 65 rings, or 4.0625 MiB. Small shell output normally keeps
many queued views in one ring. queued_output_bytes reports pinned ring
allocation; queued_output_payload_bytes reports bytes waiting to be parsed.
A pane created through the main path uses sonicterm-vt-loop. A pane created
directly in a torn-out window uses sonicterm-vt-loop-child. A pane whose PTY
spawn fails remains visible but has no PTY reader, writer, or VT worker.
The pane worker receives a chunk and holds that pane's parser lock for
Parser::advance_with_replies and its keyboard-input snapshot. A reply-producing
dispatch yields the consumed prefix, events, and replies; the worker releases
the lock before handling events and batching replies, then resumes the remaining
suffix. Complete replies reach the separate reply FIFO outside parser locks.
Plain ASCII A takes the parser's printable fast path to
Performer::print_graphic. Other printable UTF-8 reaches the same operation
through vte. Controls and escapes use execute, csi_dispatch, osc_dispatch,
esc_dispatch, or DCS hook/put/unhook. Kitty graphics APC input is
intercepted before vte.
The performer applies the current foreground, background, bold, italic, underline, inverse, and hyperlink id. The URI remains in the hyperlink registry. The performer then calls the grid.
Grid::put_char_styled_in_region stores A as a width-one Cell. It advances
the cursor in the ordinary case, marks the row dirty, advances the row content
sequence, and advances the coarse grid revision.
At the right margin, autowrap sets a one-past-edge cursor and pending_wrap.
The next printable character performs the wrap. Only that actual transition
marks the destination Line as soft-wrapped from its predecessor; a pending
wrap alone records nothing persistent. LF, VT, FF, IND, NEL, screen erases,
structural region scrolling, inserted or deleted lines, row recycling, non-reflow
resize, and row_mut clear provenance where continuity cannot be proved. An edit
that changes the wrapping row's last column (a write there, EL0, EL2, EL1 or ECH
reaching it, ICH, DCH, or a combining mark on a last-column cell) revokes the wrap
it passes to the next row. Erasing or writing a continuation row from column 0
keeps its incoming wrap, as xterm and WezTerm do, so zsh's \r ESC[K redraw of a
wrapped line stays one logical line. The bit is packed into the row's
existing content-sequence word, so Line does not grow, and it participates in
row equality and hashing. Without autowrap, the cursor stays on the final
column.
Dirty means “this row changed.” The dirty bit, content sequence, wrap provenance, and grid revision are separate bookkeeping signals for repaint work, logical line identity, content identity, and coarse frame identity.
Local-target lookup can walk backward and forward through at most 32 visible rows joined by recorded automatic wraps, flattening at most 16 KiB while retaining a byte-to-absolute-cell map. Hard line breaks, an offscreen edge, an evicted predecessor, or a 33rd row fail closed. On the alternate screen the walk also follows inferred pane-edge continuations: a pane segment whose last column holds text continues into the next visible row's segment with the same pane edges when that segment starts with text. Those chains carry paths and URLs and stop at 32 rows and the view edges. Candidates are probed shortest first by Unicode scalar count of the displayed text, with ties going to the earlier start. The asynchronous probe key binds the ordered row fingerprints and wrap bits, screen incarnation, viewport, exact pane CWD, candidate spans, and pointed absolute cell. Activation rebuilds that key before native target revalidation, and the open worker repeats the selection over every candidate at or before the selected tier before it opens.
Cell representation is a separate concern. Wide characters use WIDE and
WIDE_CONT cells. Zero-width characters append to the lead cell's extras,
capped by MAX_CELL_EXTRAS_BYTES = 64; a code point that would exceed the cap
is dropped.
While holding the parser lock, the worker publishes the packed keyboard_input
snapshot, including keyboard modes, Kitty flags, and the protocol epoch. After
unlocking, it updates cursor_visible from returned VtEvent::CursorVisibility
events and handles clipboard, command, and media events. Calls to the event-loop
proxy also occur outside the parser lock.
Redraw requests are coalesced by bytes and time:
- 128 KiB pending output flushes immediately;
- 8 ms maximum pending age flushes a continuing stream;
- otherwise 3 ms without another chunk flushes trailing output.
At a flush boundary, the worker copies the pane's current WindowId under a
short redraw-target lock. It releases that lock and sends
UserEvent::PaneOutput { window_id, pane_id }, unless that pane already has one
outstanding. The winit thread acknowledges the pane in the window that holds it
now and requests a frame only when that window shows new output or changed
command chrome. An event for a pane no window holds is ignored.
The perf harness splits S2/default's keystroke latency at this boundary. With the counter gate on, an armed echo watch records the parse section in which the typed character first appears in the grid, then the first flush published after it, before the outstanding-event decision, then that decision. Injection to that parse section, the parse to the publication, and the publication to the end of the credited frame are the three parts; Logging defines them.
This indirection lets a pane move between windows. Transfer changes the shared
WindowId; the existing worker and child process continue unchanged. The
receiving tab is activated before its visible grids and PTYs are resized to the
destination pane rectangles, without a whole-window intermediate size.
Zoom-hidden siblings keep their prior size until they become visible.
A second pacing gate may defer streaming output to the next frame boundary.
Hardware keeps pure input redraws immediate. PTY output is bounded by the
monitor frame period. Resolved degradation also coalesces pure input redraws to
the software frame period. A timed ControlFlow::WaitUntil wakes the event loop
and requests the frame again.
On RedrawRequested, the app computes the active tab's pane rectangles. It
uses try_lock for every required inline-image store and every active-tab
parser, and moves all parser guards into the frame source the renderer assembles
from; they are released as soon as assembly ends, before presentation. If one lock is
unavailable, it drops every collected guard, records a pending redraw, and
returns without calling the renderer. The frame is complete or absent;
SonicTerm does not present a mix of old and new pane state.
For each visible pane, the app builds PaneRender with:
- stable pane id;
- mutable grid view;
- pixel rectangle and viewport;
- active status and cursor style;
- broadcast-receiver status;
- scrollbar alpha;
- shallow-cloned inline-image records with shared
Arc<[u8]>pixels.
The production call passes the pane slice plus explicit theme, cursor,
selection, copy mode, tabs, search, palette, IME, viewport, notification, and
hovered-URL data to GpuRenderer::render_releasing, with the panes lent by the
frame source. It does not construct one
aggregate RenderInputs value.
A changed FrameKey triggers work. Primary-screen changes damage dirty-row
strips; an alternate-screen change damages its complete pane. UI changes can
require the full surface. Row caches reuse unchanged glyphs and backgrounds.
An identical key skips assembly; Windows degraded presentation may reblit its
existing CPU frame. Exact keys, capacity, pane eviction, and damage rules live in
Rendering and Fonts.
Cells with compatible font style form runs. A conservative printable-ASCII run
can skip full shaping. Each cell must contain printable ASCII, no combining
extras, no wide-cell flag, and none of these ligature triggers:
= ! < > - _ : | & *. Plain A qualifies.
The shortcut is not a second font system. An atlas miss still calls
FontStack::rasterize. Unicode, combining text, fallback fonts, and
ligature-capable runs call FontStack::shape_text_with_style, which uses
HarfBuzz and maps shaped clusters back to terminal columns.
The font stack tries the configured family, configured fallbacks, then native discovery. Matching, platform rasterizers, color, and decorations are detailed in Rendering and Fonts.
Rasterization returns a bitmap and placement metrics: width, height, bearing, advance, and whether the data is monochrome, subpixel, or self-colored. This is a reusable tile, not a screen pixel.
GlyphAtlas::get_or_insert reuses a cached tile or allocates a rasterized one.
GlyphInstance records its screen rectangle, atlas UVs, foreground, and sampling
flags. If atlas eviction invalidates earlier instances in this frame, the
renderer abandons it and retries without clearing dirty rows. The next frame
disables eviction until one presentation succeeds. Atlas storage and fallback
sentinels are documented in Rendering and Fonts.
On wgpu, AtlasUpload::sync uploads dirty rectangles; a cached A needs no
upload. Drawing updates the retained offscreen frame inside its damage scissor,
blits it to the surface, submits commands, and calls queue.present(frame).
On Windows with degradation enabled, the same prepared instances are composed
into a full CPU BGRA frame and presented with SetDIBitsToDevice. CPU atlas bytes
remain unchanged; the GPU mirrors are placeholders. Color conversion, layer order,
sampling, and size limits are specified in Rendering and Fonts.
Windows CPU presentation calls finish_successful_frame only after
SetDIBitsToDevice returns success. wgpu calls it after command submission and
queue.present(frame). The wgpu present call itself has no success result for a
later compositor failure.
finish_successful_frame stores the new FrameKey and increments the successful
frame count; it clears no dirt. The presented frame's receipts are applied at the
window's next collection: when the same parser is held and the pane's size generation and
screen epoch still match, it clears the pane's dirty rows except those dirtied after assembly.
Before a wgpu draw:
- timeout or occlusion invalidates the key and requests another redraw;
- outdated or suboptimal also reconfigures the surface;
- lost recreates the surface;
- validation errors return an error.
None of those acquisition paths clears dirty rows. An eviction-aborted frame
also leaves them set. RenderMode::Noop stores a key but does not present or
clear dirty rows because it produced no image.
After a drawn frame completes, the window compositor and display system scan out
the newly presented pixels. The echoed A is now visible.
Font, DPI, theme, surface, and pane-layout changes invalidate the affected frame/cache identity. See Rendering and Fonts for the exact operations; Architecture Internals defines when dirty rows may be acknowledged.
Dropping PtyHandle cancels I/O and terminates the child, then performs bounded
platform-specific close and reaping. Incomplete cleanup is not success. See
Runtime Lifecycle for close order and
Architecture Internals for Unix/ConPTY deadlines.
Validation and release evidence are described in
Development and Release, not inferred from this journey.
| Boundary | Normal reason |
|---|---|
| Local input owner | palette, search, copy/READONLY mode, IME, or a key binding consumed it |
| Child program | echo is off, the program drew something else, or it emitted nothing |
| PTY input | the message was too large, the queue was full, or the writer disconnected; the app shows an error |
| Pane process | PTY spawn failed, so the visible pane has no worker |
| Frame collection | a parser or image lock was busy; the whole frame was deferred |
| Renderer | a surface or atlas recovery path requested a later frame |
| Cache | work was reused; the visible result is unchanged |
| Step | Primary paths |
|---|---|
| Keyboard and IME routing | crates/sonicterm-app/src/app/{window_event,window_keyboard}.rs |
| Key encoding | crates/sonicterm-app/src/app/key_encoding.rs |
| Intent/effect PTY boundary |
crates/sonicterm-app-core/src/{intent,effect,reducer,state_machine}.rs, crates/sonicterm-app/src/app/{effects,input_dispatch}.rs
|
| PTY queues and threads | crates/sonicterm-io/src/pty.rs |
| VT workers and redraw coalescing | crates/sonicterm-app/src/app/{spawn_pane,child_tabs,child_window,child_window_redraw,redraw_target}.rs |
| VT parsing | crates/sonicterm-vt/src/vt.rs |
| Cell insertion and dirty rows | crates/sonicterm-grid/src/grid.rs |
| Frame collection | crates/sonicterm-app/src/app/{window_event,child_window_redraw}.rs |
| Pane frame type | crates/sonicterm-render-model/src/pane_render.rs |
| Damage, caches, glyph instances, and presentation | crates/sonicterm-gpu/src/{core,row_quad_cache,software_frame,software_windows}.rs |
| Fonts |
crates/sonicterm-engine/src/fontstack.rs, crates/sonicterm-font/src/
|
| CPU glyph atlas and row glyph cache | crates/sonicterm-text/src/{glyph_atlas,row_glyph_cache}.rs |