dtterm: modern enhancements#1
Open
runlevel5 wants to merge 16 commits into
Open
Conversation
Promote dtterm's per-cell colour fields, the enhValue transport typedef,
and the parser-side colour state variables from the original 4-bit /
8-bit shape to a full 32-bit packed encoding.
The encoding carries a one-byte mode flag in the high bits and a 24-bit
payload in the low bits:
bits 24-31 mode -- 0=DEFAULT, 1=INDEXED, 2=DIRECT_RGB
bits 0-23 payload -- palette slot or 0xRRGGBB
DEFAULT is all-zero, so the zero-initialised state of fresh cells
continues to mean 'use the widget default fg / bg' without any per-cell
fix-up at allocation time. SGR 39 / 49 round-trip back to DEFAULT
instead of getting confused with INDEXED 0.
User-visible behaviour is unchanged: the SGR handler now stores
ENH_MAKE_INDEXED(1..8) for 30-37 / 40-47, and the renderer decodes that
back to the existing colorPairs[1..8] slots. No new escape sequences
are recognised yet; this change just lifts the type ceiling that
previously capped colour at 16 indexed slots, so the 256-colour and
24-bit truecolour additions can land as small follow-ups.
Per-cell enhancement memory grows from ~4 to ~12 bytes (roughly 640 KB
extra for an 80x24 window with 1000-line scrollback at the defaults).
Files touched:
cde/lib/DtTerm/TermPrim/TermPrimBuffer.h
- enhValue: unsigned char -> uint32_t
- ENH_MODE_*, ENH_COLOR_MODE, ENH_COLOR_PAYLOAD, ENH_IS_DEFAULT,
ENH_MAKE_DEFAULT, ENH_MAKE_INDEXED, ENH_MAKE_RGB helpers
cde/lib/DtTerm/Term/TermBufferP.h
- DtTermEnhPart: fgColor:4 / bgColor:4 bit-fields replaced by
full uint32_t fields; video / field / font kept as bit-fields
cde/lib/DtTerm/Term/TermBuffer.h
- COLOR_MASK widened from 0x0F to 0xFFFFFFFFU
cde/lib/DtTerm/Term/TermData.h
- enhFgColorState / enhBgColorState widened to uint32_t in both
DtTermDataRec and VtSaveCursorRec
cde/lib/DtTerm/Term/TermFunction.c
- _DtTermVideoEnhancement uses ENH_MAKE_DEFAULT / ENH_MAKE_INDEXED
for SGR 0 / 30-37 / 39 / 40-47 / 49
cde/lib/DtTerm/Term/TermEnhance.c
- _DtTermEnhProc decodes the packed colour value through a small
helper and indexes colorPairs[] with the resolved slot
Co-Authored-By: Trung Le <trung.le@ruby-journal.com>
Extend the indexed-colour palette from 8 ANSI slots to the full 16
slots used by xterm-compatible bright-colour SGR sequences, and apply
the standard bold-brightens-fg convention so a bold ANSI 30-37 fg
renders with the brighter variant.
SGR 90-97 -> bright fg, stored as ENH_MAKE_INDEXED(9..16)
SGR 100-107 -> bright bg, stored as ENH_MAKE_INDEXED(9..16)
SGR 1 + 30-37 fg -> bright variant at render time (xterm convention)
The widget's colorPairs[] array grows from 9 to 17 entries (default +
16 ANSI slots). Slots 9..16 are seeded with xterm-style RGB defaults:
slot 9: bright black #808080
slot 10: bright red #ff0000
slot 11: bright green #00ff00
slot 12: bright yellow #ffff00
slot 13: bright blue #5c5cff
slot 14: bright magenta #ff00ff
slot 15: bright cyan #00ffff
slot 16: bright white #ffffff
The destroy loop's upper bound is updated to cover all 17 slots, which
also fixes a pre-existing off-by-one that left the slot-8 (white)
pixels unfreed when the widget was destroyed.
The bold-brightens-fg promotion lives in _DtTermEnhProc rather than
the SGR handler so SGR 22 (bold off) restores the original colour
without requiring the parser to remember whether the explicit colour
was a 30-37 or a 90-97 sequence. Promotion only fires for
ENH_MODE_INDEXED payloads 1..8 -- ENH_MODE_DEFAULT and future
ENH_MODE_RGB values pass through untouched.
Files touched:
cde/lib/DtTerm/Term/TermData.h
- VtColorPairRec colorPairs[9] -> colorPairs[17]
cde/lib/DtTerm/Term/TermColor.c
- _DtTermColorInit seeds slots 9..16 with bright RGB defaults
- _DtTermColorDestroy iterates all 17 slots (was 8)
cde/lib/DtTerm/Term/TermFunction.c
- _DtTermVideoEnhancement recognises SGR 90-97 / 100-107
cde/lib/DtTerm/Term/TermEnhance.c
- _DtTermResolveColorPair accepts indexed payloads 1..16
- _DtTermEnhProc applies bold-brightens-fg for INDEXED 1..8
Add support for the xterm 256-colour palette and rewrite the SGR
parameter loop into a peek-ahead state machine that consumes the
compound colour sub-sequences atomically:
38;5;N indexed 256-colour foreground
48;5;N indexed 256-colour background
38;2;R;G;B direct-RGB foreground (parser only; renderer in a
follow-up change)
48;2;R;G;B direct-RGB background (parser only; renderer in a
follow-up change)
The maximum number of parameters per CSI sequence is raised from 16
to 32 in both NPARAM (Term/TermParse.c) and NUM_PARMS
(TermPrim/TermPrimParserP.h) so compound sequences like
'0;1;38;2;255;128;0;48;2;64;32;16;4' fit with headroom.
The render-side resolver is reorganised. _DtTermResolveColorPair
returned a colorPairs[] index, which can only express the static
0..16 slots. In its place are two Pixel-returning helpers,
_DtTermResolveFgPixel and _DtTermResolveBgPixel, that handle DEFAULT,
the 16 named slots and -- when the payload is 17..256 -- delegate to
the new lazy 256-palette resolver. This keeps the rest of
_DtTermEnhProc agnostic to the colour mode and gives Phase D
(direct-RGB rendering) a single hook to extend.
The xterm 256 palette is allocated lazily as colours appear on screen:
TermData.h Pixel pal256[256]; Boolean pal256Allocated[256];
TermColor.c _DtTermColor256RGB returns the canonical xterm
RGB triple (16..231: 6x6x6 cube using xterm's
{0x00,0x5f,0x87,0xaf,0xd7,0xff} step table;
232..255: 24 grey steps from 8 to 238).
_DtTermResolve256Pixel allocates each entry on
first use via XAllocColor and caches the Pixel;
falls back to the widget's default fg if the
server's colormap is exhausted.
_DtTermColorDestroy now batches the populated 256-palette slots into a
single XFreeColors call alongside the existing colorPairs cleanup.
A new helper _DtTermSetColor (TermFunction.c) writes a packed enhValue
to fg or bg and stamps it on the cell at the cursor. The parser uses
it for the 38;5 / 38;2 / 48;5 / 48;2 paths; the legacy SGR 30-37 /
90-97 paths continue to go through _DtTermVideoEnhancement, which is
re-entered for any non-colour parameter in a chained sequence.
Files touched:
cde/lib/DtTerm/TermPrim/TermPrimParserP.h
- NUM_PARMS 20 -> 32 (sized for parms[NUM_PARMS])
cde/lib/DtTerm/Term/TermParse.c
- NPARAM 16 -> 32
- _DtTermCharAttributes rewritten as a peek-ahead loop
cde/lib/DtTerm/Term/TermData.h
- Pixel pal256[256] + Boolean pal256Allocated[256]
cde/lib/DtTerm/Term/TermColor.h
- _DtTermResolve256Pixel extern
cde/lib/DtTerm/Term/TermColor.c
- _DtTermCube6 cube-step table
- _DtTermColor256RGB (static helper)
- _DtTermResolve256Pixel (lazy allocation + cache)
- _DtTermColorDestroy frees the populated 256-palette slots
cde/lib/DtTerm/Term/TermEnhance.c
- _DtTermResolveColorPair (int) replaced with
_DtTermResolveFgPixel / _DtTermResolveBgPixel (Pixel)
- _DtTermEnhProc simplified
cde/lib/DtTerm/Term/TermFunction.h
- _DtTermSetColor extern
cde/lib/DtTerm/Term/TermFunction.c
- _DtTermSetColor (packed-enhValue setter)
Direct-RGB rendering (ENH_MODE_RGB) is wired through the parser but
the resolver still drops to the default for that mode -- the next
change adds visual detection and the masks-and-shifts pixel synthesis.
Wire up the rendering side of the direct-RGB SGR sub-sequences whose
parsing landed in the previous change. Pixel synthesis uses the X
visual's RGB masks on TrueColor / DirectColor displays, with no
XAllocColor round-trip and nothing to free at destroy time. On
visuals that can't represent arbitrary 24-bit colour, the request
rounds to the nearest of the 16 indexed defaults so the experience
degrades gracefully rather than failing or crashing.
Visual detection runs once per widget at _DtTermColorInit time and
caches the masks / shifts / bit-counts on DtTermData. The shift for
each channel is derived from the position of the lowest set bit in
the mask; the bit-count from popcount of the mask. This means the
synthesizer also handles non-default visuals such as 15-bit RGB555,
16-bit RGB565, and 30-bit deep-colour without special-casing.
The 8-bit-channel scaler maps cleanly in both directions:
bits >= 8 (e.g. 8 or 10): v << (bits - 8) << shift
bits < 8 (e.g. 5 or 6): v >> (8 - bits) << shift
So r=0xff on a 5-bit channel collapses to 0x1f (max), and on a
10-bit channel expands to 0x3fc (just below max). The result is then
ANDed with the channel's mask to guarantee no stray bits.
When the screen isn't TrueColor / DirectColor, the resolver picks the
nearest match from the 16 indexed defaults using straight Euclidean
distance in 8-bit RGB and routes through the existing
colorPairs[] / _DtTermColorInitializeColorPair path -- the same path
the SGR 30-37 / 90-97 sequences already use. No colormap exhaustion
risk, no separate pixel-tracking required.
The fg / bg resolvers in TermEnhance.c both gained an ENH_MODE_RGB
arm that decodes the packed 0xRRGGBB payload and hands off to
_DtTermResolveRGBPixel. The isBg flag is forwarded so the
nearest-of-16 fallback picks .fg.pixel for foreground requests and
.bg.pixel for background requests, matching what the indexed path
does.
Direct-RGB Pixels are not tracked on the destroy path: they were never
allocated, so there is nothing for XFreeColors to release. The
existing colorPairs[] and pal256 free loops are unchanged.
Files touched:
cde/lib/DtTerm/Term/TermData.h
- isTrueColor + red/green/blue mask, shift, bits triples
cde/lib/DtTerm/Term/TermColor.h
- _DtTermResolveRGBPixel extern
cde/lib/DtTerm/Term/TermColor.c
- _DtTermVisualLowBit, _DtTermVisualPopCount helpers
- _DtTermDetectVisual (one-shot at _DtTermColorInit)
- _DtTermScaleChannel (8-bit -> mask-width channel scaler)
- _DtTermNearestOf16 (Euclidean nearest indexed colour)
- _DtTermResolveRGBPixel (TrueColor synth + fallback)
cde/lib/DtTerm/Term/TermEnhance.c
- _DtTermResolveFgPixel / _DtTermResolveBgPixel ENH_MODE_RGB
arms
Ship two new terminfo entries alongside the existing dtterm one, and
document the new SGR codes recognised by the terminal emulator.
dtterm-256color colors#256, setaf/setab use SGR 30-37, 90-97, and
38;5;N depending on the requested index.
dtterm-direct use=dtterm-256color, RGB boolean, colors#0x1000000,
setrgbf/setrgbb emit the SGR 38;2;R;G;B / 48;2;R;G;B
direct-RGB sequences.
Both are opt-in -- TERM=dtterm continues to mean the historical 8-colour
behaviour. Users set TERM=dtterm-256color or TERM=dtterm-direct (the
latter implies the former) so terminfo-aware applications such as vim,
neovim, tmux, htop and less detect the extra capabilities.
Compilation runs through `tic -x` at install time so unknown extension
booleans (e.g. RGB on older ncurses) don't cause failures. The .ti
source is shipped verbatim alongside dtterm.ti via dist_cfg_DATA so
distributors who don't have tic at install time can still vendor the
file.
The dtterm(5) man page gets new entries for SGR 90-97 / 100-107 (bright
fg / bg), 38;5;N / 48;5;N (256-colour palette) and 38;2;R;G;B /
48;2;R;G;B (direct-RGB), plus a paragraph pointing readers at the new
TERM names.
Files touched:
cde/programs/dtterm/dtterm-extra.ti (new)
- dtterm-256color and dtterm-direct entries (use=dtterm and
use=dtterm-256color respectively)
cde/programs/dtterm/Makefile.am
- dist_cfg_DATA installs the new .ti
- install-exec-hook runs tic -x on it
cde/programs/dtterm/dtterm.5
- SGR 90-97, 100-107, 38;5;N, 48;5;N, 38;2;R;G;B, 48;2;R;G;B
- reference to the new TERM names
The original dtterm defaults for SGR 30-37 / 40-47 used full-intensity channels (0xffff) for every colour, which left no headroom for the SGR 90-97 / 100-107 bright variants -- bright-red rendered the same pixel as normal red, bright-green the same as normal green, and so on. The only visible distinctions were bright-black (grey) and bright-blue (#5c5cff); everything else looked identical to its plain counterpart, which made vim, htop and other apps that rely on bold-brightens-fg look washed out compared to the xterm rendering. Drop the 0/1 InitColor() macro for slots 1-8 and seed each slot with xterm's 'colour3' family RGB values instead: slot 1: black #000000 slot 2: red red3 #cd0000 slot 3: green green3 #00cd00 slot 4: yellow yellow3 #cdcd00 slot 5: blue blue3 #0000ee slot 6: magenta magenta3 #cd00cd slot 7: cyan cyan3 #00cdcd slot 8: white gray90 #e5e5e5 The bright variants in colorPairs[9..16] keep their full-intensity defaults, so the SGR 1 bold-brightens-fg promotion and the SGR 90-97 sequences now render distinctly brighter than the normal palette -- matching xterm's out-of-box appearance. Black stays at #000000 (any further dimming would shift it towards a grey that no terminal user expects). White drops to #e5e5e5 so the bright-white (#ffffff) variant is visibly different. This is a small visual tweak: applications continue to receive the same Pixel for the same SGR code, and any user who wants the previous full-intensity look can override via the *color0..*color15 resources once those land.
Expose the 16-entry ANSI colour palette and the xterm bold-brightens-fg toggle as standard Xt resources so users can theme dtterm via their .Xdefaults / app-defaults without rebuilding. *color0 .. *color7 SGR 30-37 / 40-47 fg / bg (dim ANSI) *color8 .. *color15 SGR 90-97 / 100-107 fg / bg (bright ANSI) *boldColors Boolean; True enables bold-brightens-fg Resource defaults match the dimmed palette this branch landed earlier (red3, green3, yellow3, blue3, magenta3, cyan3, gray90 for 0-7, and gray40 / pure primaries / #5c5cff / white for 8-15), so behaviour is unchanged out of the box. Implementation notes: DtTermPart (cde/include/Dt/TermP.h) gains 16 Pixel fields and one Boolean. The String -> Pixel converter is the standard XmRPixel / XtRString combination Xt already runs for *foreground / *background, so we don't ship a custom converter -- by the time _DtTermColorInit() runs, vt.colorN holds a usable Pixel on the widget's colormap. _DtTermColorInit() (TermColor.c) replaces the static RGB seeding loop with a copy from vt.color0..15 into colorPairs[1..16] and sets fgCommon = bgCommon = True for those slots. That skips the lazy XAllocColor in _DtTermColorInitializeColorPair (the resource subsystem already owns the Pixels) and lets the existing XQueryColor path derive the half-bright derivative from .pixel. _DtTermEnhProc (TermEnhance.c) consults tw->vt.boldColors before promoting an SGR 30-37 INDEXED fg to its 90-97 bright twin when SGR 1 (bold) is set. Setting *boldColors: False keeps the dim fg and renders bold via the bold font / overstrike path only. Documentation: cde/programs/dtterm/Dtterm.ad.src grows a commented block that lists the 16 resources with their default colour specs and explains the bright-vs-dim semantics. Names follow the xterm convention so a user moving from xterm can lift their existing palette config unchanged. Files touched: cde/include/Dt/Term.h - DtN / DtC string defines (17 new) cde/include/Dt/TermP.h - Pixel color0..15 + Boolean boldColors cde/lib/DtTerm/Term/Term.c - XtResource entries (17 new) cde/lib/DtTerm/Term/TermColor.c - resource Pixels into colorPairs cde/lib/DtTerm/Term/TermEnhance.c - bold-brighten gated by boldColors cde/programs/dtterm/Dtterm.ad.src - commented resource examples
The dtterm widget exports TERM=<termName> into the child shell's
environment (TermPrim.c:2539+) using the *termName resource. Until
now that resource defaulted to "dtterm", which advertises only 8
colours -- so curses-aware applications such as vim, neovim, tmux,
htop and less ran in 8-colour mode unless the user remembered to
'export TERM=dtterm-256color' themselves.
Flip the default to dtterm-256color now that the corresponding
terminfo entry ships with dtterm and is compiled at 'make install'
time via the existing 'tic -x dtterm-extra.ti' hook. dtterm-256color
uses use=dtterm, so every capability the old TERM=dtterm provided
remains available -- callers strictly gain colors#256 and the
multi-step setaf / setab encoding that targets SGR 30-37, 90-97 and
38;5;N depending on the requested index.
Users who deliberately want the 8-colour TERM keep that as a
one-line override:
Dtterm*termName: dtterm
Users who want to advertise 24-bit truecolour to truecolor-aware apps
(vim with ':set termguicolors', etc.) can opt in to the heaviest of
the three terminfo entries:
Dtterm*termName: dtterm-direct
Only DtTerm's resource default changes; DtTermPrimitive keeps its
base default of "dumb" since standalone DtTermPrimitive users (rare)
should set TERM explicitly.
Falling back when the terminfo database doesn't carry
dtterm-256color is left to the user: if /usr/share/terminfo/d (or
~/.terminfo/d) lacks the entry, curses applications start in dumb
mode and the user either runs 'tic -x cde/programs/dtterm/dtterm-extra.ti'
once or overrides termName back to "dtterm". Dtterm.ad.src grows a
commented block describing both paths.
Files touched:
cde/lib/DtTerm/Term/Term.c - default for DtNtermName
cde/programs/dtterm/Dtterm.ad.src - documentation comment
Recognise five new SGR families on the wire and stamp them onto the
per-cell enhancement record. The renderer changes that turn the new
attributes into pixels live in a follow-up; this commit is
behaviour-neutral on screen and only widens / wires storage:
SGR 21 doubly underlined
SGR 53 / 55 overline on / off
SGR 58;5;N indexed underline colour
SGR 58;2;R;G;B direct-RGB underline colour
SGR 59 default underline colour
SGR 73 / 74 superscript / subscript
SGR 75 super / subscript off
Existing SGR 24 (underline off) is extended to clear both UNDERLINE
and the new DOUBLE_UNDERLINE bit, matching xterm semantics.
The new state is recorded in the existing video bit-field (widened
from 6 to 10 bits) and in a brand-new enhUlColor slot on the cell.
The slot lives at the previously-reserved enum value 2 in dtEnhID --
NUM_ENHANCEMENT_FIELDS was already 6, so the values[] array always
had room.
Storage changes:
cde/lib/DtTerm/Term/TermBuffer.h - new bits + IS_ macros + enhUlColor
cde/lib/DtTerm/Term/TermBufferP.h - video:6 -> :10; uint32_t ulColor;
cde/lib/DtTerm/Term/TermData.h - enhVideoState widens; enhUlColorState
cde/lib/DtTerm/Term/TermBuffer.c - SET/GET/DIRTY/blankEnh cover ulColor
Parser / handler changes:
cde/lib/DtTerm/Term/TermFunction.{c,h} - _DtTermSetUlColor + new cases
cde/lib/DtTerm/Term/Term.c - saveCursor.enhUlColorState init
cde/lib/DtTerm/Term/TermParse.c - DECSC/DECRC ulColor + 58 peek
Nothing visible to the user yet -- the renderer still ignores
DOUBLE_UNDERLINE, OVERLINE, SUPERSCRIPT, SUBSCRIPT and ulFg. Adding
that lives in the next change.
Make the SGR sequences parsed in the previous commit actually visible.
TermEnhInfoRec gains a Pixel ulFg; four new TermENH_* flag bits live
alongside the existing SECURE / UNDERLINE / OVERSTRIKE flags.
TermEnhance.c populates info->ulFg from the per-cell enhUlColor slot
(defaulting to info->fg when the cell has ENH_MODE_DEFAULT) and lights
up the new flag bits from the cell's video bit-field.
The TermFontRenderFunction signature and _DtTermPrimRenderText grow a
Pixel ulFg parameter. Each renderer (FontRenderFunction,
FontSetRenderFunction, LineDrawRenderFunction) uses ulFg when drawing
underline / double-underline / overline lines and restores the GC
foreground to fg afterwards.
The selection-highlight fast-path in TermPrimRender.c and
TermPrimRenderMb.c (which bypasses the renderFunction and draws its
own underline) gets the same three-line block for under / double /
over using enhInfo.ulFg.
Superscript / subscript shift the XDrawImageString baseline by a
third of the cell height in FontRenderFunction. The bg rectangle
XDrawImageString paints moves with it, so a freshly-painted cell
stays consistent; cells repainted over a previously-coloured bg may
briefly show the prior bg above or below the shifted glyph until the
next full redraw (acceptable trade-off vs. doing a separate
XFillRectangle + XDrawString pre-fill).
Backward compatibility:
- ulFg defaults to fg when the cell has no SGR 58, so plain SGR 4
underline still renders in the foreground colour.
- DOUBLE_UNDERLINE and OVERLINE flags are zero in the absence of
SGR 21 / 53, so no extra lines are drawn for legacy content.
- SUPERSCRIPT / SUBSCRIPT are zero unless SGR 73 / 74 fires, so
the baseline doesn't shift for legacy content.
Files touched:
cde/lib/DtTerm/TermPrim/TermPrimBuffer.h
- TermENH_DOUBLE_UNDERLINE / OVERLINE / SUPER / SUB flags
- TermIS_* macros
- Pixel ulFg in TermEnhInfoRec
cde/lib/DtTerm/TermPrim/TermPrimRenderP.h
- TermFontRenderFunction grows a Pixel ulFg parameter
cde/lib/DtTerm/TermPrim/TermPrimRender.{h,c}
- _DtTermPrimRenderText grows a Pixel ulFg parameter
- selection-highlight fast-path uses ulFg, draws new decorations
- default-fill of enhInfo also seeds enhInfo.ulFg
cde/lib/DtTerm/TermPrim/TermPrimRenderMb.c
- same updates as Render.c for the multibyte path
cde/lib/DtTerm/TermPrim/TermPrimRender{Font,FontSet,LineDraw}.c
- signatures gain Pixel ulFg
- underline / double-underline / overline drawing uses ulFg
cde/lib/DtTerm/TermPrim/TermPrimRenderFont.c
- Y-shift for SUPERSCRIPT / SUBSCRIPT
cde/lib/DtTerm/Term/TermEnhance.c
- resolve ulFg from enhUlColor
- set the new TermENH_* flag bits
dtterm's CSI parser previously only treated ';' (0x3B) as a parameter separator; the ECMA-48 standard sub-parameter form using ':' (0x3A) fell through the default error handler and aborted the sequence. Modern terminals (kitty, wezterm, recent xterm) accept both, and neovim emits the colon form for SGR 38 / 48 / 58 by default when nvim_set_hl is used with truecolour highlights. Add ':' entries to the two CSI parse tables -- left_bracket_table (initial state after ESC [) and left_bracket_table_no_Q (subsequent parameter accumulation, after any digit) -- routing them identically to ';' via _DtTermParsePushNum. This makes the parameter stream ESC [ 3 8 : 5 : 208 m ESC [ 3 8 : 2 : 255 : 128 : 0 m ESC [ 5 8 : 5 : 196 m ESC [ 5 8 : 2 : 0 : 200 : 255 m land in context->parms[] as three / five-int sequences in the same shape the existing 38/48/58 peek-ahead loop in _DtTermCharAttributes already consumes for the ';' form, so the SGR handler doesn't need any change. Caveat for strict ECMA-48 sub-parameter semantics: The standard distinguishes ';' (parameter separator) from ':' (sub-parameter separator within one parameter). With this simple "treat colon like semicolon" approach, dtterm collapses both into a flat parameter list, so the SGR 38 / 48 / 58 sequences (which are the overwhelmingly common use of ':') work transparently. However, the rarer SGR 4:N form (where N is an underline-style sub-parameter: 2=double, 3=curly, 4=dotted, 5=dashed) would be parsed as two parameters '4' (UNDERLINE on) and 'N' (whatever SGR N means), not as "underline with style N". This matches what xterm did until fairly recently and is what most apps fall back to when their output is piped through a `;`-only terminal. Adding richer underline-style sub-parameter handling would need a sub-parameter list per parameter in the parser context, which is a bigger change deferred for now.
Add support for the xterm / DEC cursor-style sequence so apps such as
vim, neovim and tmux can switch the cursor between block, underline
and vertical bar to indicate insert / normal / replace mode.
Ps = 0, 1, 2 block (existing DtTERM_CHAR_CURSOR_BOX)
Ps = 3, 4 underline (existing DtTERM_CHAR_CURSOR_BAR)
Ps = 5, 6 vertical bar (new DtTERM_CHAR_CURSOR_VBAR)
The blinking-vs-steady distinction (odd / even Ps) is intentionally
ignored: dtterm keeps its existing cursor-blink resource handling so
users who turn off blink globally don't get it forced back on by an
application sequence.
Parser plumbing:
cde/lib/DtTerm/Term/TermParseTable.c
- new intermediate state _DtTermStateLeftBracketSpace
(CSI ... SP <byte>) with a single 'q' finalizer
- ' ' (0x20) entries in both left_bracket_table_no_Q and
left_bracket_table route into the new state so CSI
parameter accumulation followed by SP is recognised
cde/lib/DtTerm/Term/TermParseTable.h
- extern for the new _DtTermSetCursorStyle handler
cde/lib/DtTerm/Term/TermParse.c
- _DtTermSetCursorStyle reads parms[1] (default 1) and
XtVaSetValues the DtNcharCursorStyle resource so the
widget's regular set_values machinery handles the repaint
Renderer:
cde/include/Dt/Term.h
cde/include/Dt/TermPrim.h
- DtTERM_CHAR_CURSOR_VBAR = 3 alongside the existing
BOX (0), BAR (1, underline) and INVISIBLE (2)
cde/lib/DtTerm/TermPrim/TermPrimCursor.c
- cursorToggle replaces its BOX / not-BOX if-else with a
switch that draws:
BOX full cell (cellWidth x cellHeight)
BAR underline (cellWidth x 2 at ascent+1)
VBAR left-edge bar (2 x cellHeight)
INVISIBLE nothing this toggle
Implement xterm's OSC 10 / 11 / 12 dynamic colour control:
ESC ] 10 ; spec ST set default foreground colour
ESC ] 11 ; spec ST set default background colour
ESC ] 12 ; spec ST set cursor colour
ESC ] N ; ? ST query, respond \e]N;rgb:RRRR/GGGG/BBBB\e\\
`spec` is any string XParseColor accepts: rgb:RR/GG/BB, #RRGGBB, or a
named colour from rgb.txt.
Also fixes a pre-existing dispatch drop: when an OSC payload was
terminated with ST (ESC \\) instead of BEL (0x07), get_stringBS_table
called _DtTermPrimParserNextState (just reset the state) instead of
_DtTermChangeTextParam (the actual dispatcher). All BEL-terminated
OSC sequences worked, but ST-terminated ones were silently dropped --
which is what xterm-spawned applications often emit by default. The
fix is a one-entry change in get_stringBS_table so both terminators
route to the dispatcher.
Implementation:
cde/lib/DtTerm/Term/TermParseTable.c
- ST (`\\`) entry in get_stringBS_table now invokes
_DtTermChangeTextParam instead of _DtTermPrimParserNextState
cde/lib/DtTerm/Term/TermParse.c
- new static helper _DtTermOscDefaultColour parses '?' (query)
or any X colour spec (set) and either responds via
sendEscSequence or updates the widget
- cases 10 / 11 / 12 added to the OSC dispatcher switch
For SET:
- 10 / 11 push through XtVaSetValues XtNforeground / XtNbackground
so the widget's set_values machinery triggers a proper repaint
- colorPairs[0] is also updated so the cached default fg/bg matches
- 12 pokes tw->term.tpd->cursorGC directly via XSetForeground; the
cursor picks up the new colour on its next blink toggle
For QUERY:
- reads the current Pixel, runs XQueryColor for the RGB, and emits
rgb:RRRR/GGGG/BBBB with ST terminator -- the standard xterm
response format consumed by curses, vim, neovim and tmux
Implement xterm's OSC 4, the per-index palette control:
ESC ] 4 ; N ; ? ST query palette entry N
ESC ] 4 ; N ; spec ST set palette entry N
ESC ] 4 ; N1 ; spec1 ; N2 ; spec2 ; ... ST set multiple
`spec` is any string XParseColor accepts. N maps to dtterm's storage:
0..7 -> colorPairs[1..8] (dim ANSI palette)
8..15 -> colorPairs[9..16] (bright ANSI palette)
16..255 -> pal256[N] (lazy 256-colour cache)
For SET on a previously-owned slot, the old pixel is XFreeColors'd
before the new one is allocated so the colormap usage stays steady on
PseudoColor visuals; on TrueColor this is a no-op. After applying
the change the widget area is XClearArea'd with exposures = True so
the existing on-screen cells re-render with the new colour.
For QUERY, the response is the canonical xterm reply
ESC ] 4 ; N ; rgb:RRRR/GGGG/BBBB ESC \\
with ST terminator. Querying an entry that hasn't been lazily
allocated for the 256-palette cache forces allocation now, so a
follow-up render is consistent with what the application thinks the
palette holds.
Multi-pair payloads (the most common form in actual programs) work
because the helper walks the ';'-separated stringParms buffer with
strtok_r, handling each `N;spec` pair independently.
Files touched:
cde/lib/DtTerm/Term/TermParse.c
- new static helper _DtTermOscPalette
- case 4 added to _DtTermChangeTextParam
- <string.h> include for strtok_r if not already pulled in
Implement xterm's OSC 52 clipboard-set sequence:
ESC ] 52 ; sel ; base64-data ST
When the application emits OSC 52 with base64-encoded text, dtterm
decodes the payload and pushes it onto the Motif CLIPBOARD via the
same XmClipboardStartCopy / XmClipboardCopy / XmClipboardEndCopy path
the existing "Copy" menu uses.
The behaviour is gated behind a new boolean resource
Dtterm*allowClipboardOps: False (default)
so a malicious or compromised program can't silently replace the
user's clipboard. This matches xterm's `allowClipboardOps` default
and is what the man page (CVE-2014-9650 era guidance) recommends.
The clipboard *query* direction (`OSC 52 ; sel ; ? ST`) is
intentionally not implemented. Letting an arbitrary program read
back the user's clipboard contents is a confused-deputy hazard
(passwords, secrets pasted from password managers, etc.), and the
practical benefit is small -- callers that actually need round-trip
clipboard access typically use platform helpers like xclip / pbpaste
through their own piped IPC.
PRIMARY selection ('p') is mapped onto CLIPBOARD too because
dtterm's existing PRIMARY ownership is wired to the cell-level
selection machinery in TermPrimSelect.c, and re-owning PRIMARY for a
free-form OSC 52 payload would clobber that. Most applications that
care use 'c' (or no selection char at all, which defaults).
Files touched:
cde/include/Dt/Term.h - DtN/DtC defines for allowClipboardOps
cde/include/Dt/TermP.h - Boolean allowClipboardOps on DtTermPart
cde/lib/DtTerm/Term/Term.c
- resources[] entry, default False
cde/lib/DtTerm/Term/TermParse.c
- _DtTermB64Decode: small in-place base64 decoder
- _DtTermOscClipboard: gated handler that decodes and
XmClipboardCopy's the result
- case 52 in _DtTermChangeTextParam
Recognise the OSC 8 hyperlink sequence used by modern ls / git / gh /
fastfetch / browsers:
ESC ] 8 ; params ; URL ST start a hyperlink for subsequent cells
ESC ] 8 ; ; ST end the active hyperlink
The current URL is tracked on DtTermData (td->linkUrl, single-slot:
nested links are not modelled). While a URL is active, a new
LINK_ACTIVE bit is OR'd into the per-cell video bit-field, and the
renderer treats LINK_ACTIVE the same way it treats TermENH_UNDERLINE
so hyperlinked text is visibly underlined even when the application
doesn't emit SGR 4 itself.
Click-to-open is intentionally not implemented in this commit -- the
URL is just tracked. A follow-up change can hook a Button1 event
handler that looks up the URL of the clicked cell and invokes
xdg-open or the user's preferred protocol launcher.
The spec-reserved `params` field of OSC 8 (typically `id=...`) is
parsed but currently dropped; nothing dtterm renders depends on the
id grouping at this stage.
Storage / plumbing:
cde/lib/DtTerm/Term/TermBuffer.h
- LINK_ACTIVE (1 << 10) and IS_LINK_ACTIVE macro
- VIDEO_MASK extended to include the new bit
cde/lib/DtTerm/Term/TermBufferP.h
- DtTermEnhPart.video widens from :10 to :11
cde/lib/DtTerm/Term/TermData.h
- char *linkUrl on DtTermDataRec
cde/lib/DtTerm/Term/TermParse.c
- new _DtTermOscHyperlink helper splits "params;URL", manages
td->linkUrl, and OR/AND-s LINK_ACTIVE into td->enhVideoState
- case 8 added to _DtTermChangeTextParam
cde/lib/DtTerm/Term/TermEnhance.c
- _DtTermEnhProc OR-s TermENH_UNDERLINE when LINK_ACTIVE is set
on the cell, regardless of explicit SGR 4
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
To make dtterm a 1st class terminal
TODO:
Tier 1 — "you'll notice immediately"
- Currently when you run vim, htop, less, etc., they paint over your shell history and destroy it on exit.
- With alt-screen, full-screen apps switch to a secondary buffer and the original shell content is preserved when they quit.
- Effort: medium. Needs a second TermBuffer + DECSET branches. Files: TermParse.c (1049 / 1047 / 47 cases) + a buffer-swap function in TermBuffer.c.
- The stub case 1000: /* don't send mouse x and y */ has been sitting in TermParse.c:1822 waiting to be implemented for thirty years.
- Without it: tmux / vim / mc mouse modes don't work, no click-to-position-cursor in vim, no scrollwheel-paging in less.
- Effort: medium. Needs Xt event handlers on press/motion/release + protocol formatter for the four mouse modes (1006 SGR-encoding is the modern one).
- When enabled, dtterm wraps pasted text in \e[200~ … \e[201~ so shells / editors can distinguish a paste from typing.
- Mitigates "paste-bomb" attacks where pasted text containing a newline triggers immediate execution.
- Effort: small. One DECSET branch + a flag consulted in the paste / selection-insert path.
Tier 2 — "modern shell integrations want them"
URL id storage which is real work.
Tier 3 — "spec compliance / niche"
plus four new renderer line styles.
Tier 4 — major work, major payoff
visual quality jump is the biggest cosmetic change available.