Graphical front-end (GUI-TUI Bridge)

[o-sdn-o]

Terminal users always enter the terminal environment from a graphical environment. The initial interface is always graphical. Until now, we have relied entirely on third-party graphical terminal emulators which significantly limit the capabilities of console applications without providing any way to achieve the required functionality. The only way to overcome these bottlenecks is to create our own bridge from the graphical to the terminal environment.

Our own graphical front-end will open up ways for us to solve the following issues:

• Implementation of full keyboard support. Keyboard-only access #86
• Displaying fragments of characters (halves), as well as subcell-sized text. Tested Terminals #72 (comment)
• Display of extended grapheme clusters in TUI applications.
• Support for pixel graphics. Pixel graphics in terminal #164
• Additional character attributes. Cell background interpolation as SGR attribute #165
• Full support for platform-specific clipboard. Text copied outside vtm cannot be pasted with a right click #273
• Get rid of the 16-color mode in older versions of Windows.
• Support for multiple text cursors and in the correct mode.
• Placing IME in the correct location next to the caret.
• Support for grid lines.

Character Matrix

1x1	2x2	3x1

Each character is a sequence of codepoints (one or more) - this is the so-called grapheme cluster. Using a font, this sequence is translated into a glyph run. The final scaling and rasterization of the glyph run is done into a rectangular matrix of terminal cells, defined either implicitly based on the Unicode properties of the cluster codepoints, or explicitly using a modifier codepoint from the Unicode codepoint range 0xD0000-0xD02A2, the value of which is encoded by the enumeration of “wh_xy” values:

• w: Character matrix width
• h: Character matrix height
• x: Horizontal fragment selector inside the matrix
• y: Vertical fragment selector inside the matrix

Users can explicitly specify the size of the character matrix (by zeroing _xy) or select any fragment (non-zero _xy) of it placing a modifier character from the Unicode codepoint range 0xD0000-0xD02A2 right after the base character (or at the end of the grapheme cluster).

For character matrices larger than 8x4, pixel graphics should be used.

Grapheme Cluster Boundaries

By default, grapheme clustering occurs according to Unicode UAX #29 https://www.unicode.org/reports/tr29/#Grapheme_Cluster_Boundary_Rules.

To set arbitrary boundaries, the C0 control character ASCII 0x02 STX is used, signaling the beginning of a grapheme cluster. The closing character of a grapheme cluster is always a codepoint from the range 0xD0000-0xDFFFF, which sets the dimension of the character matrix. All codepoints between STX and the closing codepoint that sets the matrix size will be included in the grapheme cluster.

Another brick in the wall

At present only standardized variation sequences with VS1, VS2, VS3, VS15 and VS16 have been defined; VS15 and VS16 are reserved to request that a character should be displayed as text or as an emoji) respectively.

VS4–VS14 (U+FE03–U+FE0D) are not used for any variation sequences

• https://en.wikipedia.org/wiki/Variation_Selectors_(Unicode_block)
• https://www.unicode.org/Public/UNIDATA/StandardizedVariants.txt
• https://www.unicode.org/reports/tr51/tr51-16.html#Direction

So, let's try to play this way:

Glyph run alignment (inside the matrix)

VS	Codepoint	Axis	Alignment
VS6	0xFE05	Horizontal	Left
VS7	0xFE06	Horizontal	Right
VS8	0xFE07	Vertical	Top
VS9	0xFE08	Vertical	Bottom

• We are not operating at a low enough level to support justified alignment.
• Center/Middle is the default alignment.

Glyph run rotation (inside the matrix)

VS	Codepoint	Function
VS10	0xFE09	Rotate 90° CCW
VS11	0xFE0A	Rotate 180° CCW
VS12	0xFE0B	Rotate 270° CCW
VS13	0xFE0C	Horizontal flip
VS14	0xFE0D	Vertical flip

# state - is a three bits integer.
VS10(state) = (state & 0b100) | ((state + 0b001) & 0b011)
VS11(state) = (state & 0b100) | ((state + 0b010) & 0b011)
VS12(state) = (state & 0b100) | ((state + 0b011) & 0b011)
VS13(state) = (state ^ 0b100) | ((state + (state & 1 ? 0 : 0b010)) & 0b011)
VS14(state) = (state ^ 0b100) | ((state + (state & 1 ? 0b010 : 0)) & 0b011)

angle = 90 * (state & 0b011)
hflip = state >> 2

[o-sdn-o]

You can play with a prototype of the vtm graphical window (gui-bridge branch) on various Windows platforms starting with Win8.1 (including Window Server 2019 Core).

If vtm is launched from a graphical shell, or through the start vtm.exe command, then it automatically turns on the graphical mode.

screen_record_Thu_04.18.2024__3-02-53.76.mp4

The vtm server starts in the background, despite the fact that its interface is not yet rendered.

[o-sdn-o]

[o-sdn-o]

8x4 character matrix support

screen_record_Mon_05.20.2024_18-27-05.90.mp4

[o-sdn-o]

Of course, along with the character matrix, support for both rotation (pi/2, pi, 3pi/2) and flip (hz/vt) is required.

[o-sdn-o]

At present only standardized variation sequences with VS1, VS2, VS3, VS15 and VS16 have been defined; VS15 and VS16 are reserved to request that a character should be displayed as text or as an emoji) respectively.

VS4–VS14 (U+FE03–U+FE0D) are not used for any variation sequences

• https://en.wikipedia.org/wiki/Variation_Selectors_(Unicode_block)
• https://www.unicode.org/Public/UNIDATA/StandardizedVariants.txt
• https://www.unicode.org/reports/tr51/tr51-16.html#Direction

Let's try to play this way:

VS	Codepoint	Function
VS10	0xFE09	Rotate 90° CCW
VS11	0xFE0A	Rotate 180° CCW
VS12	0xFE0B	Rotate 270° CCW
VS13	0xFE0C	Horizontal flip
VS14	0xFE0D	Vertical flip

# state - is a three bits integer.
VS10(state) = (state & 0b100) | ((state + 0b001) & 0b011)
VS11(state) = (state & 0b100) | ((state + 0b010) & 0b011)
VS12(state) = (state & 0b100) | ((state + 0b011) & 0b011)
VS13(state) = (state ^ 0b100) | ((state + (state & 1 ? 0 : 0b010)) & 0b011)
VS14(state) = (state ^ 0b100) | ((state + (state & 1 ? 0b010 : 0)) & 0b011)

angle = 90 * (state & 0b011)
hflip = state >> 2

[o-sdn-o]

Character rotation support

[o-sdn-o]

[o-sdn-o]

[o-sdn-o]

At present only standardized variation sequences with VS1, VS2, VS3, VS15 and VS16 have been defined; VS15 and VS16 are reserved to request that a character should be displayed as text or as an emoji) respectively.

VS4–VS14 (U+FE03–U+FE0D) are not used for any variation sequences

• https://en.wikipedia.org/wiki/Variation_Selectors_(Unicode_block)
• https://www.unicode.org/Public/UNIDATA/StandardizedVariants.txt
• https://www.unicode.org/reports/tr51/tr51-16.html#Direction

Another brick in the wall: Glyph run alignment (inside the matrix)

VS	Codepoint	Axis	Alignment
VS6	0xFE05	Horizontal	Left
VS7	0xFE06	Horizontal	Right
VS8	0xFE07	Vertical	Top
VS9	0xFE08	Vertical	Bottom

• We are not operating at a low enough level to support justified alignment.
• Center/Middle is the default alignment.