Terminal should support libtickit style input

[zsimic]

Environment

Windows build number: 10.0.19042.0
Windows Terminal version (if applicable): 1.4.3243.0
WSL: ubuntu 20.04, with bash 5.0.17

Steps to reproduce

Open a new tab with a shell from ubuntu 20.04, run showkey -a, then hit backspace, then shift+backspace etc, WT sends:

• 0x7f for backspace
• 0x08 for ctrl+backspace
• 0x7f for shift+backspace
• 0x7f for ctrl+shift+backspace

(hit ctrl+d to end the showkey -a session)

Expected behavior

Expecting to be able to distinguish modifier + backspace key presses.

I like to make it so that ctrl+backspace deletes previous word, and ctrl+shift+backspace deletes everything preceding the cursor. I do so via ~/.inputrc, which works on most VT terminals:

"\e[127;5u": backward-kill-word     # ctrl+backspace
"\e[127;6u": backward-kill-line     # ctrl+shift+backspace
"\e[3;5~": kill-word                # ctrl+delete
"\e[3;6~": kill-line                # ctrl+shift+delete

I was able to make it work easily by adding this to my settings.json in WT:

{ "command": {"action": "sendInput", "input": "\u001b[127;2u" }, "keys": "shift+backspace" },
{ "command": {"action": "sendInput", "input": "\u001b[127;5u" }, "keys": "ctrl+backspace" },
{ "command": {"action": "sendInput", "input": "\u001b[127;6u" }, "keys": "ctrl+shift+backspace" },

Actual behavior

WT does not seem to distinguish modifier + backspace key presses (at least not in WSL).

[zsimic]

This may be helpful inspiration: https://iterm2.com/documentation-csiu.html

iTerm2 recently added a checkbox in their Preferences -> Profile -> Keys called "Report modifiers using CSI u".
I'm using that option, it makes everything simpler... no more need to worry about this or that weird historical keyboard handling story...

[zadjii-msft]

I'm just gonna re=purpose this as the general "Terminal should support CSI u input reporting". The weirdness here is that the Terminal is actually encoding it's input events in an even higher fidelity than CSI u would want, so that we can capture win32 input fully. Then, ConPTY is the thing responsible for sending the encoded input to the client. So I'm not sure that the Terminal could change the input mode of ConPTY, to force it to send CSI u formatted sequences. I'd think that the client would have to be responsible for asking for CSI u style input - in that case, ConPTY should definitely be able to respond correctly.

Links:

• Scenario: Improved keyboard handling #4999
• http://www.leonerd.org.uk/hacks/fixterms/

[j4james]

* http://www.leonerd.org.uk/hacks/fixterms/

Paul Evans (the author of that link), described it as "less a spec, and more of an inane rant". 😄 There was talk of creating a proper spec for it at some point in terminal-wg, but I don't think anything ever came of that. I do know that the current "spec" was considered problematic (it apparently isn't compatible with Xterm and breaks certain function keys in vim and emacs). Just FYI.

[zsimic]

CSI-u seemed to be like "as standard as it gets" :) But yeah, I'm not super well versed into what's actually most standard out there..

I did want to simply be able to distinguish backspace from shift+backspace etc...
And hopefully have a way of making it work across the stuff I use.

CSI-u + ~/.inputrc seemed like it could spare endless ultra-specific customization per terminal client...

Anyhow, as a user, all I really want is to be able to distinguish backspace from shift+backspace etc... in "as-standard-a-way as possible" :)

[cstrahan]

@j4james: [...] I do know that the current "spec" was considered problematic (it apparently isn't compatible with Xterm and breaks certain function keys in vim and emacs). [...]

I don't believe this is the case.

Edit: corrected attribution in the following paragraph.

My understanding is that support was added to xterm by @ThomasDickey (as proposed by @leonerd) in xterm patch #235 (released 2008/04/20):

add resource formatOtherKeys to provide an alternate escape sequence format for the modifyOtherKeys resource (request by Paul LeoNerd Evans).

To quote those two xterm resources docs:

       modifyOtherKeys (class ModifyOtherKeys)
               Like modifyCursorKeys, tells xterm to construct an escape
               sequence for ordinary (i.e., "other") keys (such as "2") when
               modified by Shift-, Control-, Alt- or Meta-modifiers.  This
               feature does not apply to special keys, i.e., cursor-, keypad-,
               function- or control-keys which are labeled on your keyboard.
               Those have key symbols which XKB identifies uniquely.

               For example, this feature does not apply to special control-
               keys (e.g., Escape, Tab, Enter, Backspace) Other control keys
               (e.g., Control-I, Control-M, Control-H) may send escape
               sequences when this feature is enabled.

               The default is "0":

               0    disables this feature.

               1    enables this feature for keys except for those with well-
                    known behavior, e.g., Tab, Backarrow and some special
                    control character cases which are built into the X11
                    library, e.g., Control-Space to make a NUL, or Control-3
                    to make an Escape character.

                    Except for those special cases built into the X11 library,
                    the Shift- and Control- modifiers are treated normally.
                    The Alt- and Meta- modifiers do not cause xterm to send
                    escape sequences.  Those modifier keys are interpreted
                    according to other resources, e.g., the metaSendsEscape
                    resource.

               2    enables this feature for keys including the exceptions
                    listed.  Xterm ignores the special cases built into the
                    X11 library.  Any shifted (modified) ordinary key sends an
                    escape sequence.  The Alt- and Meta- modifiers cause xterm
                    to send escape sequences.

               The Xterm FAQ has an extended discussion of this feature, with
               examples:

               https://invisible-island.net/xterm/modified-keys.html

       formatOtherKeys (class FormatOtherKeys)
               Overrides the format of the escape sequence used to report
               modified keys with the modifyOtherKeys resource.

               0  send modified keys as parameters for function-key 27
                  (default).

               1  send modified keys as parameters for CSI u.

So modifyOtherKeys (set to 1 or 2) enables an alternative encoding to disambiguate otherwise alternative key sequences, and formatOtherKeys: 1 instructs xterm to use CSI u in that alternative encoding (by default, xterm uses an earlier encoding like \E[27;modifier;code~).

See also: https://invisible-island.net/xterm/modified-keys.html

---

As @zsimic points out, iTerm2 (macOS terminal emulator, I'd guess used by 99.9% or more of macOS developers) added support (Jan 1, 2019): https://groups.google.com/g/iterm2-discuss/c/KpSlgnY_zxw?pli=1

That's documented here: https://iterm2.com/documentation-csiu.html

Support was added in vim v8.1.2134. You can read about the feature in :h modifyOtherKeys. A small excerpt:

Xterm and a few other terminals can be put in a mode where keys with modifiers
are sent with a special escape code.  Vim recognizes these codes and can then
make a difference between CTRL-H and Backspace, even when Backspace sends the
character 8.  And many more special keys.

For xterm modifyOtherKeys is enabled in the builtin termcap entry.  If this is
not used you can enable modifyOtherKeys with these lines in your vimrc: 
      let &t_TI = "\<Esc>[>4;2m"
      let &t_TE = "\<Esc>[>4;m"

Vim recognizes xterm's original CSI 27 encoding (as devised by @ThomasDickey et al) as well as the CSI u encoding (proposed by @leonerd): https://github.com/vim/vim/blob/6a0299d8f4c7a64c64d60a6bb39cfe6eaf892247/src/term.c#L4539-L4541

The tmux terminal multiplexer added support in tmux/tmux#2634.

The st terminal emulator has some patches to support this as well: https://st.suckless.org/patches/fix_keyboard_input/

[ThomasDickey]

leonerd's contribution was a description of the change; he has contributed no actual source-code to xterm.
The THANKS file in xterm's source lists the individuals who have contributed (even a single line) of source.

[cstrahan]

@ThomasDickey my apologies. I misread your

Thus, Evans provided the third implementation of CSI-u late in 2011 (after xterm in mid-2008, mintty in mid-2009).

as suggesting that this was his third implementation, after him having done xterm and mintty. Sleepy and not reading close enough.

I've amended my comment to correct that.