Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFC] New keyboard reporting protocol #3248

Closed
kovidgoyal opened this issue Jan 17, 2021 · 63 comments
Closed

[RFC] New keyboard reporting protocol #3248

kovidgoyal opened this issue Jan 17, 2021 · 63 comments

Comments

@kovidgoyal
Copy link
Owner

kovidgoyal commented Jan 17, 2021

Since the previous kitty keyboard reporting protocol was not well received, I have implemented a new one. The new proposal is backward compatible by default producing the exact same bytes as traditional terminals. Applications that want to enable more robust key handling can use progressive enhancement to request more sophisticated behavior. Notable features:

  1. Distinguish between press, repeat and release events. And the four modifiers ctrl, shift, alt, and super.
  2. Have a "game mode" where all key events generate escape codes rather than some generating text and some escape codes
  3. Disambiguate all key presses. Currently, many key presses are overlapping, generate the same bytes, the most egregious example being pressing Esc.
  4. Allow reporting the pressed key, the shifted key and an "alternate" key in the default keyboard layout for advanced shortcut matching
  5. Allow sending text encoded as unicode code points within the escape code rather than as separate UTF-8 bytes. This is useful for applications that want to customise text+key event handling.
  6. Add a way to query for support and current progressive enhancement level. Also allow saving and restoring current enhancement level on a stack.
  7. Make a canonical mapping of functional key names to PUA unicode characters.
  8. Support any key from any language that is present in Unicode.

The new protocol has been implemented in master and will be in the next kitty release. The spec for the protocol is: https://sw.kovidgoyal.net/kitty/keyboard-protocol.html

The new escape code is based on the proposal in http://www.leonerd.org.uk/hacks/fixterms/ however, it is more general and fixes various bugs in that proposal.

If there are bugs or things I have overlooked, now is the time to point them out. You can test the implementation in kitty (which is mostly complete, barring bugs) by building kitty from master and running

kitty +kitten key_demo

inside kitty, to see how key events are reported with full progressive enhancement.

@msokalski Please check if this meets your game use case

@gnachman I believe iTerm implements some version of fixterms, so you might be interested in the bugs I discovered in that proposal

@leonerd Please look over the list of bugs in fixterms and correct me if I am wrong about anything https://sw.kovidgoyal.net/kitty/keyboard-protocol.html#bugs-in-fixterms

In particular, there is some controversy about how to encode shift+key, for instance should ctrl+shift+a be encoded as CSI 97; 6 or CSI 65; 5 IMO the former is correct. One reports the actual key pressed, not the shifted form of the key. In my spec the shifted form is available as a sub-parameter.

A general note, I will not entertain bike-shedding about the escape code format. Unless there is a serious bug or concern that causes backward incompatibility/inoperability.

@msokalski
Copy link

I think there's a tiny typo in the spec. Event types, examples section, release event uses two colons, shouldn't it be semicolon after the key-code?

CSI key-code:modifier:3 # this is a release event

@kovidgoyal
Copy link
Owner Author

kovidgoyal commented Jan 17, 2021 via email

@msokalski
Copy link

msokalski commented Jan 17, 2021

I'm now confused with semicolons for other events, isn't modifier a sub-parameter in that case?

CSI key-code;modifier:1  # this is a press event
CSI key-code;modifier:2  # this is a repeat event```

@kovidgoyal
Copy link
Owner Author

Oh yeah sorry my mistake, that was indeed a typo, there is semi-colon between key-code and modifier. I misread your previous post. Now fixed.

@browser-bug
Copy link

browser-bug commented Jan 20, 2021

My keyboard specs:

rules:      evdev
model:      pc105
layout:     it
Trying to build keymap using the following components:
keycodes:   evdev+aliases(qwerty)
types:      complete
compat:     complete
symbols:    pc+it+inet(evdev)
geometry:   pc(pc105)
xkb_keymap {
	xkb_keycodes  { include "evdev+aliases(qwerty)"	};
	xkb_types     { include "complete"	};
	xkb_compat    { include "complete"	};
	xkb_symbols   { include "pc+it+inet(evdev)"	};
	xkb_geometry  { include "pc(pc105)"	};
};

When I'm typing these combinations in terminal I get the following prints:

Keys                                Output          Expected
Shift+3 PRESS                       <ffffffff>      £
CSI 51 : 163 ; 2 u
Shifted key: £ 

Shift+è PRESS                       2;2u            é
CSI 232 : 233 : 91 ; 2 u
Shifted key: é Alternate key: [

Shift+ò PRESS                       2;2u            ç
CSI 242 : 231 : 59 ; 2 u
Shifted key: ç Alternate key: ;

Shift+à PRESS                       4;2u            °
CSI 224 : 176 : 39 ; 2 u
Shifted key: ° Alternate key: '

Shift+ù PRESS                       9;2u            §
CSI 249 : 167 : 92 ; 2 u
Shifted key: § Alternate key: \

Shift+ PRESS                        ;2u             Nothing
CSI 0 :  : 57433 ; 2 u
Alternate key: RIGHT_ALT

Tested with an empty config file. Is this normal or I'm missing something? The last one I showed is particularly annoying because inside nvim it gets interpreted as <space>.

@kovidgoyal
Copy link
Owner Author

@browser-bug Can you run kitty as kitty --debug-keyboard and post the output when pressing the problem keys. From your description it looks like those keys are not generating text on your system.

@browser-bug
Copy link

browser-bug commented Jan 20, 2021

@kovidgoyal Sure thing. Following the order in which I posted.

Loading new XKB keymaps
Press xkb_keycode: 0x32 clean_sym: Shift_L composed_sym: Shift_L mods: none glfw_key: 57427 (LEFT_SHIFT) xkb_key: 65505 (Shift_L)
on_key_input: glfw key: 57427 native_code: 0xffe1 action: PRESS mods: 0x0 text: '' state: 0 ignoring as keyboard mode does not allow repeat events
Press xkb_keycode: 0xc clean_sym: 3 composed_sym: sterling text: £ mods: shift glfw_key: 51 (3) xkb_key: 51 (3) shifted_key: 163 (£)
on_key_input: glfw key: 51 native_code: 0x33 action: PRESS mods: 0x1 text: '£' state: 0 sent key to child
Release xkb_keycode: 0xc clean_sym: 3 mods: shift glfw_key: 51 (3) xkb_key: 51 (3) shifted_key: 163 (£)
on_key_input: glfw key: 51 native_code: 0x33 action: RELEASE mods: 0x1 text: '' state: 0 ignoring as keyboard mode does not allow release events
Release xkb_keycode: 0x32 clean_sym: Shift_L mods: shift glfw_key: 57427 (LEFT_SHIFT) xkb_key: 65505 (Shift_L)
on_key_input: glfw key: 57427 native_code: 0xffe1 action: RELEASE mods: 0x1 text: '' state: 0 ignoring as keyboard mode does not allow release events

Loading new XKB keymaps
Press xkb_keycode: 0x32 clean_sym: Shift_L composed_sym: Shift_L mods: none glfw_key: 57427 (LEFT_SHIFT) xkb_key: 65505 (Shift_L)
on_key_input: glfw key: 57427 native_code: 0xffe1 action: PRESS mods: 0x0 text: '' state: 0 ignoring as keyboard mode does not allow repeat events
Press xkb_keycode: 0xc clean_sym: 3 composed_sym: sterling text: £ mods: shift glfw_key: 51 (3) xkb_key: 51 (3) shifted_key: 163 (£)
on_key_input: glfw key: 51 native_code: 0x33 action: PRESS mods: 0x1 text: '£' state: 0 sent key to child
Release xkb_keycode: 0xc clean_sym: 3 mods: shift glfw_key: 51 (3) xkb_key: 51 (3) shifted_key: 163 (£)
on_key_input: glfw key: 51 native_code: 0x33 action: RELEASE mods: 0x1 text: '' state: 0 ignoring as keyboard mode does not allow release events
Release xkb_keycode: 0x32 clean_sym: Shift_L mods: shift glfw_key: 57427 (LEFT_SHIFT) xkb_key: 65505 (Shift_L)
on_key_input: glfw key: 57427 native_code: 0xffe1 action: RELEASE mods: 0x1 text: '' state: 0 ignoring as keyboard mode does not allow release events

Loading new XKB keymaps
Press xkb_keycode: 0x32 clean_sym: Shift_L composed_sym: Shift_L mods: none glfw_key: 57427 (LEFT_SHIFT) xkb_key: 65505 (Shift_L)
on_key_input: glfw key: 57427 native_code: 0xffe1 action: PRESS mods: 0x0 text: '' state: 0 ignoring as keyboard mode does not allow repeat events
Press xkb_keycode: 0xc clean_sym: 3 composed_sym: sterling text: £ mods: shift glfw_key: 51 (3) xkb_key: 51 (3) shifted_key: 163 (£)
on_key_input: glfw key: 51 native_code: 0x33 action: PRESS mods: 0x1 text: '£' state: 0 sent key to child
Release xkb_keycode: 0xc clean_sym: 3 mods: shift glfw_key: 51 (3) xkb_key: 51 (3) shifted_key: 163 (£)
on_key_input: glfw key: 51 native_code: 0x33 action: RELEASE mods: 0x1 text: '' state: 0 ignoring as keyboard mode does not allow release events
Release xkb_keycode: 0x32 clean_sym: Shift_L mods: shift glfw_key: 57427 (LEFT_SHIFT) xkb_key: 65505 (Shift_L)
on_key_input: glfw key: 57427 native_code: 0xffe1 action: RELEASE mods: 0x1 text: '' state: 0 ignoring as keyboard mode does not allow release events

Press xkb_keycode: 0x32 clean_sym: Shift_L composed_sym: Shift_L mods: none glfw_key: 57427 (LEFT_SHIFT) xkb_key: 65505 (Shift_L)
on_key_input: glfw key: 57427 native_code: 0xffe1 action: PRESS mods: 0x0 text: '' state: 0 ignoring as keyboard mode does not allow repeat events
Press xkb_keycode: 0x30 clean_sym: agrave composed_sym: degree text: ° mods: shift glfw_key: 224 (à) xkb_key: 224 (agrave) shifted_key: 176 (°) alternate_key: 39 (')
on_key_input: glfw key: 224 native_code: 0xe0 action: PRESS mods: 0x1 text: '°' state: 0 sent key to child
Release xkb_keycode: 0x30 clean_sym: agrave mods: shift glfw_key: 224 (à) xkb_key: 224 (agrave) shifted_key: 176 (°) alternate_key: 39 (')
on_key_input: glfw key: 224 native_code: 0xe0 action: RELEASE mods: 0x1 text: '' state: 0 ignoring as keyboard mode does not allow release events
Release xkb_keycode: 0x32 clean_sym: Shift_L mods: shift glfw_key: 57427 (LEFT_SHIFT) xkb_key: 65505 (Shift_L)
on_key_input: glfw key: 57427 native_code: 0xffe1 action: RELEASE mods: 0x1 text: '' state: 0 ignoring as keyboard mode does not allow release events

Press xkb_keycode: 0x32 clean_sym: Shift_L composed_sym: Shift_L mods: none glfw_key: 57427 (LEFT_SHIFT) xkb_key: 65505 (Shift_L)
on_key_input: glfw key: 57427 native_code: 0xffe1 action: PRESS mods: 0x0 text: '' state: 0 ignoring as keyboard mode does not allow repeat events
Press xkb_keycode: 0x33 clean_sym: ugrave composed_sym: section text: § mods: shift glfw_key: 249 (ù) xkb_key: 249 (ugrave) shifted_key: 167 (§) alternate_key: 92 (\)
on_key_input: glfw key: 249 native_code: 0xf9 action: PRESS mods: 0x1 text: '§' state: 0 sent key to child
Release xkb_keycode: 0x33 clean_sym: ugrave mods: shift glfw_key: 249 (ù) xkb_key: 249 (ugrave) shifted_key: 167 (§) alternate_key: 92 (\)
on_key_input: glfw key: 249 native_code: 0xf9 action: RELEASE mods: 0x1 text: '' state: 0 ignoring as keyboard mode does not allow release events
Release xkb_keycode: 0x32 clean_sym: Shift_L mods: shift glfw_key: 57427 (LEFT_SHIFT) xkb_key: 65505 (Shift_L)
on_key_input: glfw key: 57427 native_code: 0xffe1 action: RELEASE mods: 0x1 text: '' state: 0 ignoring as keyboard mode does not allow release events

Press xkb_keycode: 0x32 clean_sym: Shift_L composed_sym: Shift_L mods: none glfw_key: 57427 (LEFT_SHIFT) xkb_key: 65505 (Shift_L)
on_key_input: glfw key: 57427 native_code: 0xffe1 action: PRESS mods: 0x0 text: '' state: 0 ignoring as keyboard mode does not allow repeat events
Press xkb_keycode: 0x6c clean_sym: ISO_Level3_Shift composed_sym: ISO_Level3_Shift mods: shift glfw_key: 0 (UNKNOWN) xkb_key: 65027 (ISO_Level3_Shift) alternate_key: 57433 (RIGHT_ALT)
on_key_input: glfw key: 0 native_code: 0xfe03 action: PRESS mods: 0x1 text: '' state: 0 sent key to child
Release xkb_keycode: 0x6c clean_sym: ISO_Level3_Shift mods: shift glfw_key: 0 (UNKNOWN) xkb_key: 65027 (ISO_Level3_Shift) alternate_key: 57433 (RIGHT_ALT)
on_key_input: glfw key: 0 native_code: 0xfe03 action: RELEASE mods: 0x1 text: '' state: 0 ignoring release event for previous press that was handled as shortcutRelease xkb_keycode: 0x32 clean_sym: Shift_L mods: shift glfw_key: 57427 (LEFT_SHIFT) xkb_key: 65505 (Shift_L)
on_key_input: glfw key: 57427 native_code: 0xffe1 action: RELEASE mods: 0x1 text: '' state: 0 ignoring as keyboard mode does not allow release events

@kovidgoyal
Copy link
Owner Author

Should be fixed by my latest commit

@browser-bug
Copy link

browser-bug commented Jan 20, 2021

@kovidgoyal now <backspace> <enter>, <tab> and LEFT_CTRL/RIGHT_CTRL don't work on my end.

@kovidgoyal
Copy link
Owner Author

enter and backspace were a typo, now fixed. What does left_ctrl and
right_ctrl not work mean?

@browser-bug
Copy link

@kovidgoyal it was connected to the typo since any combination of L_CTRL/R_CTRL + wasn't sending any command. Now everything works.

@browser-bug
Copy link

browser-bug commented Jan 22, 2021

I'm signaling others unexpected key combs:

Keys                                Output          Expected
Shift+MENU PRESS                    363;2u          $
CSI 57363 ; 2 u

Shift+ENTER PRESS                      ;2u          <enter>
CSI 13 ; 2 u

Shift+BACKSPACE PRESS                 7;2u          <del>  
CSI 127 ; 2 u

Shift+PAGE_UP PRESS                     2~          Nothing
CSI 5 ; 2 ~

Shift+PAGE_DOWN PRESS                   2~          Nothing
CSI 6 ; 2 ~

Ctrl+MENU PRESS                     363;5u          ^
CSI 57363 ; 5 u

Ctrl+PAGE_UP PRESS                      5~          Nothing
CSI 5 ; 5 ~

Ctrl+PAGE_DOWN PRESS                    5~          Nothing
CSI 6 ; 5 ~
Loading new XKB keymaps
Press xkb_keycode: 0x1d clean_sym: y composed_sym: y text: y mods: none glfw_key: 121 (y) xkb_key: 121 (y)
on_key_input: glfw key: 121 native_code: 0x79 action: PRESS mods: 0x0 text: 'y' state: 0 sent text to child
Release xkb_keycode: 0x1d clean_sym: y mods: none glfw_key: 121 (y) xkb_key: 121 (y)
on_key_input: glfw key: 121 native_code: 0x79 action: RELEASE mods: 0x0 text: '' state: 0 ignoring as keyboard mode does not allow release events
Press xkb_keycode: 0x85 clean_sym: Super_L composed_sym: Super_L mods: none glfw_key: 57443 (LEFT_SUPER) xkb_key: 65515 (Super_L)
on_key_input: glfw key: 57443 native_code: 0xffeb action: PRESS mods: 0x0 text: '' state: 0 ignoring as keyboard mode does not allow repeat events
bernardo@MS-7821 ➜  ~ kitty --debug-keyboard
Loading new XKB keymaps
Press xkb_keycode: 0x32 clean_sym: Shift_L composed_sym: Shift_L mods: none glfw_key: 57440 (LEFT_SHIFT) xkb_key: 65505 (Shift_L)
on_key_input: glfw key: 57440 native_code: 0xffe1 action: PRESS mods: 0x0 text: '' state: 0 ignoring as keyboard mode does not allow repeat events
Press xkb_keycode: 0x87 clean_sym: Menu composed_sym: Menu mods: shift glfw_key: 57363 (MENU) xkb_key: 65383 (Menu)
on_key_input: glfw key: 57363 native_code: 0xff67 action: PRESS mods: 0x1 text: '' state: 0 sent key to child
Release xkb_keycode: 0x87 clean_sym: Menu mods: shift glfw_key: 57363 (MENU) xkb_key: 65383 (Menu)
on_key_input: glfw key: 57363 native_code: 0xff67 action: RELEASE mods: 0x1 text: '' state: 0 ignoring as keyboard mode does not allow release events
Release xkb_keycode: 0x32 clean_sym: Shift_L mods: shift glfw_key: 57440 (LEFT_SHIFT) xkb_key: 65505 (Shift_L)
on_key_input: glfw key: 57440 native_code: 0xffe1 action: RELEASE mods: 0x1 text: '' state: 0 ignoring as keyboard mode does not allow release events
Press xkb_keycode: 0x32 clean_sym: Shift_L composed_sym: Shift_L mods: none glfw_key: 57440 (LEFT_SHIFT) xkb_key: 65505 (Shift_L)
on_key_input: glfw key: 57440 native_code: 0xffe1 action: PRESS mods: 0x0 text: '' state: 0 ignoring as keyboard mode does not allow repeat events
Press xkb_keycode: 0x24 clean_sym: Return composed_sym: Return mods: shift glfw_key: 57345 (ENTER) xkb_key: 65293 (Return)
on_key_input: glfw key: 57345 native_code: 0xff0d action: PRESS mods: 0x1 text: '' state: 0 sent key to child
Release xkb_keycode: 0x24 clean_sym: Return mods: shift glfw_key: 57345 (ENTER) xkb_key: 65293 (Return)
on_key_input: glfw key: 57345 native_code: 0xff0d action: RELEASE mods: 0x1 text: '' state: 0 ignoring as keyboard mode does not allow release events
Release xkb_keycode: 0x32 clean_sym: Shift_L mods: shift glfw_key: 57440 (LEFT_SHIFT) xkb_key: 65505 (Shift_L)
on_key_input: glfw key: 57440 native_code: 0xffe1 action: RELEASE mods: 0x1 text: '' state: 0 ignoring as keyboard mode does not allow release events
Press xkb_keycode: 0x32 clean_sym: Shift_L composed_sym: Shift_L mods: none glfw_key: 57440 (LEFT_SHIFT) xkb_key: 65505 (Shift_L)
on_key_input: glfw key: 57440 native_code: 0xffe1 action: PRESS mods: 0x0 text: '' state: 0 ignoring as keyboard mode does not allow repeat events
Press xkb_keycode: 0x16 clean_sym: BackSpace composed_sym: BackSpace mods: shift glfw_key: 57347 (BACKSPACE) xkb_key: 65288 (BackSpace)
on_key_input: glfw key: 57347 native_code: 0xff08 action: PRESS mods: 0x1 text: '' state: 0 sent key to child
Release xkb_keycode: 0x16 clean_sym: BackSpace mods: shift glfw_key: 57347 (BACKSPACE) xkb_key: 65288 (BackSpace)
on_key_input: glfw key: 57347 native_code: 0xff08 action: RELEASE mods: 0x1 text: '' state: 0 ignoring as keyboard mode does not allow release events
Release xkb_keycode: 0x32 clean_sym: Shift_L mods: shift glfw_key: 57440 (LEFT_SHIFT) xkb_key: 65505 (Shift_L)
on_key_input: glfw key: 57440 native_code: 0xffe1 action: RELEASE mods: 0x1 text: '' state: 0 ignoring as keyboard mode does not allow release events
Press xkb_keycode: 0x32 clean_sym: Shift_L composed_sym: Shift_L mods: none glfw_key: 57440 (LEFT_SHIFT) xkb_key: 65505 (Shift_L)
on_key_input: glfw key: 57440 native_code: 0xffe1 action: PRESS mods: 0x0 text: '' state: 0 ignoring as keyboard mode does not allow repeat events
Press xkb_keycode: 0x70 clean_sym: Prior composed_sym: Prior mods: shift glfw_key: 57354 (PAGE_UP) xkb_key: 65365 (Prior)
on_key_input: glfw key: 57354 native_code: 0xff55 action: PRESS mods: 0x1 text: '' state: 0 sent key to child
Release xkb_keycode: 0x70 clean_sym: Prior mods: shift glfw_key: 57354 (PAGE_UP) xkb_key: 65365 (Prior)
on_key_input: glfw key: 57354 native_code: 0xff55 action: RELEASE mods: 0x1 text: '' state: 0 ignoring as keyboard mode does not allow release events
Release xkb_keycode: 0x32 clean_sym: Shift_L mods: shift glfw_key: 57440 (LEFT_SHIFT) xkb_key: 65505 (Shift_L)
on_key_input: glfw key: 57440 native_code: 0xffe1 action: RELEASE mods: 0x1 text: '' state: 0 ignoring as keyboard mode does not allow release events
Press xkb_keycode: 0x32 clean_sym: Shift_L composed_sym: Shift_L mods: none glfw_key: 57440 (LEFT_SHIFT) xkb_key: 65505 (Shift_L)
on_key_input: glfw key: 57440 native_code: 0xffe1 action: PRESS mods: 0x0 text: '' state: 0 ignoring as keyboard mode does not allow repeat events
Press xkb_keycode: 0x75 clean_sym: Next composed_sym: Next mods: shift glfw_key: 57355 (PAGE_DOWN) xkb_key: 65366 (Next)
on_key_input: glfw key: 57355 native_code: 0xff56 action: PRESS mods: 0x1 text: '' state: 0 sent key to child
Release xkb_keycode: 0x75 clean_sym: Next mods: shift glfw_key: 57355 (PAGE_DOWN) xkb_key: 65366 (Next)
on_key_input: glfw key: 57355 native_code: 0xff56 action: RELEASE mods: 0x1 text: '' state: 0 ignoring as keyboard mode does not allow release events
Release xkb_keycode: 0x32 clean_sym: Shift_L mods: shift glfw_key: 57440 (LEFT_SHIFT) xkb_key: 65505 (Shift_L)
on_key_input: glfw key: 57440 native_code: 0xffe1 action: RELEASE mods: 0x1 text: '' state: 0 ignoring as keyboard mode does not allow release events
Press xkb_keycode: 0x25 clean_sym: Control_L composed_sym: Control_L mods: none glfw_key: 57441 (LEFT_CONTROL) xkb_key: 65507 (Control_L)
on_key_input: glfw key: 57441 native_code: 0xffe3 action: PRESS mods: 0x0 text: '' state: 0 ignoring as keyboard mode does not allow repeat events
Press xkb_keycode: 0x87 clean_sym: Menu composed_sym: Menu mods: ctrl glfw_key: 57363 (MENU) xkb_key: 65383 (Menu)
on_key_input: glfw key: 57363 native_code: 0xff67 action: PRESS mods: 0x4 text: '' state: 0 sent key to child
Release xkb_keycode: 0x87 clean_sym: Menu mods: ctrl glfw_key: 57363 (MENU) xkb_key: 65383 (Menu)
on_key_input: glfw key: 57363 native_code: 0xff67 action: RELEASE mods: 0x4 text: '' state: 0 ignoring as keyboard mode does not allow release events
Release xkb_keycode: 0x25 clean_sym: Control_L mods: ctrl glfw_key: 57441 (LEFT_CONTROL) xkb_key: 65507 (Control_L)
on_key_input: glfw key: 57441 native_code: 0xffe3 action: RELEASE mods: 0x4 text: '' state: 0 ignoring as keyboard mode does not allow release events
Press xkb_keycode: 0x25 clean_sym: Control_L composed_sym: Control_L mods: none glfw_key: 57441 (LEFT_CONTROL) xkb_key: 65507 (Control_L)
on_key_input: glfw key: 57441 native_code: 0xffe3 action: PRESS mods: 0x0 text: '' state: 0 ignoring as keyboard mode does not allow repeat events
Press xkb_keycode: 0x70 clean_sym: Prior composed_sym: Prior mods: ctrl glfw_key: 57354 (PAGE_UP) xkb_key: 65365 (Prior)
on_key_input: glfw key: 57354 native_code: 0xff55 action: PRESS mods: 0x4 text: '' state: 0 sent key to child
Release xkb_keycode: 0x70 clean_sym: Prior mods: ctrl glfw_key: 57354 (PAGE_UP) xkb_key: 65365 (Prior)
on_key_input: glfw key: 57354 native_code: 0xff55 action: RELEASE mods: 0x4 text: '' state: 0 ignoring as keyboard mode does not allow release events
Release xkb_keycode: 0x25 clean_sym: Control_L mods: ctrl glfw_key: 57441 (LEFT_CONTROL) xkb_key: 65507 (Control_L)
on_key_input: glfw key: 57441 native_code: 0xffe3 action: RELEASE mods: 0x4 text: '' state: 0 ignoring as keyboard mode does not allow release events
Press xkb_keycode: 0x25 clean_sym: Control_L composed_sym: Control_L mods: none glfw_key: 57441 (LEFT_CONTROL) xkb_key: 65507 (Control_L)
on_key_input: glfw key: 57441 native_code: 0xffe3 action: PRESS mods: 0x0 text: '' state: 0 ignoring as keyboard mode does not allow repeat events
Press xkb_keycode: 0x75 clean_sym: Next composed_sym: Next mods: ctrl glfw_key: 57355 (PAGE_DOWN) xkb_key: 65366 (Next)
on_key_input: glfw key: 57355 native_code: 0xff56 action: PRESS mods: 0x4 text: '' state: 0 sent key to child
Release xkb_keycode: 0x75 clean_sym: Next mods: ctrl glfw_key: 57355 (PAGE_DOWN) xkb_key: 65366 (Next)
on_key_input: glfw key: 57355 native_code: 0xff56 action: RELEASE mods: 0x4 text: '' state: 0 ignoring as keyboard mode does not allow release events
Release xkb_keycode: 0x25 clean_sym: Control_L mods: ctrl glfw_key: 57441 (LEFT_CONTROL) xkb_key: 65507 (Control_L)
on_key_input: glfw key: 57441 native_code: 0xffe3 action: RELEASE mods: 0x4 text: '' state: 0 ignoring as keyboard mode does not allow release events

@kovidgoyal
Copy link
Owner Author

Those are expected, keys that produce no output normally now are encoded
so they can be used in terminal programs that dont want to adopt the
full protocol.

@craigbarnes
Copy link
Contributor

craigbarnes commented Jan 26, 2021

@kovidgoyal I've read through the docs and I'm really liking this proposal so far. It's a huge improvement over the previous protocol, in my opinion, and seems to solve pretty much every problem I've encountered w.r.t. terminal key codes. I'm planning to start working on a patch to add this protocol to the foot terminal in the near future.

However, if you'll forgive me for bikeshedding just a little, my only minor criticism would be the event type sub-param:

No, that is a sub-parameter. Parameters are separated by semi-colons and sub-parameters by colons. This is for example how, how the CSI for underline styles, work.

I agree that sub-params make perfect sense for underline styles (and RGB colors) in SGR sequences, but I can't really see how they're appropriate for key event types. It seems to me that all 3 params in this case are logically just parameters of the CSI u control function and the event type and modifiers are no more closely related than the modifiers and the key are.

I hope this isn't too much of a nitpick, but it was the only thing that struck me as somewhat odd when reading through the docs. Thank you for putting this protocol together and documenting it so well!

@kovidgoyal
Copy link
Owner Author

kovidgoyal commented Jan 26, 2021 via email

@craigbarnes
Copy link
Contributor

craigbarnes commented Jan 29, 2021

  1. I find thinking and parsing and generating grouped sets of parameters easier. Without it it means lots of empty ;;;; sections.

In practice, I guess most terminals wouldn't emit more than 1 key event as a single sequence, but I see your point.

It makes the simplest form of the escape code backwards compatible with the fixterms proposal and the legacy CSI~ encoding. Without using subparameters, you would either have to make the first two parameters key code and modifier or break backwards compatibility. The former means related things are no longer together, the latter is not desirable for obvious reasons.

Yeah, that makes sense from a backwards compatibility perspective. It just seems a little strange to see the event type as a sub-param, but not the modifiers. I don't have a particularly strong opinion either way though. Agreeing on a common protocol with a few minor back compat. artifacts seems a lot better than mindless nitpicking. The rest of the protocol also seems very well thought out, so I'll just trust your judgement and implement it as-is.

@kovidgoyal
Copy link
Owner Author

kovidgoyal commented Jan 30, 2021

Just to be clear you can have only one key event per CSIu escape code. The empty ;;;; sections when not using subgroups are caused by the fact that one there could be data that needs to be encoded in a later field of a single event while earlier fields are empty (default valued). For example, CSI keycode;;shifted-keycode;;u where the empty fields are default valued such as CSI keycode;modifier;shifted-keycode;alternate-keycode;event-type;...u

@mawww
Copy link

mawww commented Feb 2, 2021

Gave it a go in Kakoune, almost works out of the box after manually enabling it with nop %sh{ printf '\033[>1u' > /dev/tty }, it did trigger a bug Kakoune side (escape coming in as CSI 27 u was not recognized), but it work nicely after fixing that.

I think having an opt-in escape sequence to enable/disable this is the way to go, as shells tend to get confused with CSI u encoding (as I could see when enabling it in the settings on iterm2). I'd be happy to make Kakoune emit CSI > 1 u on terminal setup provided this does not confuse other terminals (I'd expect them to ignore it gracefully as its a private sequence).

Out of curiosity, what is the reason you did not go with the more classic CSI ? <number> h and CSI ? <number> l for enabling disabling ?

@kovidgoyal
Copy link
Owner Author

kovidgoyal commented Feb 2, 2021 via email

@craigbarnes
Copy link
Contributor

craigbarnes commented Feb 2, 2021

Just to be clear you can have only one key event per CSIu escape code. The empty ;;;; sections when not using subgroups are caused by the fact that one there could be data that needs to be encoded in a later field of a single event while earlier fields are empty (default valued). For example, CSI keycode;;shifted-keycode;;u where the empty fields are default valued such as CSI keycode;modifier;shifted-keycode;alternate-keycode;event-type;...u

Ah, I see what you mean now. My main concern with the sub-params is that it seems somewhat opaque as to why they're being used, even after reading the documentation. It's also the first time I've seen them used in sequences sent from terminal to client.

What do you think about using different final bytes to indicate the different event types (e.g. v=repeat, w=release) or, alternatively, using an intermediate byte?

@kovidgoyal
Copy link
Owner Author

kovidgoyal commented Feb 2, 2021 via email

@rien333
Copy link

rien333 commented Feb 11, 2021

Thanks for all your work, should be useful. Continuing from my earlier issue (#3285), I do hope that terminal programs (shells, text editors, etc.) will adopt an intelligent way to respond to the novelties introduced in this protocol. As it stands, I appear to accidentally and frequently trigger key combos (e.g. Ctrl+enter) that result in some garbage being printed, which can be quite disruptive.

What would be the best way forward here? Should I, as a user, configure programs (e.g. fish shell) to somehow deal with the novel key combos introduced in this protocol? Or should people "pressure" maintainers of other programs to implement this new protocol, by e.g. opening issues? I would be happy to do the latter. Emacs, for example, could benefit from being capable of handling more key combos, but currently doesn't understand the new protocol at all.

@kovidgoyal
Copy link
Owner Author

kovidgoyal commented Feb 11, 2021 via email

@Sleepful
Copy link

Sleepful commented Feb 17, 2021

ctrl+shift+letter isn't working in my OSX machine, I was trying to look at different layouts.

here I tried with ctrl+shift+l

Press any keys - Ctrl+C or Ctrl+D will terminate
Ctrl+LEFT_CONTROL PRESS 
CSI 57441 ; 5 u

Shift+Ctrl+LEFT_SHIFT PRESS 
CSI 57440 ; 6 u

Ctrl+LEFT_SHIFT RELEASE 
CSI 57440 ; 5 : 3 u

LEFT_CONTROL RELEASE 
CSI 57441 ; 1 : 3 u

2021-02-17 00:30:30.728 kitty[68055:2243535] updateIMEState: 0.500000, 0.000000, 0.000000, 0.000000
2021-02-17 00:30:34.385 kitty[68055:2243535] updateIMEState: 0.000000, 15.500000, 7.000000, 12.500000
on_key_input: glfw key: 57441 native_code: 0x3b action: PRESS mods: 0x4 text: '' state: 0 sent key to child
2021-02-17 00:30:34.541 kitty[68055:2243535] updateIMEState: 0.000000, 53.000000, 7.000000, 12.500000
on_key_input: glfw key: 57440 native_code: 0x38 action: PRESS mods: 0x5 text: '' state: 0 sent key to child
2021-02-17 00:30:34.880 kitty[68055:2243535] keycode: 0x7c (<cc>) mods: ctrl+shift char_count: 1 deadKeyState: 0 repeat: 0
2021-02-17 00:30:34.880 kitty[68055:2243535] text: <none> glfw_key: RIGHT marked_text: 
2021-02-17 00:30:34.880 kitty[68055:2243535] updateIMEState: 0.000000, 90.500000, 7.000000, 12.500000
Keypress matched action: next_tab
on_key_input: glfw key: 57351 native_code: 0x7c action: PRESS mods: 0x5 text: '' state: 0 handled as shortcut
2021-02-17 00:30:34.953 kitty[68055:2243535] updateIMEState: 0.000000, 90.500000, 7.000000, 12.500000
2021-02-17 00:30:35.511 kitty[68055:2243535] updateIMEState: 0.000000, 90.500000, 7.000000, 12.500000
on_key_input: glfw key: 57351 native_code: 0x7c action: RELEASE mods: 0x5 text: '' state: 0 ignoring release event for previous press that was handled as shortcuton_key_input: glfw key: 57440 native_code: 0x38 action: RELEASE mods: 0x4 text: '' state: 0 sent key to child
2021-02-17 00:30:35.514 kitty[68055:2243535] updateIMEState: 0.000000, 90.500000, 7.000000, 12.500000
on_key_input: glfw key: 57441 native_code: 0x3b action: RELEASE mods: 0x0 text: '' state: 0 sent key to child

it actually works with some letters, such as i, but it does not work with a lot of letters, such as o, p , h, j , k , l,, don't know which others.

Apparently it matches a next_tab action, but my kitty.conf doesn't have such configuration...

Here is for ctrl+shift+k

2021-02-17 00:39:13.033 kitty[68055:2243535] updateIMEState: 0.000000, 603.000000, 7.000000, 12.500000
2021-02-17 00:39:13.920 kitty[68055:2243535] keycode: 0x7e (<cc>) mods: ctrl+shift char_count: 1 deadKeyState: 0 repeat: 0
2021-02-17 00:39:13.920 kitty[68055:2243535] text: <none> glfw_key: UP marked_text: 
2021-02-17 00:39:13.920 kitty[68055:2243535] updateIMEState: 0.000000, 603.000000, 7.000000, 12.500000
Keypress matched action: scroll_line_up
on_key_input: glfw key: 57352 native_code: 0x7e action: RELEASE mods: 0x5 text: '' state: 0 ignoring release event for previous press that was handled as shortcuton_key_input: glfw key: 57352 native_code: 0x7e action: PRESS mods: 0x5 text: '' state: 0 handled as shortcut

scroll_line_up

I have no idea where this is coming from... I think GLFW? It's misleading because the docs say ctrl+shift+l is bound to cycling the layout, but the next_tab takes over. However, if I use map alt+shift+l next_layout then I can cycle layout just fine. I did not find anything in the docs that would point me towards turning off this matched action effect.

@kovidgoyal
Copy link
Owner Author

ctrl+shift is kitty_mod and is used for kitty shortcuts

@orki
Copy link
Contributor

orki commented Mar 31, 2021

In case the request was not clear, @kovidgoyal are you willing to consider adding hyper and meta to the protocol spec before 0.20.0, now that there exists an actual implemetantion in pull request #3430 ? I'm happy to provide patches to the keyboard protocol documentation. This has been long sought by terminal emacs users for decades.

@kovidgoyal
Copy link
Owner Author

Yes, I have no objections to adding more modifiers. One can never have
too many modifiers :)

@kovidgoyal
Copy link
Owner Author

Note the protocol has been updated to add support for four more modifiers, hyper, meta, num lock and caps lock. Also, I neglected to assign the keypad begin key a code, this has now been fixed.

@dankamongmen
Copy link
Contributor

@kovidgoyal I've read through the docs and I'm really liking this proposal so far. It's a huge improvement over the previous protocol, in my opinion, and seems to solve pretty much every problem I've encountered w.r.t. terminal key codes. I'm planning to start working on a patch to add this protocol to the foot terminal in the near future.

@craigbarnes did foot ever gain support for the protocol? I've recently added support and discovery to Notcurses, but even with foot 1.8.2-144 i don't get any response to my CSI > u query, and thus i don't discover support for it there.

fwiw, i agree that this is a tremendous and necessary improvement over what existed before.

@kovidgoyal
Copy link
Owner Author

@dankmongmen You might want to test with iTerm2 as well, it seems to have some kind of partial support for this protocol: gnachman/iTerm2@bd3865b and https://gitlab.com/gnachman/iterm2/-/issues/9506

@msokalski
Copy link

@kovidgoyal I think it would be great to maintain list of terminals supporting this protocol { name [+link], [version,] full / partial }.
Probably, bottom of your specification page is a good place for it.
Btw, thanks for making this protocol a real thing!

@kovidgoyal
Copy link
Owner Author

kovidgoyal commented Oct 6, 2021 via email

@dnkl
Copy link

dnkl commented Oct 6, 2021

@craigbarnes did foot ever gain support for the protocol? I've recently added support and discovery to Notcurses, but even with foot 1.8.2-144 i don't get any response to my CSI > u query, and thus i don't discover support for it there.

@dankamongmen answering for @craigbarnes here; foot-1.8.2 does not implement the kitty kbd protocol. Neither does the recently released 1.9.x. We still intend to implement it, but I don't have a timeline for when it'll be ready.

@kovidgoyal
Copy link
Owner Author

And apparently iTerm is planning to implement it too: https://gitlab.com/gnachman/iterm2/-/issues/10017

@kovidgoyal
Copy link
Owner Author

Some progress on implementing it in foot: https://codeberg.org/dnkl/foot/issues/319 and https://codeberg.org/dnkl/foot/pulls/811

@dankamongmen
Copy link
Contributor

Some progress on implementing it in foot: https://codeberg.org/dnkl/foot/issues/319 and https://codeberg.org/dnkl/foot/pulls/811

awesome! so long as the "report current support" is implemented correctly, Notcurses will pick it up automatically (after attempting to set 11), and not send XTMODKEYS.

@dnkl
Copy link

dnkl commented Nov 25, 2021

@dankamongmen the plan is to add support incrementally. If we initially support "disambiguate escape codes" (0b1) only, will notcurses handle that gracefully? Assuming foot returns \e[?1u when notcurses tries to set \e[>3u.

I think that's what you are saying, but just wanted to make sure....

@dankamongmen
Copy link
Contributor

@dankamongmen the plan is to add support incrementally. If we initially support "disambiguate escape codes" (0b1) only, will notcurses handle that gracefully? Assuming foot returns \e[?1u when notcurses tries to set \e[>3u.

I think that's what you are saying, but just wanted to make sure....

yes. Notcurses will use whatever you can offer. i add all possible/known keyboard inputs to my input automaton. right now a \e[?1u and a \e[11u have the exact same effect -- Notcurses will not bother sending XTMODKEYS. take a look at

to get the full story

@dankamongmen
Copy link
Contributor

@dnkl i forgot to mention notcurses-input. it shows you exactly what notcurses thinks you've pressed, including release/hold/press status and modifiers. you might find it useful, and i'd love to hear any cases where i've failed to handle something properly. it also handles mouse inputs and even some signals.

@dnkl
Copy link

dnkl commented Dec 8, 2021

foot-1.10.3 implements all flags/modes. I.e. we now support the full Kitty keyboard protocol, barring bugs (of which there are obviously zero...)

@dankamongmen
Copy link
Contributor

foot-1.10.3 implements all flags/modes. I.e. we now support the full Kitty keyboard protocol, barring bugs (of which there are obviously zero...)

appreciate the help flushing out a few bugs from notcurses! i hope notcurses-input was helpful. glad to see this protocol getting taken up!

@kovidgoyal
Copy link
Owner Author

kovidgoyal commented Dec 9, 2021 via email

@rsmath
Copy link

rsmath commented Dec 7, 2022

I am new to this thread but I am facing a problem. Sometimes I press something in my Kitty terminal that makes some of my .vimrc mappings don't work. How do I fix this?

Reading this thread, I think I have to set Seen modifyOtherKeys: true but where/how do I do that?

Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests