On X11, dead keys can only be differentiated by scancode

[nandoflorestan]

Kivy offers only 2 keyboard events: on_key_down and on_key_up. The way this currently works makes it impossible to develop a game, or game-like UI, with full control of what is happening on the physical keyboard.

(At least on Linux) Kivy's on_key_down really behaves like the on_keypress event found in every other GUI framework, and therefore should be renamed to on_keypress. I mean, if I hold down a key, this event is fired multiple times (according to the repetition configuration of the operating system or windowing system). This behavior is typical of a keypress event and is intended for text editing.

The normal behavior (outside of current Kivy) for an event named on_key_down is to fire only once when I press a key. Suppose that key was CTRL; now if I press the C key, on_key_down is fired again. When I release each key, the on_key_up event is fired. With this I mean, these events provide low level control of the keyboard. And then we should have a separate pair of events for the currently available high level control of the keyboard.

I am almost certain that the operating system provides both APIs: 1. keypress events to input characters with repetition rate configured in the OS, and 2. key_down events for purposes other than text entry. Frameworks such as Qt expose both APIs to the application.

Theoretically it is possible for me to get the behavior I want from the behavior offered by Kivy. If I simply accumulate the pressed keys in a set, I will know exactly what is going on with the keyboard, since then I could simply watch changes to the set. However, this still has a problem with dead keys:

On a keyboard with extra keys for accented characters (used in so many countries), although the TextInput widget works fine -- accepting dead keys and resulting on an accented letter on the second keystroke --, there is really no way to know which accent key was pressed (before the second one resolves the resulting character). On an ABNT2 keyboard, the code Kivy gives for both accent keys is the same: 1073741824. If I am developing a game or game-like interface (for purposes other than editing text), then these keys are unusable, since it is impossible to differentiate them on keydown!

(By the way, pyglet reports 2 different key codes for those 2 keys: 206158430208 and 146028888064.)

The same key code is reported for both keys in the key_up event, so in that case it is impossible to differentiate the 2 keys, too. This seems to indicate the issue is the same on key_up. (At least on Linux) Kivy is using the wrong API from the OS: the one for text entry, but the event names are those commonly used for key watching.

The solution, then, would be:

1. Provide a pair of new events that give the developer low level control of the keyboard, which is currently missing from Kivy.
2. Rename the currently ill-named events to proper names, but provide the current synonyms for backwards compatibility, with a Python warning (from warnings import warn) of deprecation, which gets printed out only when the synonyms are used.

[matham]

I do agree with what you're saying, that it's annoying to have to do the conversion from keypress type event to figuring out what keys were pressed. And I guess from what you're saying you can't even always do that reliably currently.

So I think it would be worth fixing this. However, I don't think we can rename the events or change the meaning of the existing events without causing significant API breakage, which I don't think is worth doing here. But also, rather than adding an additional event(s) we should first check if we can somehow modify the existing API in a backward compat way to delver this new info.

The major issue though is that we get the keyboard events from SDL2 for most OSs so we are limited by what it can do. I did find this keyboard function which could be helpful, but I'm not sure it adds anything new.

[nandoflorestan]

EDIT: I think this is the wrong version of the SDL...

Please take a look at this page. Scroll down to the section "Game-type Input". It sounds like the SDL_KEYDOWN event behaves the way a game needs it (without repetition).

If that is the exact same event that Kivy uses (I did not check), then: who is creating the repetition?

I do think Kivy needs at least one extra event, because in the same app, in some circumstances you want to listen to repetition, and in other cases you don't.

[nandoflorestan]

If I try to help by restricting my search to SDL2, I find this:
https://stackoverflow.com/questions/22156815/how-to-disable-key-repeat-in-sdl2

[matham]

That's interesting. We could potentially filter out repeats, if that works. Or at least make it configurable.

However, you also mentioned something about accented keys and it seems like removing repeats won't solve that problem? Because IIUC, the issue is that with key combos the individual keys are not correctly resolved?

I can't really test it, but can you show a log describing the dead keys issue using e.g. this app:

from kivy.app import App
from kivy.core.window import Window


class KeyboardApp(App):

    def build(self):
        Window.bind(on_key_down=self._window_keyboard)
        Window._system_keyboard.bind(on_key_down=self._keyboard)

    def _window_keyboard(
            self, window, keycode, scancode, codepoint=None, modifiers=None,
            **kwargs):
        print(f'Window: keycode={keycode}, scancode={scancode}, modifiers={modifiers}')

    def _keyboard(self, _, keycode, text, modifiers):
        print(f'Keyboard: keycode={keycode}, text={text}, modifiers={modifiers}')


if __name__ == '__main__':
    KeyboardApp().run()

[nandoflorestan]

Since filtering repeats is so easy to do (just add to and remove from a set), I can do it myself in the application. It does mean I am implementing an event that comes for free in PySide, though. Even so, I am happy to redefine this ticket as dealing only with the issue the app developer can do nothing about:

PROBLEM: 2 extra ABNT2 keys are confused on keycode 1073741824. Somehow the TextInput widget works fine for accented characters, but the on_key_down event reports the 2 dead keys with that keycode.

Example output of your test app. I test 3 different keys:

Window: keycode=1073741824, scancode=47, modifiers=[]
Keyboard: keycode=(1073741824, ''), text=None, modifiers=[]
Window: keycode=1073741824, scancode=52, modifiers=[]
Keyboard: keycode=(1073741824, ''), text=None, modifiers=[]
Window: keycode=231, scancode=51, modifiers=[]
Keyboard: keycode=(231, ''), text=ç, modifiers=[]

The last key is also an additional key of the ABNT2 keyboard (c with cedilla). It is working better than the others -- the only thing wrong is that the tuple contains an empty string.

The top 2 keys show the main problem.

Thank you for the code. I did not know that we can get the scancode, which seems to be reported correctly above, for both dead keys (52 and 47). One limitation is that the scancode is only reported when one listens on the global application object (not on a widget), which to me seems arbitrary. I do prefer to listen on a widget, why can't I get all the data.

However, you can see them reporting one and the same keycode, which is the issue. Maybe text=None because those are dead keys, but then, how can I know what glyphs they represent under the current keyboard layout?

Anyway, if pyglet reports 2 different keycodes, that behavior is definitely superior to Kivy's.

[nandoflorestan]

If there's an equivalent bug in SDL2 we might find it at
https://github.com/libsdl-org/SDL/issues?q=dead+keys
...however most of the results fly above my head.