support unicode?

[enthus1ast]

it would be nice when illwill also supports unicode.
Looking at the (posix) code part it also could be simple to capture them

eg:
https://www.utf8-chartable.de/
the german letter: "Ö" is encoded as 195.int 150.int
and the current getKeyAsync captures it correctly as

charsRead = 2
[195, 150 .....]

but since the current getKeyAsync returns a Key it will not compose properly.
Maybe implement a

getKeyUnicode(): tuple[key: Key, rune: string]

function that has the representation of the key in the rune string?

The inputSeq IS the correct rune!

      var inputSeq = ""
      for i in 0..<charsRead:
        inputSeq &= char(keyBuf[i])

echo inputSeq
Ö

[enthus1ast]

maybe something like this: https://github.com/enthus1ast/illwill/blob/unicode/illwill.nim

[liquidev]

+1, this would also enable easy async text input like GLFW's char callback.

[enthus1ast]

i was thinking about this, and i'll try to explain my thoughts in another issue.

There are other issues with the current getKey implementation, mainly: there is for eg. ShiftA and CtrlA but, at least windows, posix i have not checked yet, can detect both modificator keypresses at once.

Maybe its a good idea to deprecate getKey in favor of a getInput proc that returns the rune + all modificators to the keypress and/or the mouse with all modificators.

I'll try to get a pr done when the mouse one is merged.

[johnnovak]

Yeah, I haven't really investigated the UTF-8 issue to be honest. I was just happy to get it working with an English layout. If you can get this working reliably on all platforms, then yeah why not! Also a reminder, keyrepeat needs to work properly. From memory, UTF-8 handling with the standard windows terminal is a bit of a nightmare, so good luck :)

[enthus1ast]

@johnnovak the good thing is the mouse pr already implements the windows console input (in fact the code already captures all events, but filters them for only mouse events).

[johnnovak]

@enthus1ast Yeah from memory using the windows console directly is the way to go (as opposed to stdin/stdout).

[enthus1ast]

sorry @johnnovak i'm busy with other projects right now, but maybe when the time comes :)

[johnnovak]

@enthus1ast sure :) btw, i don't need unicode myself, i'm happy with latin1, but i can see why it can be a problem with German letters. a PR is welcome whenever you get around to doing it :)

[enthus1ast]

@johnnovak any feelings about pulling in winim?

[enthus1ast]

@johnnovak what do you think, should i upgrade getKey to also handle unicode or introduce a completely new getInput proc (that does mouse, unicode, old Key styles, maybe even modifiers)

[johnnovak]

If you are planning to do such thing, think long and hard first whether you're willing to implement Unicode support properly for Windows, Linux & Mac.

[enthus1ast]

wip https://github.com/enthus1ast/illwill/tree/municode

[enthus1ast]

what i'm currently aiming for is:

that it do not crash cmd.exe when typing non ascii eg: öäü

keep the old api (getKey) working if you wish and/or i could deprecate it
in favour of:
eg.:
- getKeyUnicode
- or
- getInput (my favorite)

the new api currently looks like this:

type KeyUnicode* = tuple[key: Key, rune: string]
proc getKeyUnicode(): KeyUnicode

the key is the old Key from illwill the rune string is the unicode symbol(s).
I like the getInput name more, since we're not getting just keys, but also eg.: mouse.

Another thing i would like to change, is the modifier keys reporting.
Currently we have CtrlA but i think it would be better to provide an Object with every input
that lists the state of the Modifier keys. A little like the Mouse Input currently works:

type Modifier = object
    ctrl*: bool ## ctrl was down on event 
    shift*: bool ## shift was down on event

the downside of this is, that you then cannot have a big case that reacts on CtrlA,
but must check on A + modifier.

I have to check if posix can report both at the same time.

Another good addition would be, to also let illwill report resize events.
Windows can report these in their events, for posix i have to again check.

What do you think?
If i have a little more stable api, i would create a draft pr, where we could better talk about the code.

[hvbargen]

@enthus1ast Did you manage to solve the umlaut problem in your fork?

I'm new to nim, but I like it. My very first project should be a simple directory chooser command-line utility.
I thought I'd use illwill or your illwillwidgets library to create a simple TUI for it, but as soon as I hit e.g. the ä key, the terminal crashes. ATM this is a show-stopper for all non-english users.

I understand that @johnnovak isn't eager to solve this. But did you solve it?
If yes, am I missing something obvious?

I tried the illwill program showing the key names. I also tried chcp 65001 before calling it.
As soon as I hit an umlaut key, the whole console windows closes.

[enthus1ast]

@hvbargen

As soon as I hit an umlaut key, the whole console windows closes.

Yes the terminal crashes.

Have a look at this branch:
https://github.com/enthus1ast/illwill/tree/municode

there i was working on unicode, but i do not know in which state i left it.
I'm not actively working on this atm.

If you like you could try it and maybe fork it.

Edit: Basically the windows "mouse code" is where all events are captured, they're just filtered for mouse stuff.
You must replace the old illwill windows keyboard code with the event capturing and emit the correct illwill keyboard events.

Edit2: I got a little sloppy with the unicode branch(es) because i lost some uncommited code in a notebook crash, then swapped the notebooks a few times.

[hvbtup]

OK (I'm the same person), I'll take a look at it.
Yesterday I found out that the easiest way to avoid the terminal crashes was to use _wgetch from conio.h instead of _getch.
This seems to return an integer which, if not negative, directly means the Unicode code point.

[johnnovak]

If anyone wants to solve this, the solution must be cross-platform and work equally well on Linux and macOS in addition to Windows if you want it to get merged.

I haven't used the library for years now, so it won't be coming from me in foreseeable future, but happy to receive PRs.