Arrow keys all report as ASCII code 27 with readCharacter()

[bdw429s]

When reading a key with ConsoleReader.readCharacter() all arrow keys return code 27. There is also extra text that shows up in the prompt such as [A [B [C and [D which seems to imply that the arrow keys are two bytes and only the first byte is being read. I think this might be related to https://github.com//issues/100

This is 64 bit Windows 7 with the bleeding edge of Jline.

[fantasyzh]

In my opinion, all that jline provide is line editing, and the only main API would be readline().
It suprised me that you rely on the readCharacter() method (which I thought an internal method of ConsoleReader)

[bdw429s]

I am building a CLI with JLine with interactive commands that wait for a keystroke from the user to continue processing before control is returned back to the prompt. For example, a "more" command that can have data piped into it and the user can press "space" to output a line, or "enter" to output a page. The command calls readCharacter in a loop until its out of data and then returns to the prompt. Or it could be as simple as a "pause" command that says "press any key to continue" and then waits for the next keystroke.

What library would you suggest using to capture keystrokes from the user that aren't specifically part of a "line" typed at a prompt?

[trptcolin]

I think readCharacter() is definitely something we should support - any chance you know whether this is a new issue (not in 2.11)?

[fantasyzh]

The point is that: there is no portable way to represent those control keys(arrow keys, etc.) with a single character. That is why we use ANSI escape sequences, key bindings, and why we have so many problems before on differents OS.
jline filter readCharacter() results with key mappings to get Operations.

[bdw429s]

I appreciate the internal difficulties in capturing complex keystrokes, and I think the fact that most of these issues have been solved with readLine() is promising that the same logic can be refactored to apply to a single character at a time. Perhaps abstract the keymap stuff to recognize one stroke at a time in a way that can be utilized by readLine() or stand alone for single character recognition.
I think this will be a great feature to allow people to build interactive programs that react to user input while executing in a CLI context.

[bdw429s]

I'm still interested in seeing this issue solved, if for no other reason than to be able to use my keyboard's arrow keys to control my ASCII "snake" game I made in CommandBox :)

On this note, I would be tempted to take a whack at this myself, but there is an absolute metric ton of cruft in the readLine() method that appears to have to do with making a VI editor or something. Is all that really necessary? It seems to me someone decided to make a Java version of VI and coded a bunch or VI-specific escape key support into JLine. Shouldn't that be abstracted out a bit so it's not part of the JLine core? If nothing else, can we subclass the reader with a VI version that does all that stuff to lighten up the class?

[gnodet]

As it has been stated, jline's main goal is the readLine editing. All the vi/emacs stuff has been added to be much closer to gnu readline...

Anyway, to support arrow keys correctly, you need to use the KeyMap class to register the sequence/operations you want, then find a mapping operation from the sequence of chars you get.

I've implemented a nearly complete less pager on top of jline:
https://github.com/apache/karaf/blob/master/shell/commands/src/main/java/org/apache/karaf/shell/commands/impl/LessAction.java

I'm sure there's room for improvements though...

[bdw429s]

Holy cow, that's a lot of code! Here my implementation of a more command in the CommandBox CLI
https://github.com/Ortus-Solutions/commandbox/blob/master/src/cfml/system/commands/more.cfc

I'm not sure I follow what you're saying with the KeyMap stuff. I'm familiar with what that is, but there's no way for me to access more than one character at a time (an arrow key is a sequence of two characters) and I don't want the user to have to press "enter" and capture the entire line.

The ability to read a single character sequence from the console seems like it would be a great feature and it would certainly make my live easier when using JLine for interactive CLIs like CommandBox.

[gnodet]

Yeah, less is quite complicated.
The more command requires much less code:
https://github.com/apache/karaf/blob/master/shell/commands/src/main/java/org/apache/karaf/shell/commands/impl/MoreAction.java

Note that in the above code, the support for arrow keys is completely broken and does not work...

Anyway, in the less code, I could make the readOperation() something available from the ConsoleReader:
https://github.com/apache/karaf/blob/master/shell/commands/src/main/java/org/apache/karaf/shell/commands/impl/LessAction.java#L680-L726

You would give a KeyMap as an argument, and it would return the Operation translated from the input stream.

[bdw429s]

@gnodet

Yeah, less is quite complicated.

Well, you're also using Java. CommandBox is written in CFML which dispenses with a lot of the boilerplate :)

Can you explain what the purpose would be to pass a keyMap into such a method? I'm just thinking about how readLine() works, and there's no need for the calling code to worry about the keymap-- that's taken care of internally. A method to read a single character from the input seems like it would deserve the same level of abstraction since it's right along the same lines from an API perspective:

• Read all the characters up until the user hits "enter"
• Read the next character entered

Also, if we can get this figured out, I'll be able to actually play CommandBox snake with my array keys!
https://www.ortussolutions.com/blog/commandbox-snake-all-in-good-fun

[gnodet]

Characters can't be easily used to represent arrow keys. There's no single representation on various systems.
So either you read the character stream and parse the sequence that represent arrows yourself (up would usually be \033[0A for example), or you create a simple KeyMap and let readBinding() do the work for you.

You can easily create a KeyMap

enum Action {
    Up, Left, Right, Down,
        Quit, Retry
};

KeyMap map = new KeyMap();
map.bind("\033[0A", Action.Up);
map.bind("e", Action.Up);
map.bind("E", Action.Up);
map.bind("\033[0B", Action.Left);
map.bind("s", Action.Left);
map.bind("S", Action.Left);
map.bind("\033[0C", Action.Right);
map.bind("f", Action.Right);
map.bind("F", Action.Right);
map.bind("\033[0D", Action.Down);
map.bind("c", Action.Down);
map.bind("C", Action.Down);
...

Object o = reader.readBinding(map);
if (o == Arrow.Up) {
  // this is the up arrow
}

alternatively, you can use

char c = reader.readCharacter();
if (c == 27) {
    c = reader.readCharacter();
    if (c == '[') {
        c = reader.readCharacter();
        if (c == '0') {
            c = reader.readCharacter();
            if (c == 'A') {
                // this is the up key
            }
        }
    }
}

The point of using KeyMap is that you can bind your Action.Up enum to different key combinations, such as the up arrow key or the E / e characters without having to care about the mapping.

[bdw429s]

Thanks @gnodet for the explanation. How would I go about reading the next keystroke using the default keyMap that the ConsoleReader uses? Will getKeys() return that? Also, what will be returned from readBinding() if the sequence typed doesn't match anything in the keyMap?

I'm just trying to think about how I can keep from duplicating any work already done in JLline and wrap all this up into a nice function that will read the next keystroke and basically account for anything without having to map every single key.

[gnodet]

The KeyMap used while reading a line is available using ConsoleReader.getKeys(), so it's just a matter of calling

 reader.readBinding(reader.getKeys());

However, the internal key map has all the VI / emacs line editing stuff, not sure you would really want that for a snake game...

The readBinding method will block until a matching binding is found or EOF. If your keyMap is not full (i.e. not all characters map to something), characters that can't be mapped to anything will be silently discarded (see the junit example below where 'a' and 'd' are discarded).

https://github.com/jline/jline2/blob/HEAD/src/test/java/jline/console/ConsoleReaderTest.java#L658-L675

[bdw429s]

not sure you would really want that for a snake game...

Well, it's much bigger than just that. CommandBox is basically a command framework for CFML developers that just uses Jline for the console interactions. Commands (like the "more" I linked to previously) are implemented as a component that extends a base class that provides them with helpers for ASNI formatting, asking the users for information, and a way for dynamic tab completion, etc. So, some of the built-in methods are ask() (read next line), confirm() (read read and convert to boolean), and waitForKey() (read next keystroke) which you can see here:

https://github.com/Ortus-Solutions/commandbox/blob/master/src/cfml/system/Shell.cfc#L190-L206

This is a generic utility that command authors can use to collect a single keystroke from the user and it abstracts away the actual console reader. I don't have any idea what all the possible keystrokes are that will need to be mapped by every possible command. I just need to assume that a command author might want to listen for any possible keyboard input at runtime and have this method return the next thing the user presses.

I understand that key presses that don't correspond to a unicode character will need to be mapped and I can try to do that for stuff like arrow keys, pg up/down,etc (though I'd prefer not to duplicate any logic JLine has already accomplished). However mapping every possible character that can be entered, especially once you take into account character like "©" seems a little impossible.

Is there a way that we can capture a single keystroke, and if it has an unicode representation, return that, otherwise just have a map for the rest of the buttons on my keyboard (up, down, etc)? What I'm looking for here needs a bit more abstraction than what the readBinding() method is going to give me. If you look at things like the java.awt.event.KeyEvent class as an example of how this is commonly done-- a list of constants representing common "special" keys is predefined, otherwise you just get the unicode character.

[gnodet]

We can't really be more generic than using KeyMap.
Everything when reading a command line using ConsoleReader.readLine() is handled through readBinding().

First, jline can't support key pressed, key released, or any such event. The reason is that it's main goal is terminal interaction, and those are not available in virtual terminals such as ssh or telnet. For example the fact the user presses the SHIFT key and later release it will never go through such channels. They are not available from stdin either afaik.

So we don't have access to keyboard events, just a stream of characters. What the KeyMap and readBinding do is map certain sequences of characters to something.
If you want to retrieve the next character from the stream, just use readCharacter as you do. If you want to do the mapping, use readBinding, but there's no way we can return virtual keys code, as we don't have them anymore in jline. They have been handled before and translated to a character stream. Some information has been lost and we can't really reconstruct it.

That's why arrow keys are received as the sequence <ESC>[0A for example, but we don't really know it's an arrow key that has been pressed.

But again, the readBinding does indeed allow you to do what you want imho.

static class KeyPress {
   private final int keyCode;
   private final char keyChar;
   public KeyPress(int keyCode, char keyChar) {
      this.keyCode = keyCode;
      this.keyChar = keyChar;
   }
}

Object[] map = new Object[256];
for (int i = 0; i < 256; i++) {
    map[i] = Operation.SELF_INSERT;
}
KeyMap keyMap = new KeyMap("custom", map, false);
keyMap.bind("\033[0A", new KeyPress(VK_UP, 0));
keyMap.bind("\033[0B", new KeyPress(VK_LEFT, 0));
...

KeyPress readKeyPress() {
    Object o = consoleReader.readBinding(keyMap);
    if (o == Operation.SELF_INSERT) {
        return new KeyPress(0, consoleReader.getLastBinding().charAt(0));
    } else if (o instanceof KeyPress) {
        return (KeyPress) o;
    } else {
        throw new IllegalStateException();
    }
}

With such code, you can return objects that can mimic awt keyboard events, but I think that should be part of your platform, not jline.

[bdw429s]

We can't really be more generic than using KeyMap.

Heh, well you can't be any more flexible, but I guess I was looking for a more "turn key" solution to this. This is a swiss army knife, but all I needed was a box cutter.

First, jline can't support key pressed, key released, or any such event.

Sure. And to be clear, I didn't expect it to. The AWT reference was purely to show that other libraries abstract their users from the mundane details of the escape codes going on behind the scene of a single key press.

but we don't really know it's an arrow key that has been pressed.

Well, let's be honest-- yes we do. This is a problem that has already solved as evidenced by the fact that the JLine library already allows me to interact with it using keys like up, left, and right. I can appreciate that some massaging and introspection of the character stream is necessary to divine this, but I guess that's been my point all along. The rest of the API of JLine that I'm familiar with has abstracted this nicely away from the developers using it so it just feels a little odd that we're pushing it off on the devs here to worry about something that feels like a lower-level intricacy.

Sure, someone has to worry about what comprises an "up" key, and assuming you're a developer on JLine, I can appreciate that it's something you're used to having to account for. I'm just suggesting that there's value in a Jline method that that allows developers to interact with a terminal at a higher level of abstraction, on par with other languages/libraries. I'm glad to have JLine's abstraction in the rest of its API, especially when I see tickets like #100 that show the pitfalls of depending on specific escape codes that apparenlty can even be different based on locale settings.

How do you feel about JLine including some constants somewhere that represent all the common escape codes for easy mapping, or at least some documentation you can point me to that lists them? I don't mind mashing all the keys on my keyboard to see what gets spit out, but that feels a little wrong.

Thanks for the final code sample-- I'll see what I can do to mimic this. Am I correct that I'll need to use the bleeding edge or wait for the next release to do this since readBinding() isn't accessible in the current stable release?

[gnodet]

I've recently added support for infocmp which provides the character sequences for various keys.
Given this information is missing for windows, we could add it somehow

• adding such information for the windows terminal (it's currently missing them)

This would allow

   keymap.bind(terminal.getStringCapability("cursor_left"), ...)

I'm also open to make those values available as constants on the WindowsTerminal.

And yes, readBinding is not yet available in any release, though you should be able to copy that method into your own code if you want.

[ASemeniuk]

This conversation is rather old, yet the jLine library still looks like the only available cross-platform option for detecting (synchronous) key presses in command-line environment. So here are my findings so far:

Indeed, the best way to read complex (consisting of more than 1 character) key sequences is to use KeyMap approach with readBinding() method. However, it has few downsides:

• you have to explicitly specify the comprehensive list of key bindings you would like to read - any key not listed in your KeyMap will be ignored by readBinding() method. This may not be a big deal so snake games and such (where you have limited number of commands), but could become a problem if you need to enter an arbitrary text;
• ESC key is not correctly processed, if you need bind something to "\u001b". Since char 27 starts all of the complex sequences, the binding for "\u001b" will be triggered by any other complex key (arrow keys, PgUp, PgDon, Home, Insert etc.), unless this key itself has a binding within your KeyMap. So in order to deal with the problem, you would have to add all complex key bindings (which are numerous - don't forget about Ctrl+Something) to your KeyMap.

So, the only viable option is the one also discussed here: call readCharacter() once, then check whether the char is 27, then call readCharacter() again etc. There is a catch, though: you will not be able to distinguish a simple ESC key - if it was pressed, then subsequent call to readCharacter() will stop your program until next key is pressed. Not be a big problem if reading is performed in separate thread, but in case of synchronous input, you would need to somehow know, whether this char 27 is ESC key inself, or the beginning of the complex sequence. The only way I found possible to do it is the following (something similar is used by readBinding() method itself):

1. Acquire a reference to NonBlockingInputStream which is wrapped by ConsoleReader:

consoleReader reader = new ConsoleReader();
NonBlockingInputStream stream = (NonBlockingInputStream) reader.getInput(); 

2. Use the peek() method to check whether stream has some other characters (part of the complex sequence following the 27 code) in it:

if (stream.peek(100) > -2) {
    //It is a sequence. Use readCharacter() method to get the rest of the characters.
} else {
    //It is an ESC key
}

Here 100 is the timeout in milliseconds for stream to wait for input before returning result. The idea here is to specify relatively small value, which is not enough for human to type the second key after the first one was pressed (so the sequence is the complex key not and not just too keys pressed one after another). Value -2 is the special code meaning that stream reached its end.

This is not ideal, of course. But I can't think of any other way to perform the task needed.

[gnodet]

Note that JLine 3 has added noMatch and unicode fields to the KeyMap allowing to proceed otherwise discarded sequences.
Coupled with the range method, mapping all the Ctrl keys can be done in a single line:

        bind(viins, SELF_INSERT, range("^@-^_"));

[bdw429s]

@ASemeniuk All my needs have been (mostly) met in JLine 3, where I can provide my users writing arbitrary interactive CLI task runners the ability to capture most of the possible keystrokes. You can see here that I simply dug through the code and mapped all the common key strokes and capture them, returning a bit of pre-defined text to represent the special multi-character bindings. I still wish I had a way in JLine to do this automatically, but this works for me.

Here is my code (CFML, a dynamic JVM language)
https://github.com/Ortus-Solutions/commandbox/blob/development/src/cfml/system/Shell.cfc#L311-L401

And here's the docs I provide my users:
https://commandbox.ortusbooks.com/task-runners/task-interactivity#waitforkey

[ASemeniuk]

Thanks for the replies.
My task is a bit different though, as I need to detect keypresses synchronously (meaning app is stopped until user inputs anything) and I do not provide any visual feedback when key is pressed. So readCharacter() was the only method I really needed - and I could not find its analogues in jLine3. Seeing the waitForKey() example, I keep thinking that having some simple (as in 1-line) method for this task would be a great feature. But it works nevertheless, so I am happy as is.

[gnodet]

@ASemeniuk I'm all for adding what's needed in JLine3, but I fail to understand the exact purpose. JLine3 provides access to the NonBlockingReader object through the Terminal#reader() method. What kind of processing do you need exactly for the esc key ? If you want the same processing that in the jline2 readCharacter method, you can do the same easily with jline3:

public int readCharacter(Terminal terminal) throws IOException {
    int c = terminal.reader().read();
    if (c == ESCAPE) {
        int next = terminal.reader().read(100);
        if (next >= 0) {
            return next + 1000;
        }
    }
    return c;
}

[ASemeniuk]

@gnodet There is not much I need, actually. Just so when I bind "\u001b" to some operation and use readBinding() method, this operation is triggered only by ESCAPE key - not arrow keys or PgUp/Dn, Home, Insert etc.