Incorrect behavior on Home, End, etc. under screen

[senia-psm]

Reproduce steps: start screen, start Ammonite using sbt projectName/test:console (I can't download https://git.io/vOdoc), tap Home, tap Enter:

@
SyntaxError: found "\u001b[1~", expected Import | Prelude ~ BlockDef | Expr | End in

End, Enter:

@
SyntaxError: found "\u001b[4~", expected Import | Prelude ~ BlockDef | Expr | End in

Same for Alt+arrow.
Home, End, etc works fine for default scala repl under screen and for Ammonite without screen.
Difference between export under screen and without screen is in TERM, TERMCAP, WINDOW, SHLVL, STY, OLDPWD variables. TERM="xterm" TERMCAP="" WINDOW="" SHLVL="1" STY="" OLDPWD="" sbt projectName/test:console command doesn't fix Ammonite under screen and doesn't break Ammonite without screen.

[Blaisorblade]

I can also reproduce this under tmux, a screen replacement that also sets TERM=screen.

Workaround: use C-a/C-e (that is, Ctrl-a for Home, or Ctrl-e for End).

TERM="xterm" TERMCAP="" WINDOW="" SHLVL="1" STY="" OLDPWD="" sbt projectName/test:console command doesn't fix Ammonite under screen and doesn't break Ammonite without screen.

Makes total sense to me.

Background: Different terminal emulators use different escape sequences for the same commands, so native *x programs need to use libraries to interpret those escape sequences (that is, termcap/terminfo, or higher-level libraries wrapping those, such as readline or ncurses). Those libraries are hard to access on the JVM, hence I guess Ammonite doesn't use them. Indeed, escape sequences are hardcoded in terminal/src/main/scala/ammonite/terminal/SpecialKeys.scala.

Working hypothesis: screen sends for Home/End a different escape sequence than xterm, hence Ammonite-REPL does not understand it.

[lihaoyi]

Is termcap/terminfo accessible via shelling out? That would be the easiest, since we already shell out to get TTY height/width, set terminal state, etc. like we do here

If not we can add it to the list of hardcodes. I'm guessing JLine has fixed/doesn't-have this problem, so whatever they're doing probably works for us too.

(I know nothing about terminals and just hacked this stuff together, so don't take my word for anything, and feel free to tell me the code I have in terminal/ is totally wrong)

[Blaisorblade]

I just learned that commands tput khome and tput kend should do what you want. Furthermore, "\u001b[1~" and "\u001b[4~" literally match the sequences, so there isn't extra weird stuff going on.

$ tput khome|hexdump -C
00000000  1b 5b 31 7e                                       |.[1~|
00000004
$ tput kend|hexdump -C
00000000  1b 5b 34 7e                                       |.[4~|
00000004

What I looked up: man tput and man terminfo; the latter has a list of capabilities, that is keys in the terminfo database — there seems to be one per special key 😱

But I don't fully get terminals either — I've carefully avoided entering that deep swamp 😉 . So I'd ask "what about using JLine for this" — Scala's REPL apparently fixed the same issue between 2.11.0 and 2.11.7 (I'd guess by updating JLine — its version changed from 2.11 to 2.12.1), and they do shell out, but read the whole terminfo entry for the terminal at once.

I'm also confused that you have those hardcodes and use JLine? Any chance this is also a quick hack because interfacing more to JLine is the right thing but hard to do, or is JLine too restrictive?

[lihaoyi]

I'm also confused that you have those hardcodes and use JLine? Any chance this is also a quick hack > because interfacing more to JLine is the right thing but hard to do, or is JLine too restrictive?

If you want to use JLine, go ahead!

// Ammonite on Linux/OSX and JLineWindows on Windows
repl.frontEnd() = ammonite.repl.frontend.FrontEnd.JLineUnix

(Documented in http://lihaoyi.github.io/Ammonite/#Configuration)

Put it in your ~/.ammonite/predef.scala and you'll have a JLine terminal forever more.

I, for one, want my input-syntax-highlighting, and multiline editing capabilities, so I'm willing to beat through a few bugs to make it work =D

JLine is pretty restrictive. In particular, input highlighting and multiline editing/history are basically never going to happen. I've tried! So I went and made my own thing and left JLine as a fallback for people who like the old way

[Blaisorblade]

Thanks, I see. tput khome/tput kend seems the easiest fix then, but I'll leave this to you.

I've seen some of the shelling out code that you pointed me to overlaps a lot with https://github.com/jline/jline2/blob/master/src/main/java/jline/UnixTerminal.java — and the interaction with infocmp might be useful.
But it seems best I shut up unless I have a working PR with a refactoring, should I ever get started on this — and this might be lots of work for a small benefit. (I'm just taking a look at the project for now.)

[lihaoyi]

It seems like JLine has much of the logic around these terminal key things in this file https://github.com/jline/jline2/blob/92792c59a42713d55d434526ddf0361993d3f741/src/main/java/jline/internal/InfoCmp.java

The question is how can we use it to do the same thing...

[lihaoyi]

Oddly enough, this exists in JLine master but not in 2.12 I'm using. I wonder why that's the case

[Blaisorblade]

Yes, that's the parsing of the infocmp output, they get all info with a single command.
Apparently that code is also in JLine 2.13, introduced in jline/jline2@3b0c474. Before that, they just hardcoded more escape sequences and other stuff.