Emacs ANSI support

[hvesalai]

Emacs comint announces it self as DUMP terminal, but it still support ANSI colors.

JLine Terminal should detect emacs based on the INSIDE_EMACS environment variable and turn on ANSI support for it.

I could make a PR and test it if somebody gave me a few pointers where this would be best implemented in JLine.

[hvesalai]

@fommil We need to keep track of this issue so that emacs-sbt-mode does not break when sbt and scala start using JLine3

[hvesalai]

Here are the changes comint mode makes to the environment before starting a process if the system is using terminfo

TERMCAP=
TERM=dumb
COLUMNS=290
INSIDE_EMACS=24.5.1,comint
EMACS=true

If the system is using termcap, the values are

TERM=emacs
TERMCAP=emacs:co#290:tc=unknown:
INSIDE_EMACS=24.5.1,comint
EMACS=true

(some versions have EMACS=t instead of true)

[hvesalai]

Starting with emacs 26.1 one can define the TERM used with terminfo. If there is some other value that would make this work without changes to JLine, we could use that

See https://github.com/emacs-mirror/emacs/blob/master/lisp/comint.el#L462

[gnodet]

The code to look at would be in TerminalBuilder:
https://github.com/jline/jline3/blob/master/terminal/src/main/java/org/jline/terminal/TerminalBuilder.java#L230

I'm not sure at this point which jline Terminal implementation would be best suited for running inside emacs. Does emacs actually create a pty or is it a simple emulation of a terminal ?

[hvesalai]

TL;DR of below is that emacs is like a dump terminal (no terminal emulation) that still can interpret some ANSI control codes, mostly the colors.

The long version:

In emacs all console interaction happens with the help of comint-mode. Comint sets the environment as described above and then starts the given process redirecting the process output to a given editor buffer. Input to the process can only be sent with a special command, i.e. users input will not be directly sent to the process. This means that features such as completion are not normally available.

So to answer your question, no, there is no pty or even actual terminal emulation. Comint-mode just redirects the process output to the given editor buffer. However as emacs does support colors, there is support for interpreting some ANSI control sequences and turning them into emacs colors. Also some in-line movement may be supported, but things like movement between lines or relative to terminal screen is not supported and are not applicable.

Furthermore, while the processes output comes to the end of the buffer, the user can move the cursor freely within the buffer. As said, input to the process is sent with a special command that sends either the input given to it as an argument, or alternatively it picks up the input from the cursor location (which again, can be anywhere) using special rules (such as sending everything on the line of the cursor, except maybe a prompt that is detected with a separate regexp).

The use case for supporting emacs in JLine is that developers using emacs routinely run commands from within emacs. A trivial example is running a compilation command and getting back the compilation errors, which emacs can detect from the buffer using regular expressions and turn into links to the respective files.

An other example is support for special programs, such as the scala REPL, which is supported by the sbt-mode. In this case the mode implements some of the features available through a normal terminal, such as command completion.

[hvesalai]

The most important thing though is that while emacs comint-mode makes an attempt to suppress input echo, it does not work properly. Hence echo should be off for emacs.

[gnodet]

Here's an experimental branch 1b10052.
I haven't tested on emacs and it does need some work, but that's an idea on how to solve the problem. I think it's very similar to supporting idea and eclipse...

[hvesalai]

Added a few comments to the branch

[fommil]

@gnodet awesome, thanks for looking into this. Do you think a backport to jline2 would be feasible? The entire scala tooling ecosystem is using jline2 and it would be years before they can migrate to jline3. They are also very unlikely to make patch releases to old versions of the core tools to facilitate this change, so we will have to document how users can use a binary-compatible jline2 locally.

[hvesalai]

@fommil I was planning on making similar PR to jline2, there is already an issue where I asked if that would be ok and @gnodet said yes. We also need to work on the draft gnodet made and make it work in both Java8 and Java9.

[hvesalai]

@gnodet Is your target for JLine3 Java8 or Java9?

[gnodet]

Java 8 for now.

[gnodet]

The https://github.com/jline/jline3/compare/issue-235-ansi branch contains a JDK 8 compatible fix I think.

[hvesalai]

The branch looks good as far as ANSI. What about echo? Reading the code I gather that it should be disabled by default?

[gnodet]

Yes, there should be no echo for dumb terminals.