Support all of xterm's control sequences [$100]

[p-e-w]

This is a meta-bug that summarizes the fact that Final Term does not yet have full support for all control sequences defined in the xterm specification (http://invisible-island.net/xterm/ctlseqs/ctlseqs.html) which causes a number of complex shell programs to display their output incorrectly.

Since these errors are in fact unrelated to the programs in question and all caused by the lack of control sequence support, the related issues will be linked to this one and closed as duplicates.

There is a $100 open bounty on this issue. Add to the bounty at Bountysource.

[MarkRijckenberg]

I am also impacted by this issue (issue #121). The alsamixer application is not working correctly, another ncurses based application.

[godeater]

Is the idea to write all the escape sequence handling again from scratch? Or would using a library which already does it be preferable?

https://launchpad.net/libvterm/ is a nearly feature complete implementation, leonerd, it's author and maintainer is highly responsive, and active on Freenode (can be found in #perl regularly).

[p-e-w]

@godeater: That is a very interesting library which I did not know about at all until now. Were I starting to write Final Term now, I would definitely give it a try.

Two comments on the current state:

• The part that libvterm handles (terminal output parsing) is already fully complete in Final Term (https://github.com/p-e-w/finalterm/blob/master/src/TerminalStream.vala) and, I believe, solved in a very nice manner 100% conformant to the xterm specification. libvterm actually seems to parse only a subset of that specification according to http://bazaar.launchpad.net/~leonerd/libvterm/trunk/view/head:/doc/seqs.txt. Notably, it's apparently lacking support for 24-bit color SGR sequences, which Final Term already has.
• The difficult part is not parsing, but interpreting the resulting sequences, i.e. translating them into screen actions (difficult mostly because there is little to no documentation for it). libvterm seems to offer that too, with its own TTY model, but since Final Term needs some non-standard actions to provide its semantic features, that part most likely has to be coded from scratch.

One trivial example where xterm-style interpretation is definitely not what I want in Final Term is line wrapping (otherwise "terminal reflow" would be impossible to achieve).

Nevertheless, I shall have a closer look at libvterm!

[godeater]

Not sure if this is the right place for a back-and-forth conversation on this, but in light of having nowhere else to do it :) :
leonerd has also written pangoterm, using libvterm (and pango, obviously!) - so perhaps see if you can get him interested in final-term too ;)

[p-e-w]

Thank you again, pangoterm looks awesome! It even supports double-height and width lines 😮, which cannot be said for many terminal emulators (I certainly hadn't planned to support it in Final Term, but if the competition has it... 😸).

Fascinatingly, this blog post by LeoNerd also describes optimizations he made to his terminal infrastructure that so closely mirror some of the performance improvements I pushed in the past week that it's almost uncanny... I'd love to hear LeoNerd's thoughts and input on Final Term, I'm sure there's a lot I could learn from both him and his code.

Will contact him...

[solusipse]

Irssi is another ncurses based application that is unusable in the current state.

[aurium]

Waiting for this issue closed to make finalterm more then a cool toy. :-)

[glittershark]

Have you considered using vttest to ensure interpretation of all xterm control sequences? The website says that the source code should be self-documenting, and that it was used by xterm itself to ensure feature-completeness as far as control sequences go.

[p-e-w]

@glittershark: I am aware of vttest but much of what vttest tests for is actually irrelevant in practice (such as double-height lines, which are not supported by most of the established terminals either), while many important features (e.g. related to GUI scrolling) are in turn not tested for.

[ibqn]

I have tried to use vim with the newly built finalterm, and have discovered that it's still absolutely not working with it :-/

[HerringtonDarkholme]

Is there any possibility to fix this bug?

[p-e-w]

@HerringtonDarkholme: Yes. All remaining xterm control sequences must be implemented.

[IonicaBizau]

@p-e-w What's the progress on this? I'm interested mostly in VIM support.

[p-e-w]

@IonicaBizau: VIM is not working in a usable manner yet.