vim problems when TERM=xterm, but okay with TERM=linux

[lihop]

[richard-hajek]

Huh I guess xterm supports some tokens basic ASCII terminals dont have to and vim tries to use it and breaks it? Probably best to print what kind of control sequences it uses in each case

[lihop]

There also might be some issues with the way pseudoterminal is being set up. In the libtsm repo demo, I see some flags being set here https://github.com/Aetf/libtsm/blob/8d299ab6ba326d744d4359b37252878ab4c44ba9/src/shared/shl-pty.c#L104-L132.
I don't know where I got these flags from https://github.com/lihop/godot-xterm/blob/master/addons/godot_xterm/native/src/pseudoterminal.cpp#L48-L54, but it looks like they are not even being used. So I will take a closer look at pseudoterminal again soon.

[lihop]

Firstly, opening vim with .vimrc disabled (vim -u NONE test.txt) improves things a little bit. I have lots of things in .vimrc which probably aren't helping the situation:

Connecting godot-xterm via socat in verbose mode shows the following characters being sent when opening and quitting vim:

\r< 2021/06/21 20:27:40.626945  length=2 from=7436 to=7437
\r
< 2021/06/21 20:27:40.653986  length=71 from=7438 to=7508
.[?1049h.[22;0;0t.[>4;2m.[?1h.=.[?2004h.[1;5r.[?12h.[?12l.[22;2t.[22;1t< 2021/06/21 20:27:40.654278  length=47 from=7509 to=7555
.[27m.[23m.[29m.[m.[H.[2J.[?25l.[5;1H"test.txt"< 2021/06/21 20:27:40.654367  length=69 from=7556 to=7624
 0 lines,\r\r
0 bytes\r\r
Press ENTER or type\r\r
command to continue.[?25h


> 2021/06/21 20:27:51.009649  length=1 from=426 to=426
.> 2021/06/21 20:27:51.626827  length=1 from=427 to=427
:< 2021/06/21 20:27:51.627353  length=153 from=7625 to=7777
.[27m.[23m.[29m.[m.[H.[2J.[?25l.[2;1H.[1m.[34m~                   .[3;1H~                   .[4;1H~                   .]2;test.tx...) - VIM\a.]1;test.txt\a< 2021/06/21 20:27:51.627476  length=35 from=7778 to=7812
.[1;1H.[?25h\a.[?25l.[5;1H.[m:.[?25h> 2021/06/21 20:27:55.363752  length=1 from=428 to=428
q< 2021/06/21 20:27:55.363891  length=1 from=7813 to=7813
q> 2021/06/21 20:27:55.584403  length=1 from=429 to=429
!< 2021/06/21 20:27:55.584590  length=1 from=7814 to=7814
!



> 2021/06/21 20:28:00.460234  length=1 from=430 to=430
\r< 2021/06/21 20:28:00.460406  length=41 from=7815 to=7855
\r.[?25l.[?2004l.[>4;m.]2;urxvt\a.]1;urxvt\a< 2021/06/21 20:28:00.460467  length=101 from=7856 to=7956
.[23;2t.[23;1t.[22;2t.[22;1t.[23;2t.[23;1t.[5;1H.[K.[5;1H.[?2004l.[?1l.>.[?25h.[>4;m.[?1049l.[23;0;0t< 2021/06/21 20:28:00.461831  length=52 from=7957 to=8008
\r\r
.[1;32m[.]0;leroy@laptop: ~\aleroy@laptop:~]$.[0m > 2021/06/21 20:28:33.449693  length=1 from=431 to=431

Of interest is the "ESC [>4;m" sequence. On other terminals it doesn't seem to do anything, but on godot-xterm it is being mistaken for the underline command:

xterm:

rxvt-unicode:

godot-xterm:

Searching "vim >4;m" shows quite a few posts about this problem which started popping up after version 8.1.2134 of vim was released in Oct 2019 (~6 years after the last stable release of libtsm). This answer has quite a good explanation: https://vi.stackexchange.com/a/27400.

Documentation for the escape sequence in question, which is xterm specific:

CSI > Pp ; Pv m
CSI > Pp m
          Set/reset key modifier options (XTMODKEYS), xterm.  Set or
          reset resource-values used by xterm to decide whether to
          construct escape sequences holding information about the
          modifiers pressed with a given key.

          The first parameter Pp identifies the resource to set/reset.
          The second parameter Pv is the value to assign to the
          resource.

          If the second parameter is omitted, the resource is reset to
          its initial value.  Values 3  and 5  are reserved for keypad-
          keys and string-keys.

            Pp = 0  ⇒  modifyKeyboard.
            Pp = 1  ⇒  modifyCursorKeys.
            Pp = 2  ⇒  modifyFunctionKeys.
            Pp = 4  ⇒  modifyOtherKeys.

          If no parameters are given, all resources are reset to their
          initial values.

Source: https://invisible-island.net/xterm/ctlseqs/ctlseqs.html

In conclusion, the fork of libtsm used by godot-xterm should be patched to handle or ignore this sequence, or godot-xterm should not lie about being xterm in the TERM env var. However, for the second option I'm not sure what TERM type godot-xterm should masquerade as instead. Another option would be creating a custom termcap/terminfo entry, but I think it would be quite a lot of work verifying all the capabilities and users would also need to install this entry in their database (e.g. /usr/lib/terminfo/...) making godot-xterm quite a bit less portable.

[lihop]

\e[12?$p was the other problematic escape sequence (with .vimrc enabled) that was causing the terminal to be reset.