TUI cursor motion, SGR, and scrolling optimizations

[jdebp]

This is #6793, rebased.

[justinmk]

LGTM, @jdebp are you ready for this to be merged?

---

For reference: This felt slower to me when scrolling, but all attempts to measure it show no significant difference.

bash:

time printf 'j%.0s' {1..500} | build/bin/nvim runtime/filetype.vim -u NORC -i NONE --cmd 'set cursorline' -c 'sleep 100m' -c redraw -c '0' -c 'vsplit' +qa

python RPC connection:

import time; s=time.time(); [n.input("j") for i in range(1,500)]; n.eval('1+1'); time.time()-s

[aktau]

LGTM, although I must admit that I'm not a VT protocol warrior, so my LGTM is worse than the author taking a second look. Would be great if @leonerd could take a look, but if this is tried out in a couple of terminals (tmux, xterm, urxvt, gnome-terminal, iTerm2, Terminal.app). I've tried the latter two + tmux and it seems fine (anything I should pay more attention to?).

[jdebp]

iTerm2, Terminal.app are particuarly useful for you to test out @aktau, because they are two that I am unable to test. Look in particular for scrolling problems with vertically and horizontally split windows, and oddities in changing the cursor shape when entering and leaving insert mode; both with and without screen/tmux in the mix. With screen/tmux, check that only the current pane is affected by shape changes.

Check over SSH. Ensure that you transfer the XTERM_VERSION, COLORTERM, and KONSOLE_* environment variables across from client to server. Also try not doing that, of course.

Or go straight to #6816 and do this.

[aktau]

I've tested iTerm2 and Terminal.app locally here as to your suggestions (no tmux, no ssh). The result matrix:

term	scrolling	cursor
iterm2	OK	OK
Terminal.app	NOT OK	OK

So, Terminal.app had issues scrolling. I split my window into 4 panes (one in each corner):

| 1 | 2 |
| 3 | 4 |

What seemed to happen is that scrolling in pane 1 also scrolled pane 2 (and vice versa). The same for panes 3 and 4. This causes corruption in the screen state. See the screenshots at the end [1].

This is macOS Sierra, and the $TERM variable when I'm insude Terminal.app is:

13:22:49 neovim/neovim git:(merge-pr-6801) » echo $TERM
xterm-256color

[1]:

[jdebp]

This is as #6802 leads me to expect. Now try with #6816. Your matrix should be success in all cases.

[aktau]

This is as #6802 leads me to expect. Now try with #6816. Your matrix should be success in all cases.

Indeed, Terminal.app works correctly with #6816 (and so does iTerm2). Though I do notice sluggishness when scrolling one of the panes in iTerm2 now. Terminal.app seems nice and responsive.

The fact that it's exactly the same Neovim version + configuration makes me think it has something to do with the terminal.

However, I stopwatch timed scrolling to the bottom of the LICENSE file and these are the results:

• iTerm2/6816: 10.54s
• iTerm2/master: 10.26s
• iTerm2/6816: 10.27s
• Terminal.app/6816: 10.36s

Basically saying that either I'm imagining things or that Neovim always scrolls equally fast, but doesn't uphold the same framerate while doing so.

Anyway, everything seems good, especially from a functionality point of view.

[justinmk]

@aktau That's exactly the same observation (and measurement outcomes) I had :) #6801 (comment)

[aktau]

@jdebp, any idea about the sluggishness on iTerm2 (but not Terminal.app)? If two people saw it, it may be a real thing. How do we properly debug this? Does your newest rebase change anything there?

Also, do we need to keep this PR open if #6816 seems to subsume it? (Since there's commits in #6816 that fix bugs introduced here, I wouldn't merge it separately in order to have a more bug-free history).

[jdebp]

The rebase was just a rebase, and shouldn't have changed anything. At this point it seems wise to focus on #6816 because that has all of this and a lot more.

I suggest that you try the latest in #6816, and measure there. We seem to have finally pushed iTerm2 past the highly faulty stage, although I am still unhappy about some of the bizarre output that is in the syscall traces provided by @choco there that I do not yet have a satisfactory explanation for, and about @choco's one (I understand) remaining problem which is backspacing in command editing mode around the edge of the terminal.

I suspect that the two are related, but I have yet to grasp how. @choco's syscall output looks like malformed multibyte characters of some sort, following a cnorm sequence. I don't see a code path where anything can follow a cnorm sequence at the tail of a write() system call.

[gnachman]

If you're running into problems with iTerm2, I'd like to know if there are bugs or performance improvements I can make to help you. Especially if there are ways the terminal emulation differs from Terminal that aren't justifiable! Logs from script that produce incorrect output ought to be enough for me to track down the issue.

[justinmk]

This PR is a subset of #6816, which was merged.