Skip to content
This repository

Scrollback and alternate screen (was: Use alternate screen on smcup/rmcup ) #2

Closed
andersk opened this Issue October 13, 2011 · 45 comments
Anders Kaseorg
Collaborator

When the remote application uses smcup to enter the alternate screen, the client should put the real terminal in alternate screen mode too, so that for example the mouse wheel lets you scroll in less and emacs and barnowl.

Anders Kaseorg
Collaborator

Or perhaps the client should always put the real terminal in alternate screen mode, for much the same reason that curses applications do—the scrollback history it generates in the primary screen isn’t particularly useful anyway.

Keith Winstein
Owner

In normal use (like line-by-line typing at the shell), the real-terminal scrollback will be somewhat useful. I'm not sure I want to give that up completely, although maybe the user would be annoyed that larger prints are only there partially in the scrollback.

Keith Winstein keithw closed this February 27, 2012
Keith Winstein keithw reopened this February 27, 2012
Keith Winstein keithw closed this March 06, 2012
Lau Ching Jun

Hi, I would like to know if you have any plan on this issue? I think mosh should work just like ssh, that it will enter smcup mode when the remote application requests so. I find it quite annoying when I use vim to open a large file and it leaves a lot of scrollbacks.

Keith Winstein keithw reopened this April 15, 2012
Keith Winstein
Owner

Here is the current proposed solution:

(a) mosh-client puts the terminal in the alternate screen for the whole time

(b) mosh-server interprets SMCUP/RMCUP the way xterm does, so people's "vim" continues to work the same as in xterm (or in screen with altscreen enabled)

(c) we grab control-pgUp and control-PgDown for our own devious ends (providing proper scrollback).

Anders Kaseorg
Collaborator

Sounds reasonable to me. I proposed the following modifications to (c):

  • we also grab Ctrl-Up and Ctrl-Down (since this is what Ctrl-wheel sends in gnome-terminal)
  • but we don’t grab any keys for scrollback while the inner application is inside an alternate screen.
Scott Gifford

Do I understand correctly that the plan here is to provide scrollback similar to what a regular terminal provides? That is, if I run a command with a lot of output, I can reliably scroll back with my terminal's scrollbars to see everything that was output by that command? That is a really important usability issue for me. Thanks!

Keith Winstein
Owner

Yes, Scott, that's the plan, although you won't be able to use your local terminal's scroll bars. For now you can use screen or tmux on the server side to get this feature.

taralx

This is a cool idea. Is it possible to support mousewheel too?

Keith Winstein
Owner

Yes, if we do it with Anders's amendment, it will support the mouse wheel.

Nick Woodhams

Agreed. This is the only significant issue with Mosh at the moment. It's the only issue that has tainted my experience.

Harm Aarts
haarts commented June 11, 2012

+1

Lucas Prim

+1

Mikaela Suomalainen
Mkaysi commented June 20, 2012
Timothee "TTimo" Besset
TTimo commented June 20, 2012

I agree. I'd much rather read a whole page of PGP garbage :)

Mitar
mitar commented June 20, 2012

I agree. I'd much rather read a whole page of PGP garbage :)

+1

(I ... just ... couldn't resist ...)

Nick Woodhams

+1 to that

Mikaela Suomalainen
Mkaysi commented June 21, 2012
m-o-e commented June 23, 2012

+1 for fixing scrollback in mosh (showstopper for me)

-2 Mkaysi

Amir Salihefendic
amix commented July 30, 2012

+1, I sometimes cat a file on the server to copy it locally and not supporting scroll back prevents this.

Vlad Andersen

+1 for fixing scrollback (the only real problem with mosh so far).

Yuval Greenfield

+1 for fixing scrollback, -1 for mkaysi, I wonder why mosh can't just behave like ssh on this. I have huge directories I "ls" all day so I have to use ssh.

Like SSH secure shell, but allows mobility and more responsive and robust.

Keegan McAllister
Collaborator

@ubershmekel: You could use screen or tmux on the remote side for scrollback. Or simply ls | less.

I wonder why mosh can't just behave like ssh on this.

I'm glad you asked. :)

SSH is simply conveying a stream of bytes from server to client; it doesn't care what the bytes represent. So the local terminal (XTerm or what-not) gets the entire history in order, and can scroll.

By contrast, Mosh tracks the current state of the terminal on the server side, and then synchronizes this representation between client and server. This is the key design decision which enables Mosh's unique features -- roaming, local echo, and better utilization of bad connections. On the other hand, it complicates some things which are simple in SSH's world of carrying bytes in order.

To support scrollback we will need to add history to this terminal state object, and augment the protocol so the client can request bits of history on demand, as the user scrolls. (Without doing it on demand, we lose a key advantage of Mosh over SSH, which is that commands which spew output will update at a fixed "framerate" rather than lagging the connection.)

This is certainly doable, and is obviously something that a lot of people want. But it's not really "just do the same as SSH" and it's not a trivial change. In the meantime, I think running screen or tmux on the server side is a fine workaround (and has many other advantages).

Mitar

I must say that running screen on the server was exactly the thing I hoped to avoid with mosh. This is what I was doing with SSH when I was afraid that connection will be broken because of IP change or something. With mosh I hoped that I could just run it and everything else would be cared automatically.

You could maybe even integrate mosh with screen. :-)

@kmcallister

I wonder if mosh could grow an alternate mode where it skips all the pty magic and just acts like regular ssh.

This would of course remove the benefit of latency optimization, but I believe many people (at least myself) are only interested in the connection-persistence anyway. In this mode a display would be restored simply by playing back the last n bytes unconditionally. Keeping track of the local/remote buffer offsets in order to trigger the right amount of playback at the right times should actually be fairly easy then...

Keegan McAllister
Collaborator

With mosh I hoped that I could just run it and everything else would be cared automatically.

You can run

mosh user@host -- screen -dR

This also helps with another problem. If a mosh-client dies unexpectedly (say, because the client machine lost power), the corresponding mosh-server will hang around uselessly. But running screen -dR will cause the other screen process to exit, killing that old mosh-server. You can of course make this more sophisticated with named screen sessions and such.

Keegan McAllister
Collaborator

@m-o-e: What you describe is doable, but I'm reluctant to add alternative modes to Mosh which fundamentally change the operating principle. Perhaps something like autossh would meet your needs?

Shad Sterling

I tried using Mosh for the connection persistence, and nothing I read prior to using it gave me a clear indication that it was anything but an ordinary shell with an improved transport. When Mosh discarded some important output I had no idea what was going on, and I wasted a few days (calendar time, several hours actual troubleshooting time) looking at everything else because it didn't occur to me that a tool for persistent connections and better latency would also throw away my data. After that, I can't trust you, not because the way Mosh works is unreasonable, but because the way it is described fails to convey that you can't count on receiving the complete output of any command. (I suspect it also fails to convey the goals that drove the decision to make it work that way, but that's not important.) At the very least, the conditions for discarding data should be given a prominent position where no new user who reads anything about Mosh could reasonably fail to miss it. If I had read such a notice before my output was lost, even if I hadn't understood the warning I would have traded those days of aggravation with a moment of "Oh that's what that meant!"

Mitar

I cannot agree with Polyergic. I think site is nicely done and also interface with the underlined characters nicely tell that there is some prediction being done. I didn't have problems of not understanding what is happening. The only thing which surprised me was that --predict=never still didn't make output work as I was used.

So I would maybe just urgue for a switch (I can then bash-alias) which would transfer everything. It can still predict.

So I see three options here:

  • prediction + not transfer everything (useful for high latency, low bandwidth links)
  • prediction + transfer everything (useful for high latency, high bandwidth links)
  • no prediction + not transfer everything (useful for just connection persistence)

I fall into 2. category. Trans-Atlantic links to Europe have 100ms latency, but they are fast. I would rename mosh to tash in this case: trans-atlantic shell. :-)

@kmcallister

Emulating a terminal (screen) inside an emulated terminal (mosh), in order
to emulate native functionality (scrollback) of the enclosing terminal emulator
(xterm, iterm) does not and can not possibly result in an acceptable user experience.

That is the underlying problem here. Of course neither is mosh to blame for
30 years of stagnation in the terminal emulator business, nor can it cleanly
resolve the situation from where it's currently sitting in the stack.

Hence my proposal of a 'dumb-mode', as a short-term bandaid
for the next few decades.

Shad Sterling

@mitar, I don't see any disagreement between what I said and what you said; restating things tersely, what I see is this:

I said: Mosh loses data in an unexpected way; descriptions of Mosh should set expectations to match its behavior.

You said: The Mosh website is nice; the prediction indicators are nice; an option to transfer everything would be nice.

What am I missing? Did you see a clear notice somewhere that Mosh won't keep all of the output of your commands?

Keegan McAllister
Collaborator

@m-o-e

Emulating a terminal (screen) inside an emulated terminal (mosh), in order to emulate native functionality (scrollback) of the enclosing terminal emulator (xterm, iterm) does not and can not possibly result in an acceptable user experience.

Well, could you describe what about the user experience is unacceptable? This would be useful information for us.

In general, the world of computers is full of apparently ridiculous levels of nesting and emulation which nevertheless provide a perfectly fine user experience.

@kmcallister

Foremost the frankenstack does not support native scrolling. That's a showstopper.
'copy-mode' and virtual scrolling hacks are no substitute. There's also a plethora of more subtle
issues, but you (as the author of a terminal emulator) don't need me to tell you about those. ;)

As said, I'm not blaming mosh for the ridiculous terminal-situation of today - that's been dire
since long before mosh even existed. Given the project scope mosh had little choice other
than crouch on the shoulders of gnomes, like everyone else, and for that it isn't even faring too bad.

I'm just saying that with the supposed 'dumb'-mode mosh might work for those of us who have
no problems with latency and who would like the persistence, but can not tolerate a broken scrollback.

A bandaid on top of the bandaid, so to say.

Just until someone finally takes a heart and supersedes VT100 terminal emulation altogether...

Tim Spencer

I, also would love to have a "dumb" mode that lets me use normal scrollback in my terminal window to look at the history of what I did before or scroll through the output of a large command.

I haven't done vt100 stuff since my BBS days, so please feel free to smack me down, but couldn't you do both things? Every time your screen state changes in such a way that you have to scroll a line up off of the screen, couldn't you just go to the bottom and do a cr/lf, scrolling the top line off the screen and into scrollback, then continue syncing the screen state like you do now?

Clearly, we could lose some output if we go offline and stuff scrolls off the page on the server, but I'd be OK with that. So long as it worked while I'm online and interacting with the shell, I'd be happy. I'd be even happier if you kept track of when stuff scrolled off the top on the server side that we didn't see and scrolled off some sort of marker like "mosh: 60 lines scrolled on server while you were offline", but that's certainly in bonus round territory. :-)

To be clear, I'm expressing a preference here. If it's hard, and conflicts with your goals, then that's fine too. I'll use mosh regardless, because the stateless connection stuff really complements the way I work. :-) Thanks for creating this!

Benoit Sigoure

@timspencer I think what you're suggesting sounds reasonable but may not be easy to implement. My understanding is that mosh doesn't keep track of the byte stream, it just looks at the current state of the terminal and computes a diff with the previous state, and sends updates to the client, periodically. If you cat /dev/urandom, the screen will be updating very quickly, but mosh won't send everything to the client, it'll just take regular snapshots and send diffs. At least that's my understanding, I haven't read the source code.

Having said that, it would be awesome indeed to have a streaming mode where mosh just streams everything and only does connection persistent plus its prediction things.

Anders Kaseorg
Collaborator

Can we separate this into two issues? The issue I reported is that mosh should put the terminal into alternate screen mode. This is a clear bugfix that we can implement today. It’s required for gnome-terminal’s mouse-wheel-to-arrow-key emulation in curses applications. (Don’t confuse that with the fancier xterm mouse-event protocol that nobody uses.)

The second issue that’s grown out of this thread is that mosh doesn’t support useful scrollback history. It’s true that one side effect of putting the terminal into alternate screen mode would be that mosh’s useless scrollback history would become inaccessible. I consider that a feature, not a bug. If we later want to go put a lot of work into designing a system to add useful scrollback, that would be awesome. But that’s a totally separate feature request, and it’s not needed for what I understand to be the common case where the user is just running screen inside mosh anyway. I propose reopening #122 for that.

We should fix mosh now to put the terminal into alternate screen mode, with a command-line switch to disable it (like less -X) for people who think broken scrollback is more important than a working mouse wheel.

Keith Winstein
Owner

Anders, ok, I'm sold on this for 1.2.4 if it just means adding smcup and rmcup to Terminal::Emulator::open() and Terminal::Emulator::close(). Would you be willing to submit a pull request? (I assume we want the actual terminfo calls to remain in src/terminal/terminaldisplayinit.cc).

davidjao

I consider gnome-terminal's mouse wheel to arrow key emulation to be a horrible bug. There are lots of people who agree with me: https://bugzilla.gnome.org/show_bug.cgi?id=518405

Unfortunately, the GNOME developers do not agree with the bug interpretation, and neither it seems does andersk. I will try to convince you (futilely, I suspect) that it is indeed a horrible bug. The reason why mapping wheel events to arrow keys is bad is because scrolling the mouse wheel is supposed to be a nondestructive operation. My mental model of the mouse wheel is that scrolling the wheel is supposed to only change your view of the terminal scroll buffer. It should never alter the terminal state itself. Unfortunately, in many situations, such as IRC clients and many other scenarios which are mentioned in the GNOME bugzilla thread, the GNOME behavior of emulating arrow keys does in fact result in permanent loss of an application's input buffer state whenever the mouse wheel is bumped. For example say I am in the middle of typing a line when I bump the mouse wheel. The mouse wheel infuriatingly triggers a keyboard arrow press, which wipes out the line that I was typing. Fixing the individual applications is problematic given the large number of applications that exhibit the problem.

The fact that mosh fails to use alternate screen mode has been a huge blessing for me, because it means that accidental mouse wheel events don't destroy IRC client state from inside mosh. Since all of my IRC client usage occurs within mosh, this (unintended?) feature of mosh completely neutralizes the underlying GNOME bug for me. I would be very sad to see this behavior go away.

If you do choose to implement this completely demented idea, please (as andersk suggests) provide a switch to disable it. It's not that I prefer broken scrollback. It's that I hate data loss above all other considerations, and unwanted arrow keys result in data loss.

(Regarding the second issue in this bug, wherein mosh causes data loss of scrollback contents, I of course strongly support any efforts to change mosh to preserve scrollback data. There is nothing in the world that is more important than avoiding data loss.)

Anders Kaseorg
Collaborator

davidjao: gnome-terminal already provides a checkbox (Edit → Profile Preferences → Scrolling → Use keystrokes to scroll on alternate screen) to disable the arrow key emulation entirely. But nobody disagrees that there might as well be a command line switch for those who need to work around this on a per-application basis for some reason.

davidjao

This checkbox only exists on Ubuntu and derivatives of Ubuntu. It is not included in the upstream gnome-terminal, and other distributions such as Fedora, Debian, Arch, etc. do not provide it. Of course you can patch it in yourself, but that is a pain.

I understand that we all agree a command line switch is desirable but I am worried that it might be left out or regarded as unimportant. That's why I wanted to comment on why it is important.

Anders Kaseorg andersk referenced this issue from a commit in andersk/mosh January 27, 2013
Anders Kaseorg Put the real terminal in alternate screen mode
Closes #2

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2b8ca53
Keith Winstein keithw referenced this issue from a commit January 27, 2013
Anders Kaseorg scripts/mosh: Add --no-init option to disable alternate screen mode
Signed-off-by: Anders Kaseorg <andersk@mit.edu>

Closes #384. Closes #2.
ea3ad78
Keith Winstein keithw closed this issue from a commit January 27, 2013
Anders Kaseorg Put the real terminal in alternate screen mode
Closes #2

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
ed42d31
Keith Winstein keithw closed this in ed42d31 March 10, 2013
davidjao

Thanks everyone, the --no-init option works perfectly to disable alternate screen mode for those of us who don't want it.

In the spirit of keeping the documentation up to date, could we mention in the man page (in the ENVIRONMENT VARIABLES section) that the MOSH_NO_TERM_INIT variable inhibits smcup/rmcup?

Simón

For me the "--no-init" option doesn't resolve the scrollback problem with mouse wheel.

Antoine Pitrou

"--no-init" is completely broken here with the XFCE terminal: some part of the output is forgotten in history, and the scrollbar gets updated inconsistently. Why can't mosh just behave as ssh does? Being opinionated is fine, but it would also be nice to benefit from the networking improvements without having different terminal emulation decisions forced on the user.

dtenenba

The

mosh user@host -- screen -dR 

workaround works. What would be the equivalent command for using tmux instead of screen and ensuring no tmux processes are left behind when I disconnect?

Thanks.

dtenenba

Actually I spoke too soon. The screen workaround has issues. If I use it in two separate terminal windows, one after the other, the second window hijacks the screen in the first window. Would be great to have scrollback to eliminate the need for this. Thanks.

Anders Kaseorg
Collaborator

@dtenenba, this is not a support forum for tmux and screen. Read the manpages (tmux(1), screen(1)), especially the parts about the -d option to tmux attach-session, and the -d, -r, -x options to screen.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.