Support X forwarding

[keithw]

In theory, we could easily support X forwarding (with roaming!) by just conveying a verbatim octet stream in both directions over the underlying network layer -- the same kind of statesync object that we use for user input.

We wouldn't get predictive local UI, but we would get the other benefits of Mosh (support for sleep/wakeup and IP roaming).

[dt]

being able to setup arbitrary octet streams could be useful for port forwarding and agent forwarding too.

[peterjeremy]

There are techniques for optimising X11 over WANs that can significantly reduce bandwidth requirements - eg Xpra (http://www.xpra.org). That said, X11 over a 3G link is going to be painful at best because it's not possible to remove the RTT between input & display. Note that most Xclients need lots of RTTs to start up.

I support the idea that SSP be enhanced to allow multiple datastreams over a single "connection".

[staticfloat]

I'm very interested in this, @keithw, can you give a short outline of what work needs to be done to support this? Tunneling and X11 forwarding are high on my "want" lists from mosh (roaming tunneling over ssh is a dream come true) so I'd contribute what I can.

[keithw]

The biggest blocker at this point is probably coming up with a channel for arbitrary congestion-controlled octet streams that roam and handle intermittent connectivity gracefully. Then we can send anything (including reliable X protocol messages or other arbitrary stream) over that.

[TheSkorm]

+1
If you allow IP across mosh, then TCP would handle data loss, and UDP applications should already know how to deal with data loss. Great for easy VPN setup.
SOCKs proxy would be pretty nice too.

[suan]

+1, and this would make vim's clipboard=unnamedplus clipboard syncing work, which is HUGE.

[phmarek]

I'd like to ask for only-when-currently-connected stream forwarding, to avoid using much memory when the client is not available.

My use case would be dbus notifications, sent by irssi within a screen (accessed via mosh); when the client is attached, these should pop up as usual.

[ukd1]

Any plans for this @keithw?

[mgalgs]

We wouldn't get predictive local UI, but we would get the other benefits of Mosh (support for sleep/wakeup and IP roaming).

You're forgetting the biggest benefit that this feature would add: X11 Forwarding! :)

[jinpoc]

+1

[githubhjs]

+1

[twilightsparklehf]

+1

[rodneyrod]

So what's the progress on this feature?

[eminence]

I do not believe that anyone is currently working on this

[brainstorm]

Related issues: #120, #337 ... there's even a $50 bounty for that last one.

[virtualdxs]

Guys can we stop spamming people with +1

[ghost]

Current I use ssh to initial the connection:

ssh -YCt myserver tmux a

Then after that I just use mosh:

mosh myserver -- tmux a

Now I can use X application (gtkwave) in the mosh session. Is that a kind of X forwarding? Do I loose any great feature provided by mosh by doing that?

I tried to merge these two step into on by:

mosh --ssh="ssh -YC" myserver -- tmux a

But the X forwarding doesn't work.

[computergeek125]

I tried your command on Ubuntu 16.04 and Centos 7, and I got a "no session" error. I don't have much experience with tmux, so I can't help you there.

However, I did find this:
When executing mosh --ssh="ssh -YC" server, I can see that mosh seems to only use the ssh session to initialize the mosh session. Once mosh is running, ssh is nowhere to be found in ps. X11 forwarding requires SSH to be running the whole time.

[stites]

Another error I'm seeing when running mosh --ssh "ssh -YC" server is xset: unable to open display "localhost:10.0" (10.0 being the prefix of the server in this case). Would this be caused by the same problem of ssh stopping for you, @computergeek125 ?

[computergeek125]

Oof, that was a while ago.

Recreating my old setup, yes, I did get that.

Can't seem to get my exact tmux error back, but that's probably my lack of tmux knowledge speaking again.

[andersk]

That is all expected. Mosh uses SSH only for the initial login and closes the SSH connection immediately after starting mosh-server.

(Even if Mosh were to try to leave the SSH connection open, it would just be a normal SSH connection over normal TCP. It wouldn’t gain any ability to survive network disconnections and roaming the way Mosh users expect. There would be no advantage over just using SSH directly. Even worse, it would potentially hide problems like #730 where Mosh would only live as long as the SSH session and appear unreliable in a way that’s difficult to debug.)

[tomachinz]

i had an idea - how about firing up a new "proxy" ssh connection each time mosh either detects an IP address change or somehow when the client requests data...... say you were monitoring the IP address and the mosh client detects a change in gateway etc. it could check for parameters like -D 8888 (dynamic socks proxy), -L local port forwards, -R remote etc. perhaps this would need to terminate the older ip address processes to free up the local ports for the new ssh proxy process???

i understand the reason mosh does not support X11 is that mosh is a virtual TTY terminal emulator. but perhaps mosh can start ssh client sessions in the background to support custom parameters? this is one of my connect strings:
ssh -i TrustedNetworks.pem root@wherever.co -L *:10022:localhost:10022 -L *:12511:localhost:12511 -L *:15801:localhost:15081 -L *:15901:localhost:15901 -L *:19802:localhost:19802 -L *:20022:localhost:20022 -L *:22511:localhost:22511 -L *:25801:localhost:25081 -L *:25901:localhost:25901 -L *:29802:localhost:29802 -L *:40022:localhost:22 -L 8888:127.0.0.1:80 -L *:40022:localhost:40022 -L *:42512:localhost:42512 -L *:45801:localhost:45081 -L *:45901:localhost:45901 -L *:49802:localhost:49802 -L *:20022:localhost:20022 -L *:22511:localhost:22511 -L *:25800:localhost:25800 -L *:25900:localhost:25900 -L *:29802:localhost:29802 -L *:28080:localhost:28080

yep thats a string i used to use at some stage in my life.

@keith you mention you thought the main blocker to be needing to have:

...a channel for arbitrary congestion-controlled octet streams that roam and handle intermittent connectivity gracefully...

excuse my ignorance, but.... correct me if wrong but I understand Mosh is currently doing it's awesome magical business regarding intermittent and dynamic IP environs by maybe setting up a UDP listening port and wait for a listener that is able to report the PID code / authentication token that matches a live running user session on the box in question. how about passing that string straight through to the server yeah? after cleansing it for XSS vulns. :)

[lilydjwg]

@tomachinz Use autossh to persist the connection? Or use systemd.socket to connect on demand?

Currently I'm using mosh for the terminal and ssh for X forwarding. It mostly works, except that I have to start the ssh tunnel every day (I'm using password authentication for this box). My ssh connection is as stable as mosh because it uses the Wireguard VPN, and mosh is still great in this use case for its quick feedback.

I don't want mosh to use my ssh in its own way. It's already disturbing because the ssh connection mosh has used isn't multiplexed so I have to connect again when I need it. (I don't use the mosh script for high latency hosts for this reason.)

[computergeek125]

@lilydjwg Interesting thought. Going that direction, I'd rather see autossh than systemd sockets for the implementation of this. One of the best parts of Mosh is how platform agnostic it is. If you place a dependency on systemd, that limits it to systemd-based Unix and Linux systems- potentially locking macOS, Windows, Cygwin, Windows Subsystem for Linux, and even init-based linux systems out of this feature.

As for whether it might work, I'd say it's at least a good start. SSH will automatically tear down a connection it thinks is broken (albeit after some delay), and autossh can start it right back up with the new IP. This solves the problem of keeping a socket "open" across IP changes, but it still leaves up the problem of a persistent connection. When autossh respawns a connection, it's not the same connection and some software might not handle this well.

Something like http(s) or RDP traffic that can handle a socket getting reset would be fine, but I don't know if X11 is as fault-tolerant (my knowledge is stronger in OSI layers 2-6 rather than layer 7 protocols).

On topic of tunneling connections, auto-respawning a tunneling ssh from mosh is related to various proxy/jump features like #970 #285 (there are a lot of ProxyCommand and jumphost github issues floating about and probably at least half of them are duplicated).

[tomachinz]

@lilydjwg actually, not at all, my thought was much more primitive - to literally pipe the users command text straight through to... is it the shell interpreter? eg... from the client side.

mosh -Y tomachi@server

so on the other client end, the mosh client runs local:

ssh -Y tomachi@server

I can see there is more to it than meets the eye. Or perhaps it could all be done in the client? It's the client that needs to free up listening ports in some cases, but also the server may need to free up remote listening ports if there is a drop-out.

I reckon a handshake / or reset command between client and server is needed: upon reset both the client and server seek and destroy the old ssh connection; 2 seconds later the client retries the ssh outbound again. That would work for my tunnels: local, remote and dynamic AFAIK.

Here you can see me switch from local LAN connection (10.0.0.1) on my phone to GSM mobile (118.148.204.212). mosh is uber cool really....

By running this command I did some debug:

netstat -Waltnpuc | grep -E "ssh|mosh"
You can see a mosh connection start and finish with this command, but not see it change from LAN to Wifi and back and forth. Finally disconnection I see the UDP server cut off.

I guess X11 uses export display etc hadn't thought of that...

[tomachinz]

Or leave the sshd process running in a background thread. Then kill it during the reset.

[computergeek125]

@tomachinz I've done nested X11 tunnels (ssh -tY jump ssh -Y server) before I learned how to do jump hosts (ssh -J jump -Y server), so from an SSH perspective that works, the problem we need to address here is the link between the X11 agent[?] on the server side and the X11 server on the client side.

AFAIK (correct me if I am wrong), all ssh -Y does is tell the programs running on the server to send their rendering calls to the SSH tunnel rather than a "locally" running X11 server attached to a real or virtual graphics card. After traversing the SSH tunnel, the SSH client spits the rendering calls out into the X11 server running on your ssh client machine (which, in the case of macOS, autostarts XQuartz)

I think picking up the calls on the server side and (once they're on the client side) getting them to the local X11 server is fairly straightforward- the problem is getting them over the network between the server and client.

IIRC, X forwarding is a special case for SSH's native TCP forwarding. Way back in 2012 when this thread was started, @keithw commented:

The biggest blocker at this point is probably coming up with a channel for arbitrary congestion-controlled octet streams that roam and handle intermittent connectivity gracefully. Then we can send anything (including reliable X protocol messages or other arbitrary stream) over that.

That's the problem we need to solve.

The premise of TCP is to have a reliable connection layer where everything is (mostly) guaranteed to arrive at the application layer. If this doesn't happen, the stream breaks or delays data until the whole stream can arrive in-order. TCP by itself has limited buffers, so can only sustain limited disconnect periods before needing to destroy and recreate the connection. It might be a bit problematic, since the mosh-server and mosh-client would likely have to buffer the data on both sides of the pipe, then replay the buffers when a connection is reestablished. All that without even talking about congestion control.

VPNs (like Cisco IPSec) can deal with TCP over UDP. Possibly worth investigating how they do that?

[trygveaa]

+1, not sure why people are downvoting +1s?

Because it notifies everyone that follows this issue and contributes nothing to the discussion.

There isn't any alternative way to express our interest in this feature.

Yes there is, add a thumbs up reaction to the first post in the issue.