-
Notifications
You must be signed in to change notification settings - Fork 728
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Multiplexing more sessions into same UDP port #295
Comments
I think we could probably do this. The best idea I've had for how it would work is:
2b) If there is a /etc/mosh.conf specifying use of a multiplexing daemon but no "-p" flag, the server prints an error message telling the user to upgrade their client.
Characteristics of the design:
|
If you can think of any idea for how this could work with existing clients (by only modifying the server), I would love that. You could imagine a moshd that would take incoming packets (with no port number prepended) and somehow infer which subset of mosh-servers to send them to (maybe by tracking the sequence numbers, or maybe by letting the mosh-server request all incoming packets from a particular IP address that it's "attached" to?). It's ok to send the packets to multiple mosh-servers because the ones where it doesn't decrypt will just ignore it. But this seems like it may be a flakier design overall than explicitly indicating the port number. |
I assume the port 22 UDP proxy would add client source address/port info to the datagram before forwarding to the requested protocol. The thing about clients getting to send UDP to any port on the server is... not cool. Options include: a) the server tells the proxy what port it has open then the proxy allows datagrams only to those ports, b) define a UDP port range for this? but this gets us back to session number limits, c) ?? I prefer (a). You'll want the server to tell the proxy what port it will use in a way that other unprivileged local processes can't. |
Also (though you already knew this, it's worth restating): this is probably the single most important feature w.r.t. getting enterprises that have SSH gateways to setup Mosh gateways. Godspeed. |
Yeah, we may also want to wrap the whole thing in another encryption key (which the moshd would have), to satisfy the people who hate that we send sequence numbers and port numbers in the clear. (Even though SSH, TLS and DTLS send sequence numbers in the clear too.) |
Seems like a waste to add that additional encryption when the ciphertext will be going only over IPC to... processes spawned by the sender. What does matter is that users not be able to extract that info out of mosh servers, and that there not be a DoS as a result of re-writing the port number in the datagram sent by the client (but the latter was already a requirement). The protocol by which a server gets spawned and it then tells the proxy what UDP port it opened, that is critical. |
While this would be lovely and is for sure necessary, this:
is not a desirable feature... |
Sorry, I should be clear that this "additional encryption" would be going from the client to moshd, and would encrypt the "real" port number and sequence number. This is to satisfy the people who want there to be some ambiguity between different Mosh sessions to the same server. We probably will need to find a way to stop (4) as well (remote hosts gain the ability to send UDP packets to ANY port). |
@keithw How would there be any ambiguity, given there'll be a different remote IP and port number for each session? (Not that I have anything against encrypting the port and/or sequence numbers if someone really wants to, I just don't see what extra security it buys anyone... However point number four above would be a very serious flaw if implemented.) |
@keithw: well, you're already not getting privacy protection for the target port number (you're not using ESP). Sure, you could get a slight measure of traffic analysis protection from encrypting this in the packets to the proxy, but it's not much when you consider that the replies will come back from an identifiable port number on the server. To me this would be unnecessary complexity since now you'd have to have two sets of session keys: one for the client<->proxy and one for the client<->server. |
You don't have to tell me! Participate in the discussion on mosh-devel if you want to have an impact. People like Jacob Appelbaum and Seth Schoen want this, despite the fact that the privacy improvements will be only statistical (and will depend on the presence or absence of credible chaff). |
Guys, you reject crypto agility, and generic composition of unencumbered AEAD cipher modes on complexity grounds but would accept a dubious traffic analysis protection feature? Come now, be consistent :) |
Rather than relaying the packets to another UDP port on the local machine, why not open a UNIX socket in a specific directory (specified by the mosh.conf file) such as /var/run/moshd/60001.sock, then 60001 refers to the UNIX socket's filename, not a udp port. Would potentially break when servers are running on non-*NIX-like platforms, but you could easily allow falling back to local udp sockets in such an instance (and it's not likely to be a major use case of mosh having the server run on such platforms anyway). Another option is to have mosh.conf specify a whitelisted udp port range and only those ports can be used, the mosh-client respecting the port range too. Additional encryption seems rather silly, I'd compare the security to what ssh currently offers, if ssh doesn't offer it, there's no real gain to offering it with mosh unless it can be done for free. Encrypting already encrypted data doubles your encryption overhead, and is hardly free, not to mention any possible key negotiations required... |
A |
@nvx: +1 |
Is there still interest in implementing this via the approach described by keithw? |
Yes please. On Tue, Dec 31, 2013 at 9:38 AM, zedinosaur notifications@github.comwrote:
|
I think it would still be really nice. |
instead of the client referencing the port it could reference a session id. the id then maps to the actual port in use. that way only mosh ports can be accessed. greetings, eMBee. |
Making it a session ID will then require some sort of negotiation between "moshd" and the mosh-server. If we can encode the actual port number it keeps things simpler. I would actually modify keithw's proposal slightly and instead of just adding a static additional 2 byte value it I would abuse the second highest bit in the seq number to add a flag for whether a new, unencrypted header is present. Then after the seq number add a byte for the size of the unencrypted header. Then the unencrypted header would just be a proto and could contain the port number. These makes the overhead 5 bytes instead of 2 but makes things must easier to extend in the future. |
Assumptions:
|
What would be the appropriate permissions on that directory? World-writeable? And to make those sockets work they'd either need to be world readable/writeable or moshd would need to run as root. I think getting the permissions right here would be tricky but keeping everything on UDP makes things easier permissions-wise. |
Socket permissions would have to be fairly permissive, for example: srwxrwxrwx 1 mysql wheel 0 Jan 1 19:04 /tmp/mysql.sock= The advantage of using UNIX sockets is it immediately solves the problem of using moshd to bypass firewall rules to talk to arbitrary UDP sockets. |
OK thinking about it more this I think the socket based approach could work well. When doing the ssh handshake the mosh-server could generate a random 32-bit session id and open a socket based on that id (eg /tmp/mosh-). Then the client would embed the session ID in the packets as I described above and moshd would use that to figure out which socket to send the packet to and forward it there. Hopefully we could use SCM_CREDENTIALS to make sure we're only getting messages from who we expect. |
zedinosaur, are you already working on bringing this feature to life ? I've been thinking about this exact issue, and have just stumbled across this thread. I've got one hole punched in my firewall for mosh, and don't really care to punch any more holes. I was thinking about the simplest way to get multiple mosh connections without having to perturb the mosh codebase too much. I thought this might be possible (at least for prototyping) using the "--client" and "--server" options on the mosh client front end. I think this would allow the current non-multiplexing client and server to remain unchanged. Normally a mosh-client connects directly to a mosh-server. In the multiplexing case: mosh-client --- mux-client-endpoint <-----------------> mux-server-endpoint --- mosh-server Effectively, mosh --client="mosh-client-mux" --server="mosh-server-mux" could set this up. For example, the program mosh-client-mux program could configure and run the mosh-client and mux-client-endpoint programs on the client machine. Similarly on the server side. Once prototyped, the mosh-client-mux and mux-client-endpoint could be folded into the mosh-client, and mosh-server-mux could be folded into the mosh-server. The mux-server-endpoint would have to remain as a separate piece of functionality (effectively moshd). While this scheme nicely allows the mosh-server to remain UDP based and run with few if any alterations, the problem is that the mux layer needs to know the UDP port on which the mosh-server is listening. |
If introducing a new transport layer to mosh-client can be considered, then the suggestion to use a Unix domain socket is a good one. Using this approach, I would think about having moshd (the server endpoint) accept UDP packets from external mosh-clients, and also listen and accept Unix socket connections from local mosh-servers. The name of the Unix socket that moshd is listening would be known to mosh-server or published somewhere. When instructed to use the multiplexing mode, the mosh-server connects as a Unix socket client to moshd. All connection data is subsequently passed to and fro on this connection. Creating the connections in this way avoids the need to have a (potentially world writable) directory filled with connections, pipes and sockets that come and go. |
Hi, A few thoughts :
This should be doable with relatively little modification to either mosh-server and mosh-client with relatively thin wrappers if desired. |
I've collected my thoughts together in a more concrete manner at : https://github.com/earlchew/mosh/blob/issues/295/dev/issue-295.md |
I can't t think of a better way to do it than how you've described there @earlchew - that has my vote over all the other suggestions! Should hopefully solve the NAT problem too. |
I've added a candidate design diagram at : https://github.com/earlchew/mosh/blob/issues/295/dev/issue-295.md |
Is there any news about this very feature? |
@earlchew's proposal is the formal, correct approach to solving this problem. But I have an idea for a pragmatic, simple solution to this problem: Take @earlchew's proposal, but strip out the session ID-- moshd and mosh-server just wrap packets in the existing format with a header indicating the remote IP/port. Instead, moshd keeps a dictionary of the remote hosts sending incoming packets, and remembers which remote host any given mosh-server responds to. On receipt of a packet from a previously unknown remote host, simply send it to all connected mosh-servers. Since a mosh-server simply drops packets that aren't for its session, either one or zero mosh-servers will respond to the packet. moshd can obviously fairly quickly figure out which mosh-server is associated with that remote host. This scheme has one big win and one big lose:
|
Clever, I like it. The last lose isn't really a loss as the information is already exposed anyway. For the DDoS implication isn't really that bad anyway - in most cases the moshd would be running on the same host as the mosh-servers anyway, so no bandwidth exhaustion, just some potentially wasted CPU time (other hosts would likely have their own dedicated UDP ports forwarded). Or if there was a dedicated moshd for multiple hosts, it'd likely be on the same LAN which would have much more bandwidth than the WAN link. Additionally, moshd could be configured to rate-limit roaming attempts, and the moshd UDP port would likely be unknown to an attacker (a firewall could be configured to simply blackhole probes to unknown UDP ports, which would be the same behaviour if a packet was sent to moshd and was unknown by all mosh-clients) unless said attacker was able to successfully authenticate to the ssh server to start a mosh-server of their own to retrieve the configured UDP port for moshd which would further limit such an attack (and if an attacker can ssh to your box, chances are there's better ways they could launch a DoS attack against your things). Also worth noting, if you're expecting to support hundreds of connections, chances are you're going to have a public IP you can throw on the box anyway so you wouldn't be using moshd anyway. Usecase for moshd I see as more for smaller use. Big win for client compatibility rocks though. |
There is at least one more problem with my scheme that @earlchew's proposal avoids: an untrusted local user can connect to moshd and reply to all packets. This doesn't have to be a DoS for other local users, moshd can just forward a given remote to all repliers, but the bad actor does effectively get tcpdump on one direction of everyone else's session. The classic solution here is to restrict access to the moshd socket to setuid mosh-server binaries, but that adds a new attack surface to mosh-server, of course. |
Also, since my proxy scheme makes no attempt at authentication, it could be used as a semi-generic UDP proxy. That would be bad in some situations, if you have users that you can't trust. |
I wish I could give a look to earlchew's "diagram". |
The previous method of embedding diagrams seems to have rotted. I've updated the link and it seems to have fixed the issue. |
Any news on that front ? |
Will there be a fix for this at any given time? |
I don't think this will happen. The application would become a completely different one. |
This feature becomes tractable if we ever have a privileged "moshd" listener to route the user sessions. |
Currently mosh uses a port range to support multiple sessions to the same host. In the corporate world with many firewalls and many restrictions, opening ranges a thousand ports wide is always a hassle if not impossible. Looking at it with a security mindset it does not look much better.
So - the feature request is:
Create option/change default behavior to allow mosh to use one single UDP port on the server side to multiplex more sessions into - just like we are used to with SSH and many other protocols.
The use of UDP/22 could be considered and/or the registration of an new low port number at IANA.
Alternatively - just make it configurable.
One of the advantages of using port 22 is that it is already used for SSH and interactice shells and convincing the Mordacs of this world to allow UDP in addition to TCP would be easier.
The text was updated successfully, but these errors were encountered: