Proxy unix socket?

[takluyver]

Proposed change

Where jupyter-server-proxy launches a process, allow a configuration option to specify that the process will listen on a Unix socket rather than TCP/IP. By default, I would assume that JSP creates a temp directory and invites the child process to make a socket inside it.

Our JupyterHub instance (optionally) runs single-user servers on a shared node, with each user's server running in their own system user account. In this scenario, I believe that once I've launched a proxied server, any other user can connect to it and make requests, either by going to /proxy/<portno>, or by running arbitrary code from a notebook or a terminal. I'd be happy to hear I'm wrong about this! Unix sockets can have permissions like files, so if you get these right, only the user a socket belongs to can connect to it.

This also avoids the race condition where the parent picks an unused port, but then something else binds it before the child can. The parent process instead creates a temp directory to pass to the child.

This could also be extended to the explicit proxying, to allow a Unix socket path in place of host+port. But that gets more complicated, because you need to work out how much of the URL is the socket path, and how much is the path to forward in the HTTP request.

This was touched on in #1, but the focus of that is different. https://gist.github.com/bdarnell/8641880 contains an example of using tornado AsyncHTTPClient with a Unix socket - it probably needs updating, but the tornado docs look like something similar should still work.

Alternative options

• Send some secret with the proxied requests which the application can check to validate that a request came through the right user's proxy server. I think this is already possible by configuring request_headers_override. But it relies on every application which might be sensitive implementing the check, and all parties ensuring the secret doesn't leak. Relying on the OS to block connections to Unix sockets seems safer - though I'm not a security expert.
• Ask the user to authenticate again in the proxied app, and use standard web app mechanisms to manage that. But part of why I'm interested in JSP is to avoid dealing with authentication, because users are already signed in to JupyterHub.
• Document the risks of using JSP where multiple users can share one node. 😞

Who would use this feature?

• Developers who want to expose a web app to a single JHub user, without implementing security mechanisms themselves
• Users who want to run a web app through JHub for their own use without letting other people act on their behalf

(Optional): Suggest a solution

• Check if the client code in the gist still works, update it as necessary
• Define the config for using a Unix socket and how it works with process launching (re-use {port} for the socket path? Or a new name?)
• Integrate support for launching a process with a Unix socket and proxying requests to it.

[welcome]

Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! 🤗

If you haven't done so already, check out Jupyter's Code of Conduct. Also, please try to follow the issue template as it helps other other community members to contribute more effectively.

You can meet the other Jovyans by joining our Discourse forum. There is also an intro thread there where you can stop by and say Hi! 👋

Welcome to the Jupyter community! 🎉

[takluyver]

BTW, I realised this while writing a minimal example of a proxied application to check that I knew what was going on: hello_jupyter_proxy.

[manics]

In this scenario, I believe that once I've launched a proxied server, any other user can connect to it and make requests, either by going to /proxy/, or by running arbitrary code from a notebook or a terminal.

Yes, that's correct. I thought the docs said (or used to say) this is recommended for use in a container or similar environment for exactly this reason.... can't seem to find any mention of this though 😞

I think the big difficulty is getting the proxied application to work with any of these solutions:

• A unix socket only works if the proxied application will use it (in general they won't).
• Anything related to headers or auth tokens requires the proxied application to be modified to process it

[takluyver]

A unix socket only works if the proxied application will use it (in general they won't).

I realise I missed some context: I am looking at writing small applications specifically to be run through JSP. So I can make them work with a Unix socket from the start. 😉 Obviously this would be something apps opt into, but I think it's not that unusual for an application itself to listen on a Unix socket, and a proxy running separately expose that to TCP.

And the context for that is that there are things where it might be helpful to give people a web app, but having one which is visible from outside is either a technical hassle (SSH tunnelling / proxy config) or an admin hassle (forwarding a public port; edit: and dealing with authentication). JSP offers a nice way round that - it's kind of like the HTTP equivalent of writing a GUI app and telling people to use it in remote desktop.

[ryanlovett]

I agree that this would be nice to have, even if not all web services support it. For example novnc/websockify (jupyter-desktop) supports unix sockets, but the open source version of rstudio (jupyter-rsession-proxy) does not.

One could theoretically lock down a web service within a network namespace to prevent unauthorized access but to set that up I think you'd need elevated privileges.

[takluyver]

Thanks! Yes, I'm particularly interested in what can be done without special privileges - I don't have sudo access to our cluster. 🙂

In terms of the config, the least effort option would be to have a new entry in the config dictionary 'unix': True which would cause it to make a temp directory and pass {port} to the child process as something like /tmp/tmp5gm6gqn0/socket.

• You could say that reusing {port} is confusing, and it should be a separate variable like {sock_path} instead. Only one of these variables would ever be useful, and maybe reusing the same one makes it easier to switch between Unix & TCP sockets. 🤔
• You could also allow specifying a str/bytes path to a socket, like specifying a fixed port number for a TCP socket. This would require deciding what to do if the socket already exists before we start the program - error, delete it and launch the program, or launch the program and let it decide. And then there are abstract namespace sockets as well...

[takluyver]

I spent a little while looking into what this would take, and I think it should be fairly doable. The trickiest piece looks like the websocket support - tornado's WebSocketClientConnection doesn't have the same option to pass in a resolver object, which is the way to make AsyncHTTPClient use a Unix socket. I can see ways to get round that, but none of them are especially nice.

I'll hopefully start working on this soon, if no-one says otherwise, but I'll leave websockets out of the first draft.

[takluyver]

I've had a go at this in #337.