Transparent SSH SOCKS proxy for Qubes

[hdevalence]

Hi. I'd like to implement a transparent SOCKS proxy for Qubes using SSH.

This way a user can transparently tunnel all traffic from a particular AppVM through another server, without having to configure a VPN or worry about a fail-open setup.

I think it should work roughly as follows:

1. The user creates the proxyvm and enables the qubes-ssh-proxy service:

   qvm-create -p sys-ssh-proxy
   qvm-service sys-ssh-proxy -e qubes-ssh-proxy
   

2. In the template, the user installs the ssh proxy:

   sudo dnf install qubes-ssh-proxy
   

3. In the proxyVM, the user configures the ssh proxy:

   sudo ssh-keygen -q -N "" -C qubes-ssh-proxy -t ed25519 -f /rw/config/id_qubes_ssh
   sudo sed -i 's/XXXX/user@host/' /rw/config/qubes-ssh-proxy.service
   cat /rw/config/id_qubes_ssh.pub
   

4. The user adds the ssh public key from the previous step to their .authorized_keys file on the server.

5. The user sets the new ProxyVM as the NetVM for some AppVM. All TCP traffic from the AppVM is sent through the ssh tunnel and all UDP traffic is dropped.

Internally, ssh is run as a systemd service, so that restarting on failure, logging, etc. all happens automatically, and iptables sends incoming traffic through the tunnel.

Next, it would be good to automate the setup using the configuration management, so that a user just has to input user@hostname and they're given the generated ssh key to add to the server.

[Zrubi]

Hi,

Actually I have a working solution for this.
The only requirements are:

• redsocks (we need to make rpm package)
• iptables (with some magic settings ;)

My intention was not to make a failsafe/anti leaking setup - but it is completely depends on the actual iptables rules.

[Zrubi]

Hi,

Actually I have a working solution for this.
The only requirements are:

• redsocks (we need to make rpm package)
• iptables (with some magic settings ;)

My intention was not to make a failsafe/anti leaking setup - but it is completely depends on the actual iptables rules.

[Zrubi]

The current ipmlementation of qubes-firewall service are dropping the user changes ONLY in filter table, but leaving (saving) the nat table:

IPTABLES_SAVE=$(iptables-save | sed '/^\*filter/,/^COMMIT/d')
OUT=$(printf '%s\n%s\n' "$RULES" "$IPTABLES_SAVE" | sed 's/\\n\|\\x0a/\n/g' | iptables-restore 2>&1 || true)

This makes really hard to implement any custom rulesets needed such a service. because if it is calling the /rw/config/qubes-firewall-user-script There will be duplicated user defined rules/chains - unless I hack it around in that script...

@marmarek Is there any reason for this behavior?

Why not drop all the rules and recreate them from scratch?

[Zrubi]

The current ipmlementation of qubes-firewall service are dropping the user changes ONLY in filter table, but leaving (saving) the nat table:

IPTABLES_SAVE=$(iptables-save | sed '/^\*filter/,/^COMMIT/d')
OUT=$(printf '%s\n%s\n' "$RULES" "$IPTABLES_SAVE" | sed 's/\\n\|\\x0a/\n/g' | iptables-restore 2>&1 || true)

This makes really hard to implement any custom rulesets needed such a service. because if it is calling the /rw/config/qubes-firewall-user-script There will be duplicated user defined rules/chains - unless I hack it around in that script...

@marmarek Is there any reason for this behavior?

Why not drop all the rules and recreate them from scratch?

[marmarek]

This is one of the reason for discussion on new firewall API on qubes-devel.

[marmarek]

This is one of the reason for discussion on new firewall API on qubes-devel.

[andrewdavidwong]

@Zrubi, are you still working on this (or plan to)? (Asking for tracking purposes.)

[andrewdavidwong]

@Zrubi, are you still working on this (or plan to)? (Asking for tracking purposes.)

[Zrubi]

I'm not planning to work on this. Mainly because I don't believe in "leak prof" things the mass are "needs" (referring to a "leakproof" VPN hacks)

Currently I have a working transparent sock proxy solution but that is not worth publishing until the Qubes new firewall API is not released. The current Qubes firewall scripts may breaking/not refreshing the custom firewall rules.

[Zrubi]

I'm not planning to work on this. Mainly because I don't believe in "leak prof" things the mass are "needs" (referring to a "leakproof" VPN hacks)

Currently I have a working transparent sock proxy solution but that is not worth publishing until the Qubes new firewall API is not released. The current Qubes firewall scripts may breaking/not refreshing the custom firewall rules.

[andrewdavidwong]

Understood. Thanks!

[andrewdavidwong]

Understood. Thanks!

[hollyboy]

Hi, I used this tutorial https://linuxaria.com/article/redirect-all-tcp-traffic-through-transparent-socks5-proxy-in-linux
for set up ssh/proxy in ProxyVM, but i can't change settings IPTables for direct proxify traffic to another VM. Can you help me about this problem? Sorry about my English)

[hollyboy]

Hi, I used this tutorial https://linuxaria.com/article/redirect-all-tcp-traffic-through-transparent-socks5-proxy-in-linux
for set up ssh/proxy in ProxyVM, but i can't change settings IPTables for direct proxify traffic to another VM. Can you help me about this problem? Sorry about my English)

[hollyboy]

It's very important for my purposes. Thank you in advance.

[hollyboy]

It's very important for my purposes. Thank you in advance.

[hollyboy]

P. S. I am not really understand about it.

[hollyboy]

P. S. I am not really understand about it.

[hollyboy]

Can it help me https://www.qubes-os.org/doc/network-bridge-support/ ?

[hollyboy]

Can it help me https://www.qubes-os.org/doc/network-bridge-support/ ?

[entr0py]

I would suggest taking a look at Qubes VPN documentation: https://www.qubes-os.org/doc/vpn/

It's not the same but you can get a feel for how traffic can be transparently routed through a proxyVM using iptables and learn about aspects that are specific to Qubes. Then, you'll need to move redsocks redirection from nat:OUTPUT to nat:PREROUTING and set up appropriate forwarding / dns rules.

[entr0py]

I would suggest taking a look at Qubes VPN documentation: https://www.qubes-os.org/doc/vpn/

It's not the same but you can get a feel for how traffic can be transparently routed through a proxyVM using iptables and learn about aspects that are specific to Qubes. Then, you'll need to move redsocks redirection from nat:OUTPUT to nat:PREROUTING and set up appropriate forwarding / dns rules.

[andrewdavidwong]

@hollyboy: Please do not hijack GitHub issues for personal tech support. Please send a message to the qubes-users mailing list instead.

[andrewdavidwong]

@hollyboy: Please do not hijack GitHub issues for personal tech support. Please send a message to the qubes-users mailing list instead.

[Zrubi]

As I mentioned earlier, I have a working transparent SOCKS proxy solution based on redsocks:
http://darkk.net.ru/redsocks/

Now I'm created an .rpm package because there was no pre-build binary distributed jet for fedora.
I would not be able to handle the hassle to include it to the official fedora repos, but I think it is worth to include in qubes repos at least.

Sending the SRPM package to qubes-devel. This is building fine under F24, and F23
(and probably others too)
(I'm gonna make my scripts public as an usage example under Qubes)

[Zrubi]

As I mentioned earlier, I have a working transparent SOCKS proxy solution based on redsocks:
http://darkk.net.ru/redsocks/

Now I'm created an .rpm package because there was no pre-build binary distributed jet for fedora.
I would not be able to handle the hassle to include it to the official fedora repos, but I think it is worth to include in qubes repos at least.

Sending the SRPM package to qubes-devel. This is building fine under F24, and F23
(and probably others too)
(I'm gonna make my scripts public as an usage example under Qubes)

[jpouellet]

For being very fail-closed, you can use SSH's ProxyCommand option to invoke a qrexec service in a separate domain, which allows you to set the NetVM of the ProxyVM to none! The qrexec service can then enforce confidentiality and integrity of the channel and authenticity of the remote host by SSHing itself. You can then use SSH's tun/tap support (-w, -oTunnel={ethernet,point-to-point}) to transparently tunnel everything including UDP, broadcast ethernet frames, etc.

By far the most flexible and accident-proof solution I've found.

edit: for bonus leak-proofness, you can even make the ProxyVM actually be a NetVM so that you can't accidentally give it a NetVM.

[jpouellet]

For being very fail-closed, you can use SSH's ProxyCommand option to invoke a qrexec service in a separate domain, which allows you to set the NetVM of the ProxyVM to none! The qrexec service can then enforce confidentiality and integrity of the channel and authenticity of the remote host by SSHing itself. You can then use SSH's tun/tap support (-w, -oTunnel={ethernet,point-to-point}) to transparently tunnel everything including UDP, broadcast ethernet frames, etc.

By far the most flexible and accident-proof solution I've found.

edit: for bonus leak-proofness, you can even make the ProxyVM actually be a NetVM so that you can't accidentally give it a NetVM.

[v6ak]

Redsocks seems to be in Debian repository, so there seems to be no need to compile it unless you insist on Fedora. This way, you also get security updates.

How do you configure iptables for redsocks? EDIT: It seems that this is a good startpoing: https://gist.github.com/afriza/1097210 EDIT2: Hmm, not so easy. It does not affect the connected VMs and it is not much suitable for firewall script.

BTW, I've seen NetworkManager-SSH that is available in both Debian and Fedora repository. But this does not use SOCKS. It uses TUN/TAP over SSH. Its practical downside is that it has some requirements on the SSH server and the documentation requires root. So, it is not as easy solution as it looks.

[v6ak]

Redsocks seems to be in Debian repository, so there seems to be no need to compile it unless you insist on Fedora. This way, you also get security updates.

How do you configure iptables for redsocks? EDIT: It seems that this is a good startpoing: https://gist.github.com/afriza/1097210 EDIT2: Hmm, not so easy. It does not affect the connected VMs and it is not much suitable for firewall script.

BTW, I've seen NetworkManager-SSH that is available in both Debian and Fedora repository. But this does not use SOCKS. It uses TUN/TAP over SSH. Its practical downside is that it has some requirements on the SSH server and the documentation requires root. So, it is not as easy solution as it looks.

[Zrubi]

On 02/11/2018 05:56 PM, Vít Šesták wrote: Redsocks seems to be in Debian repository, so there seems to be no need to compile it unless you insist on Fedora. This way, you also get security updates. How do you configure iptables for redsocks?

iptables -I INPUT -i vif+ -p tcp --dport <redsocks port> -j ACCEPT iptables -t nat -I PREROUTING -p tcp -d <target destination> -j REDIRECT --to-port <redsocks port> where <redsocks port> defined in the actual redsocks config, and <target destination> is the network range you can reach via parent parent proxy defined in redsocks with this rules, all the forwarded tcp packets matching the <target destination> will be passed to the local redsocks instance.

BTW, I've seen NetworkManager-SSH <https://github.com/danfruehauf/NetworkManager-ssh> that is available in both Debian and Fedora repository. But this does not use SOCKS. It uses TUN/TAP over SSH. Its practical downside is that it has some requirements on the SSH server and the documentation requires root. So, it is not as easy solution as it looks.

I'm already talked with the author. TLDR; the current solution is forced by the limitation of NetworkManager. He may try to redesign it to support SOCKS proxies as well. danfruehauf/NetworkManager-ssh#41 danfruehauf/NetworkManager-ssh#66

…

-- Zrubi

[Zrubi]

On 02/11/2018 05:56 PM, Vít Šesták wrote: Redsocks seems to be in Debian repository, so there seems to be no need to compile it unless you insist on Fedora. This way, you also get security updates. How do you configure iptables for redsocks?

iptables -I INPUT -i vif+ -p tcp --dport <redsocks port> -j ACCEPT iptables -t nat -I PREROUTING -p tcp -d <target destination> -j REDIRECT --to-port <redsocks port> where <redsocks port> defined in the actual redsocks config, and <target destination> is the network range you can reach via parent parent proxy defined in redsocks with this rules, all the forwarded tcp packets matching the <target destination> will be passed to the local redsocks instance.

BTW, I've seen NetworkManager-SSH <https://github.com/danfruehauf/NetworkManager-ssh> that is available in both Debian and Fedora repository. But this does not use SOCKS. It uses TUN/TAP over SSH. Its practical downside is that it has some requirements on the SSH server and the documentation requires root. So, it is not as easy solution as it looks.

I'm already talked with the author. TLDR; the current solution is forced by the limitation of NetworkManager. He may try to redesign it to support SOCKS proxies as well. danfruehauf/NetworkManager-ssh#41 danfruehauf/NetworkManager-ssh#66

…

-- Zrubi