wireguard: unable to route all traffic through interface

[tmplt]

Issue description

I'm unable to route all traffic through a properly configured Wireguard interface using both networking.wireguard and wg-quick(8); when using allowedIPs = [ "0.0.0.0/0" "::/0" ] I'm unable to ping any system outside my own, except for my router.

If some other IP range is allowed, say 10.100.0.0/24, I am able to ping the interface, but my traffic is not routed through it.

Steps to reproduce

1. Store a valid Wireguard configuration in /etc/wireguard/wg-test.conf with AllowedIPs = 0.0.0.0/0,::/0 under [Peer].
2. Run wg-quick up wg-test.

Or alternatively:

1. Define a Wireguard interface via networking.wireguard with allowedIps = [ "0.0.0.0/0" "::/0" ].
2. Rebuild system.

Technical details

 - system: `"x86_64-linux"`
 - host os: `Linux 4.14.83, NixOS, 18.09.1420.5d4a1a3897e (Jellyfish)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.1.3`
 - channels(root): `"nixos-18.09.1420.5d4a1a3897e"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

[anderspapitto]

likewise. I can ping the interface with allowedIPs of 10.100.0.1 exactly, but with a catchall rule of 0.0.0.0/0, I can't even ping the interface anymore

[anderspapitto]

one potential fix for this might be implementing the network user-namespace based approach I've feature requested in #52411

[Moredread]

I needed to set networking.firewall.checkReversePath = "loose";. Does that work?

[tmplt]

@Moredread, the issue remains.

[nzhang-zh]

I was able to route (almost) all traffic, by either

• excluding the server endpoint ip from peers.*.allowedIPs,

or

• running ip route add <server_endpoint_ip> via <gateway_ip> dev <interface> after wg interface is brought up

[tmplt]

I was able to route (almost) all traffic

What traffic is not routed?

[nzhang-zh]

@tmplt
I was a bit lazy and did not specify all public ip ranges when excluding server endpoint from allowedIPs.
Should be routing all traffic if you do though.
Option 2 should cover anything other than the server endpoint itself, but I would have to add more rules to exclude private ip ranges.

[colemickens]

I wasn't able to get this working, even after adding a manual route to the wireguard server (through my default gateway and regular eth0). Any suggestions?

[johnae]

I have the same problem. Doesn't matter whether I use wg-quick or the nixos options approach. Wireguard otherwise works fine when having only specific IP:s allowed, but now I wanted to route all traffic.

[johnae]

So I solved my specific problem a bit differently. I now move the wireguard interface(s) into a different Linux network namespace and then start my compositor (sway) within that network namespace. This works really well for me with the added benefit of effectively forcing all processes run within sway such as firefox, mail client etc. to use only the wireguard interface(s) (because there aren't any other interfaces). It also prompted me to write this simple utility to do what I need: https://github.com/johnae/netns-exec

While not exactly what the wireguard dev suggests (I found it a bit too messy/difficult to launch dhcpd, wpa_supplicant or iwd + wireguard within a different namespace), it's close enough for me.

[colemickens]

Hi @johnae, interesting solution - is there a benefit to your netns-exec over using ip netns exec?

I arrived at a similar solution by taking the existing, (seemingly complete or abandoned) wg-netns project and creating my own wg-netns fork that works with NixOS and warns against a couple common sources of DNS leaks. It uses ip netns exec underneath, but achieves a similar effect as you've described - I'm left with a tmux session (easily changed to sway or whatever) that is running in a netns with only lo/wg0 inside.

[johnae]

@colemickens yeah I saw that one but I wanted a setuid binary so that I could run it as my normal user. The ip command has such a large surface area so I wouldn't want to make that setuid.
I have a very basic... well "display manager" (if you squint hard) based on https://github.com/lotabout/skim and I run that on tty1 and tty2 - so I get some options like "sway private" for example. For that to be seamless combined with network namespaces I need to be able to run it as my normal user.
See https://github.com/johnae/nixos-configuration/blob/bb71d52b96f3e518decd167160357c8f4a3898cb/home/fish.nix#L9
and
https://github.com/johnae/nixos-configuration/blob/bb71d52b96f3e518decd167160357c8f4a3898cb/home/fish.nix#L110

Also messing around with the gpg-agent service from home-manager as it must run within the same network namespace when you use the gnome3 pinentry variant (this is a socket activated service). I've gotten it to work but it's still a bit wip, nothing committed yet (and certainly no pull req against home-manager, if I ever do one).

[johnae]

@colemickens I actually found another project today called exactly netns-exec (same as mine) but quite a few years old written in C - this https://github.com/pekman/netns-exec. I opted to keep my version written in Rust since it's less code and hopefully somewhat more trustworthy because of Rust but it probably doesn't matter much in the end whichever one uses.

What that other project did have however was netns-exec-bus which is a shell script for making the abstract dbus socket into a normal socket which can be used from within a namespace. So I adapted that a bit and now I don't need to run gpg-agent in any special way, can just use the service defs from home-manager which is pretty awesome overall. See here: https://github.com/johnae/nixos-configuration/blob/master/pkgs/netns-dbus-proxy/netns-dbus-proxy.sh and here: https://github.com/johnae/nixos-configuration/blob/bd1fb72978504db13642af10e7aec398b1ebe740/home/fish.nix#L9

(as a reminder - I've had issues with gpg-agent because of pinentry gnome3 which uses dbus... if you're using some other pinentry variant you probably won't have this issue).

[G-071]

I have encountered the same issue when I was trying to connect my new NixOS setup to the same wireguard server my other devices are using.

Using ip route add <server_endpoint_ip> via <gateway_ip> dev <interface> I was able to route the ipv4 traffic over the server, however, routing ipv6 still did not work.

If anyone else reading this has the same issue: In the end, I found it a lot easier to use networking.wg-quick [see PR #53043 for details) instead of networking.wireguard. With it, everything mostly worked out-of-the-box without me having to add any routes manually. I had to change only a few things in my configuration.nix (for example rename ips to address for the interface configuration).

It would be really useful / awesome to have more information on how to set this up in the NixOS Wiki.

[NilsIrl]

I am facing this issue, and used ip route add <server_endpoint_ip> via <gateway_ip> dev <interface> to fix it.

IIRC, openvpn runs this command when you start it, how come wg-quick doesn't do it?

[ethnt]

@G-071 @NilsIrl I'm facing this issue myself. When you're describing this command to fix the issue:

ip route add <server_endpoint_ip> via <gateway_ip> dev <interface>

Is this run on the server or the client? Should the server_endpoint_ip be the public IP of the server or the tunnel endpoint?

[hmenke]

I'm using good ol' stateful Wireguard interface configuration on my laptop with interactive wg-quick on the command line. It also didn't work for me but adding this line from the NixOS module to my configuration.nix made it work:

nixpkgs/nixos/modules/services/networking/wg-quick.nix

Line 309 in 05195ac

networking.firewall.checkReversePath = false;

[NilsIrl]

Is this run on the server or the client?

client

Should the server_endpoint_ip be the public IP of the server or the tunnel endpoint?

server_endpoint_ip: the public IP of the server
gateway_ip: the ip of your router
interface: the physical interface, e.g. wlp3s0

PS: I just realised that dev <interface> is optional and would personally recommend to not specify it.

[ethnt]

@NilsIrl Great, thanks. This may be a bit difficult for me, given I'd like to support mobile devices that may not be able to run commands like that, but I may be able to get it to run on my macOS client.

[amscto]

@G-071 @NilsIrl I'm facing this issue myself. When you're describing this command to fix the issue:

ip route add <server_endpoint_ip> via <gateway_ip> dev <interface>

Is this run on the server or the client? Should the server_endpoint_ip be the public IP of the server or the tunnel endpoint?

The {server ip} as in my suggested update to the wiki https://nixos.wiki/index.php?title=Wireguard&type=revision&diff=5201&oldid=4777

endpoint = "{server ip}:51820"; # ToDo: route to endpoint not automatically configured https://wiki.archlinux.org/index.php/WireGuard#Loop_routing https://discourse.nixos.org/t/solved-minimal-firewall-setup-for-wireguard-client/7577

[amscto]

Is this run on the server or the client?

client

Should the server_endpoint_ip be the public IP of the server or the tunnel endpoint?

server_endpoint_ip: the public IP of the server
gateway_ip: the ip of your router
interface: the physical interface, e.g. wlp3s0

PS: I just realised that dev <interface> is optional and would personally recommend to not specify it.

Should the endpoint route be added by https://github.com/NixOS/nixpkgs/blob/nixos-20.09/nixos/modules/services/networking/wireguard.nix

For my part I will test the use of networking.wg-quick.interfaces instead of networking.wireguard.interfaces for configuring my client and report back if that worksforme.

[amscto]

For my part I will test the use of networking.wg-quick.interfaces instead of networking.wireguard.interfaces for configuring my client and report back if that worksforme.

networking.wg-quick.interfaces worksforme (without setting DNS). I get DNS via my original non wireguarded IP connection. A separate topic for me is to understand the best way of handling DNS in a wireguarded environment.

[flokli]

networking.wireguard isn't really meant for "redirect all traffic" usecases, as elaborated in #52411. It might change in the future once we get declarative network namespace support through networkd.

However, using wg-quick should work. When it comes to debugging individual setups or server configurations, please ask in the Forums or on IRC, closing this issue.

[milahu]

{
  networking.firewall.checkReversePath = false;
}

is working, but i prefer to keep Reverse Path Filter enabled against IP spoofing attacks

• running ip route add <server_endpoint_ip> via <gateway_ip> dev <interface> after wg interface is brought up

in bash:

server_ip='....' # wireguard server address
dns_server='....' # DNS server address
wireguard_ifname='wg0'

wg-quick up "$wireguard_ifname"

# workaround for rpfilter
has_manual_gateway=false
echo "connection test: ping '$dns_server'"
timeout 1 ping -c1 "$dns_server" >/dev/null || {
  gateway="$(ip route list default | awk '{ print $3 }')"
  echo "Wireguard server seems blocked by Reverse Path Filter"
  echo "set gateway: ip route add '$server_ip' via '$gateway'"
  sudo ip route add "$server_ip" via "$gateway"
  sleep 0.5 # wait for 'ip route add' to take effect
  has_manual_gateway=true
}

if $has_manual_gateway
then echo "To disconnect from VPN, run: wg-quick down $wireguard_ifname; sudo ip route del $server_ip"
else echo "To disconnect from VPN, run: wg-quick down $wireguard_ifname"
fi