wireguard: add module

[ericsagnes]

Motivation for this change

WIP PR for adding a wireguard module.

Things done

• Tested using sandboxing
  (nix.useChroot on NixOS,
  or option build-use-chroot in nix.conf
  on non-NixOS)
• Built on platform(s)
  • NixOS
  • OS X
  • Linux
• Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
• Tested execution of all binary files (usually in ./result/bin/)
• Fits CONTRIBUTING.md.

---

[mention-bot]

@ericsagnes, thanks for your PR! By analyzing the annotation information on this pull request, we identified @edolstra, @bjornfor and @offlinehacker to be potential reviewers

[zx2c4]

You might want to update the description to say, "Optional preshared key used by the interface; most users will not need this."

[zx2c4]

The example should be 51820. See https://git.zx2c4.com/WireGuard/tree/src/socket.c#n344

[zx2c4]

Description should be: "IPs tunneled by interface for peer."

[zx2c4]

Example should be demo.wireguard.io:12913

[zx2c4]

This is an appalling description. Did you even read the man page of wg(8)? Here's something better:

This is optional and is by default off, because most users will not need it. It represents, in seconds, between 1 and 65535 inclusive, how often to send an authenticated empty packet to the peer, for the purpose of keeping a stateful firewall or NAT mapping valid persistently. For example, if the interface very rarely sends traffic, but it might at anytime receive traffic from a peer, and it is behind NAT, the interface might benefit from having a persistent keepalive interval of 25 seconds; however, most users will not need this.

You must not, under any circumstances, call this a "keepalive". Only "persistent keepalive".

[ericsagnes]

Sorry for that, I focused on having a somehow working module and kept the descriptions/example aesthetics at a bare minimum.

I was planning to copy paste the man page option description later when the module would be working.

Anyhow, my bad, will fix that ASAP.
Thanks for the review!

[zx2c4]

I was planning to copy paste the man page option description later when the module would be working.

Great! In that case, for the above replies, roll with the extended man page description over what I commented here.

[zx2c4]

This example is utter trash. I realize this is WIP, but it really shows a serious misunderstanding of what this nob does. A better example would be 25.

[zx2c4]

All of these commands should be in ExecStart, no?

You're also forgetting ExecStop, which should call ip link del.

[ericsagnes]

The systemd unit is the part I am the less happy with, and also the part I didn't had much time to work on.

I was thinking to split the unit with preStart and postStart and add options to allow users to override the phases if they have more complicated needs like namespace setups.
Do you have any opinion on this point?

[zx2c4]

I have no idea how NixOS generally implements its networking. If it's using systemd-networkd, then this should generate a .link unit or a .network unit or .netdev unit or something, and then have the wg setconf... in a service unit that depends on the former. If it's not using systemd-networkd, then you should try to integrate with whatever else NixOS is using. If NixOS is in fact just a series of ad-hoc modules like this, then the sky is the limit, and do whatever you feel is best. I'm no systemd expert, and while I'm vaguely aware of overriding parts of unit files in /etc, I don't know how extensible this would be for defining a series of phases. So, either trust your instinct here, or explain the problem to a systemd expert and ask them what they think.

[zx2c4]

Better description: "The IP address of the interface."

[ericsagnes]

Will fix that.

Do you think that this option should allow multiple IPs?
I saw that in https://git.zx2c4.com/WireGuard/tree/src/tests/netns.sh#n75 you set 2 IPs to the interface.

[zx2c4]

Do you think that this option should allow multiple IPs?

Yes, probably. Some people like to have a v4 and a v6 address, for example, or many v4 or many v6 addresses; networking people are nuts and like to do all sorts of strange things. This part, which amounts to calling ip addr add, is the same as with setting up any network interface. What does the NixOS configuration do for ordinary Ethernet cards? If it allows adding multiple IPs to eth0, then this here should allow adding multiple IPs to wg0. In other words, this option is the most ordinary and standard one of the module.

[zx2c4]

Probably this should be: "192.168.2.1/24" or something involves a CIDR.

[zx2c4]

Hi -- WireGuard author here. This is nowhere near ready to be merged. I've made several comments inline with the source to review. There's also one big thing missing here:

If the IP of the interface is 192.168.2.1/24, and the allowed IPs are 192.168.2.0/24 or a list of anything that falls within 192.168.2.0/24, then you don't need to do anything. However, if the allowed IPs contain something laying outside of 192.168.2.0/24, such as 10.0.0.0/16, then you need to add those routes to the routing table: ip route add 10.0.0.0/16 dev wg0 in the unit. If you don't understand why this is necessary, send me an email and we'll talk about it on the phone.

[ericsagnes]

@zx2c4 Thanks for all the inputs, that is very appreciated!

It is indeed a rough sketch made in very short time to have some base to talk and work on.
It is still a WIP so there is no risk it gets merged :)

[ericsagnes]

Added a commit to clean things up a little, not perfect but still an improvement.

However, if the allowed IPs contain something laying outside of 192.168.2.0/24, such as 10.0.0.0/16, then you need to add those routes to the routing table: ip route add 10.0.0.0/16 dev wg0 in the unit.

Nix expression language is not really a general purpose language and doing a such thing programmatically could end in a little complicated an bloated code.
For the moment I chose the easy alternative to add an extraRoutes option so the user can setup the extra routes they need.

[zx2c4]

Nix expression language is not really a general purpose language and doing a such thing programmatically could end in a little complicated an bloated code. For the moment I chose the easy alternative to add an extraRoutes option so the user can setup the extra routes they need.

This is completely unacceptable. Remove this option immediately. It's not okay, complicates things, and makes no sense.

If you can't do any real computation in the language, just add all the allowed-ips as routes. This is equally as valid and the linux routing table will sort things out for you. I don't know this language, but something like this should do:

foreach peer in values.peers:
${concatStrings (map (ip: ''
ip route add ${ip} dev ${name}
'') peer.allowed-ips)}

[ericsagnes]

One last quick commit to remove unacceptable and insanity :)

I have no much more time to work on this this week, so if you are in a hurry please make a PR to this branch, sorry.

[zx2c4]

Changes look good. Will wait til next week to see what's next. Thanks for your hard work on this!

[ericsagnes]

Regarding the unit ExecStart, it can run only one command unless the unit type is oneshot. (upstream documentation)

preStart and postStart are a pretty common pattern in NixOS when running multiple commands is needed. (see for example dhcpd)

[zx2c4]

Yes. In this case you do want Type=oneshot. It's not a daemon like dhcpd. It's a oneshot set of commands. It's similar to any of these units, which are also oneshot:

$ grep -rl oneshot
nixos/modules/virtualisation/rkt.nix
nixos/modules/virtualisation/azure-agent.nix
nixos/modules/virtualisation/libvirtd.nix
nixos/modules/virtualisation/brightbox-image.nix
nixos/modules/virtualisation/google-compute-image.nix
nixos/modules/virtualisation/azure-image.nix
nixos/modules/virtualisation/ec2-data.nix
nixos/modules/virtualisation/virtualbox-host.nix
nixos/modules/virtualisation/containers.nix
nixos/modules/services/databases/redis.nix
nixos/modules/services/computing/torque/mom.nix
nixos/modules/services/computing/torque/server.nix
nixos/modules/services/hardware/amd-hybrid-graphics.nix
nixos/modules/services/hardware/nvidia-optimus.nix
nixos/modules/services/hardware/tlp.nix
nixos/modules/services/amqp/activemq/default.nix
nixos/modules/services/networking/softether.nix
nixos/modules/services/networking/networkmanager.nix
nixos/modules/services/networking/firewall.nix
nixos/modules/services/networking/fan.nix
nixos/modules/services/networking/nat.nix
nixos/modules/services/web-servers/tomcat.nix
nixos/modules/services/network-filesystems/nfsd.nix
nixos/modules/services/system/cloud-init.nix
nixos/modules/services/security/fprot.nix
nixos/modules/services/mail/dspam.nix
nixos/modules/services/audio/mopidy.nix
nixos/modules/services/audio/alsa.nix
nixos/modules/services/cluster/kubernetes.nix
nixos/modules/services/monitoring/ups.nix
nixos/modules/services/monitoring/apcupsd.nix
nixos/modules/services/monitoring/das_watchdog.nix
nixos/modules/services/desktops/profile-sync-daemon.nix
nixos/modules/services/misc/gitolite.nix
nixos/modules/services/misc/sundtek.nix
nixos/modules/tasks/cpu-freq.nix
nixos/modules/tasks/swraid.nix
nixos/modules/tasks/filesystems.nix
nixos/modules/tasks/network-interfaces.nix
nixos/modules/tasks/filesystems/zfs.nix
nixos/modules/tasks/network-interfaces-scripted.nix
nixos/modules/system/boot/systemd-unit-options.nix
nixos/modules/system/boot/kexec.nix
nixos/modules/system/boot/networkd.nix
nixos/modules/system/boot/shutdown.nix
nixos/modules/system/boot/systemd.nix
nixos/modules/security/acme.nix
nixos/modules/security/audit.nix
nixos/modules/security/grsecurity.nix
nixos/modules/security/apparmor.nix
nixos/modules/programs/xfs_quota.nix
nixos/modules/installer/tools/auto-upgrade.nix
nixos/modules/config/swap.nix
nixos/modules/config/zram.nix
nixos/modules/config/power-management.nix

[fpletz]

Wow, an upstream author reviewing a NixOS module… that's a first. Thanks a lot for your feedback @zx2c4! 🍻

[zx2c4]

Not a very nice description. Try instead:

WireGuard Tunnel - ${name}

[zx2c4]

Seems better to add a useful interface before the module interface is stabilized.

[zx2c4]

@fpletz My pleasure. 😜

The latest changes look mostly good to me, but still a few things are funky:

• I don't think that a user setting preSetup or postSetup should result in those sections overriding what's currently there. Instead, what's currently in preSetup and postSetup should just be moved to the main body section. Then, when a user uses preSetup and postSetup, it will be respectively prepended and appended.
• The echo bringing the interface! lines look ugly to me, as journal should already be logging the actual command executions. If this is the way NixOS modules generally work, and you're just following convention, then okay. But if not, it seems unnecessary.

Putting these two points together yields this rearranged section:

        script = ''
              ${values.preSetup}
              ip link del dev ${name} 2>/dev/null || true
              ip link add dev ${name} type wireguard
              wg setconf "${name}" ${generateConf name values}
              ${concatStrings (map (ip: ''
              ip address add ${ip} dev "${name}"
              '') values.ips)}
              ip link set up dev "${name}"
              ${concatStrings (flatten (map (peer: (map (ip: ''
              ip route add ${ip} dev "${name}"
              '') peer.allowedIPs)) values.peers))}
              ${values.postSetup}
            '';

The descriptions of postSetup and preSetup should then be updated to reflect this change.

This is my personal opinion on the matter, but if NixOS developers have a good reason for the override behavior instead, I guess that's fine. My only suggestion, then, would be to call it overridePostSetup and overridePreSetup or the like.

[ericsagnes]

@zx2c4 Thanks for the feedback, updated the commit according to your last comment.

I added a basic example to the interfaces option, let me know if you want to change it to with more appropriate values.

Do you have any suggestion of relevant examples for preSetup and postSetup?

[zx2c4]

Hey @ericsagnes -- excellent work! Something I missed before, that disappeared in the reworkings is that there's no longer an ExecStop. ExecStop should definitely run ip link del dev ${name}.

Unrelated question: How does script work in NixOS? Does this actually get wired up to ExecStart? What's the advantage of doing it how you have it rather than having a series of ExecStart lines?

As for your questions:

It might be less confusing to have for the example these IPs instead:

...
ips = [192.168.20.4/24]
...
peers = {
   allowed-ips = [192.168.20.1/32]
...
}

One use of postSetup that might be useful is adding a custom DNS server, depending on how the user likes to manage DNS. For example, on my system I use resolvconf a silly little tool. After bringing up a WireGuard interface I usually run something like printf 'nameserver 10.200.100.1' | resolvconf -a wg0 -m 0, and when shutting the interface down (postShutdown?) I usually run something like resolvconf -d wg0. But that's just me.

[ericsagnes]

How does script work in NixOS? Does this actually get wired up to ExecStart? What's the advantage of doing it how you have it rather than having a series of ExecStart lines?

In a nutshell, script contents are inserted in a script that is called by the unit ExecStart.

There are two advantages in using script:

• NixOS modules attributes names have to be unique, so to generate a list of ExecStart commands it is needed to pass the commands as a list, for example ExecStart = [ "${pkgs.iproute}/bin/ip ..." "${pkgs.wireguard}/bin/wg ..." ... ];.
  Note that it is possible to turn the multiline string into a string list programmatically with something like builtins.filter (x: x != "") (lib.splitString "\n" multilinestring), but this is rather ugly.
• Commands in ExecStart have to be absolute.
  script on the other side can take advantage of the unit path, so users can use ip ... instead of ${pkgs.iproute}/bin/ip ... in pre/postSetup. Note that the unit path contains only iproute and wireguard, so other commands such as resolvconf must be prepended with ${pkgs.foo}/bin/. (a nice thing with ${pkgs.foo}/bin/foo notation is that the package doesn't need to be installed on the system, nix automatically manage it)

So using script make the module a little more user friendly while improving its readability/maintainability.

The added preStop is script for ExecStop, its contents are inserted in a script that is called by the unit ExecStop.

Changes are in a new commit to facilitate review, most important ones are:

• added ExecStop (via preStop).
• added a silly example for preSetup that you might want to change.
• added postShutdown to fit more use cases like the resolconf example.
• added meta.maintainers.

[Mic92]

One advantage I see, when ExecStart is used over script, that in case of an error,
one can see exactly which command has failed with systemctl status.

[ericsagnes]

@Mic92 Thanks, this is indeed a valid point in favor of ExecStart/Stop.

@zx2c4 Please let me know if you think ExecStart is preferable over script for this module.

Update:

ExecStart needs simple commands and will fail with anything that use shell notations like 2>/dev/null || true or printf 'nameserver 10.200.100.1' |.

So as an unfortunate result, the usage of ExecStart will limit the possibilities of pre/postSetup.

I tend to think that the flexibility provided by script surpass the possible advantages of ExecStart in this module case.

[fpletz]

Looks fine and mergeable to me in the current state. Will test later and merge. Thanks a lot for your work and the review! 👍

[Mic92]

Well, || true could have been replaced by:

ExecStart=-${pkgs.iproute2.bin}/ip link del dev ${name}

so this command is allowed to fail.
But the fact that nixos does not accept a list for ExecStart also looks problematic to me.

[ericsagnes]

@zx2c4 Could you review the latest changes?

[zx2c4]

Looks mostly good to me. One question is -- with what umask does generateConf write the file? It should be written with 077. If it's readable by the world, this is bad, since it contains private key material.

[ericsagnes]

Thank you for the review, I squashed the commits and removed the WIP flag so the PR is now ready to merge.

It should be written with 077. If it's readable by the world, this is bad, since it contains private key material.

Unfortunately, actually any file in the Nix store is world readable, handling private files in the nix store is an open and complex problem.

This is well known problem and other modules also suffer this limitation, but I am afraid there is nothing that can be done in this PR scope to fix that.

[Mic92]

When I tried the example, I noticed that empty ExecStart= entries were generated.
To fix this I just made the user defined commands list instead, which also saves some code:
Mic92@826baaa

[ericsagnes]

@Mic92 Good catch, thanks!

I updated the scriptToList so it catches newlines too, that should fix the problem you faced.

And thanks for the diff, this way indeed save some code, but might make the code harder to understand and to maintain on the long term.

[Mic92]

this example will not work, because it is not executed within in a shell - take a look at my version.

[Mic92]

this version preserves maintainability while still allowing multiple commands and not splitting user defined variables at newline.

[ericsagnes]

Thanks, applied!

[Mic92]

Thanks. there was a typo, I made in the postSetup example.

[basvandijk]

One important comment from @zx2c4 that wasn't addressed is the fact that wireguard.conf is stored in the world readable /nix/store. Although I'm guilty myself of storing secrets in the /nix/store (because it's so damn convenient as a deployment mechanism) we shouldn't force users to do something dangerous. It should always be there choice.

@zx2c4 one simple feature that could really help us is if you can store the private key in a file and reference it with PrivateKeyFile. This is just like SSH referencing the file storing the private key with IdentityFile. For convenience and consistency it would be good to also have PresharedKeyFile and PublicKeyFile.

Ideally PrivateKey is deprecated in favour of PrivateKeyFile and a check is added that makes sure the private key file isn't world readable.

Finally I would like to say that I'm excited that WireGuard is being integrated into NixOS. My colleague @FPtje and I are currently setting up a StrongSwan VPN and it's difficult and error prone. I'm looking forward switching over to WireGuard once it's in the next NixOS release. Thanks to anybody involved in this PR!

[Mic92]

@basvandijk note that wireguard is still experimental, which means the protocol can change in a backwards incompatible way.