how to run wg-easy in kubernetes?

[smshr]

I tried it running in the kubernetes (k3s) cluster. I get the following error:
error: error validating "wireguard": error validating data: ValidationError(Deployment.spec.template.spec.containers[0].securityContext): unknown field "sysctls" in io.k8s.api.core.v1.SecurityContext; if you choose to ignore these errors, turn validation off with --validate=false

my deployment.yml looks like this:

kind: Deployment
apiVersion: apps/v1
metadata:
  name: wireguard
spec:
  replicas: 1
  selector:
    matchLabels:
      app: wireguard
  template:
    metadata:
      labels:
        app: wireguard
    spec:
      containers:
      - name: wireguard
        image: weejewel/wg-easy
        ports:
        - name: wireguard
          containerPort: 51820
          protocol: UDP
        - name: web
          containerPort: 51821
          protocol: TCP
        env:
        - name: TZ
          value: America/New_York
        - name: WG_HOST
          value: vpn.example.com
        securityContext:
          capabilities:
            add:
            - NET_ADMIN
            - SYS_MODULE
          sysctls:
          - name: net.ipv4.conf.all.src_valid_mark
            value: "1"

from my host machine (debain):

root@nas:~# cat /etc/sysctl.conf| grep net.ipv4.ip_forward
net.ipv4.ip_forward=1
root@nas:~# sysctl -a | grep net.ipv4.conf.all.src_valid_mark
net.ipv4.conf.all.src_valid_mark = 0
root@nas:~# cat /etc/debian_version
10.9

1. Could it be because my host machine shows net.ipv4.conf.all.src_valid_mark as "0". If yes, how to turn this to "1"?
2. Or do I need to allow unsafe sysctls in kubernetes: https://kubernetes.io/docs/tasks/administer-cluster/sysctl-cluster/

[ikaruswill]

Based on what I understand. Sysctls are still in beta in versions prior to 1.21 so you might have needed to enable the feature via a feature flag when running k3s agent.

However it seems like there's no impact in excluding this sysctl. I'm currently running wg-easy on k3s 1.18, works just fine, though client speed statistics are off likely due to the missing sysctl, not a deal breaker imo.

[smshr]

I managed to run wg-easy in kubernetes:

1. Multiple tutorials mentioned that for wireguard to work in docker or kubernetes, host kernal should have wireguard module present, so I installed wireguard in the host machine. This may be overkill or not required and there may be leaner way around. If you know about it please let me know.
2. You will need to enable experimental feature MixedProtocolLBService in your kubernetes installation. My kubernetes cluster is k3s (not k8s), you can enable this feature in k3s by installing it like this:
   curl -sfL https://get.k3s.io | sh -s - --kube-apiserver-arg feature-gates=MixedProtocolLBService=true

Below is how my deployment looks like, please note that in this example I use traefik as reverse-proxy and authelia as traefik-middleware for UI authentication. Traefik and authelia configuration are beyond the scope of this post.

kind: Deployment
apiVersion: apps/v1
metadata:
  name: wireguard
spec:
  replicas: 1
  selector:
    matchLabels:
      app: wireguard
  template:
    metadata:
      labels:
        app: wireguard
    spec:
      containers:
      - name: wireguard
        image: weejewel/wg-easy
        env:
        - name: WG_HOST
          value: vpn.example.com
        - name: WG_DEFAULT_ADDRESS
          value: 10.0.0.x
        - name: WG_DEFAULT_DNS
          value: "1.1.1.1"
        securityContext:	# I am not sure if following is required, I haven't tried it without
          privileged: true
          capabilities:
            add:
            - NET_ADMIN
            - SYS_MODULE
        volumeMounts:
        - name: wireguard-volume	# stores wg-easy configuration 
          mountPath: /etc/wireguard
        - name: lib-volume
          mountPath: /lib/modules	# makes available wireguard module to pod, this may not be required
      volumes:
      - name: wireguard-volume
        hostPath:
          path: /mnt/k3s/wireguard
      - name: lib-volume
        hostPath:
          path: /lib/modules

---
apiVersion: v1
kind: Service
metadata:
  name: wireguard-service
  labels:
    app: wireguard
spec:
  ports:
  - name: wireguard
    targetPort: 51820
    port: 51820
    protocol: UDP
  - name: web
    targetPort: 80
    port: 80
  selector:
    app: wireguard

---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteUDP
metadata:
  name: wireguard-route
spec:
  entryPoints:
  - vpn
  routes:
  - services:
    - name: wireguard-service
      port: 51820

---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: wireguard-ui-route
spec:
  entryPoints:
  - websecure
  routes:
  - match: Host(`vpn.example.com`)
    kind: Rule
    services:
    - name: wireguard-service
      port: 80
    middlewares:
    - name: authelia-middleware
      namespace: admin
  tls:
    certResolver: letsencrypt

Following are from my host machine:

root@nas:~# cat /etc/sysctl.conf| grep net.ipv4.ip_forward
#net.ipv4.ip_forward=1
root@nas:~# sysctl -a | grep net.ipv4.conf.all.src_valid_mark
net.ipv4.conf.all.src_valid_mark = 0
root@nas:~# cat /etc/debian_version
11.0