Configuring Tailnet routing on your local network

[tomvoss]

There a few small tweaks that can be made to allow routing to/from other machines on the local network and account for multiple WAN links failing over.

1. Modify /etc/default/tailscaled by adding a socket and removing the userspace-networking.

FLAGS="--port 41641 --socket /var/run/tailscale/tailscaled.sock --state /data/tailscale/tailscaled.state"

2. Add local networks to tailscale's routing table (table 52) to avoid traffic being wrongly sent over the tailscale0 interface.

ROUTES_TO_ADD=$( ip route | grep "dev br" )
echo "${ROUTES_TO_ADD}" | while read -r route; do /sbin/ip route add ${route} table 52; done

3. Use a script similar to this one that ensures tailscale packets go out of the right WAN interface in the case that multiple WAN links are present: UDM Pro requires fwmark rules on table 201 tailscale/tailscale#4038 (comment)

#!/bin/sh

RULE_PRIORITY="5225"
SLEEP_INTERVAL="0.25"
TABLE=-1

getDefaultRouteTable() {
    /sbin/ip rule list priority 32766 | cut -d " " -f 4
}

updateTailscaleMarkingRule() {
    #if default route table changed then update ip rules accordingly
    if [ ! $TABLE -eq $1 ] && [ $1 -gt 0 ]
    then
            /sbin/ip rule del priority $RULE_PRIORITY
            /sbin/ip rule add priority $RULE_PRIORITY from all fwmark 0x80000 lookup $1

            TABLE=$1
    fi
}

until false; do
    updateTailscaleMarkingRule $(getDefaultRouteTable)
    
    sleep $SLEEP_INTERVAL
done

4. Add routes and exit nodes as necessary.

tailscale up --advertise-exit-node --advertise-routes="192.168.0.0/24,192.168.1.0/24" --snat-subnet-routes=false --accept-routes --reset

[notheotherben]

Hi @tomvoss, thanks for bringing this information to my attention. I assume that you've tested and are running this on UniFi OS 1.x (i.e. a 1.12.x firmware version), but looking at the backing thread in tailscale/tailscale#4038 it sounds like this is something that Tailscale should be configuring internally.

Given the need to setup and run a separate process (in this case a script) to constantly ensure that the correct routing table is configured and the requirement to manually tweak route rules, I'm going to opt to not include this change in the core tailscale-udm scripts (as we've worked around this problem by using userspace networking).

What I will do is convert this issue into a discussion and link it in the README under the section relating to exposing your Tailnet via your UDM (i.e. not requiring Tailscale to be installed on devices within your local network). I'll also be following that Tailscale thread to see what (if any) updates come up so that I can make adjustments here as needed.

[jonp92]

Just FYI I tried this on a UDM pro with 1.x firmware and it doesn't work. In fact Tailscale won't even connect at all with those settings.

[austin-millan]

Disclaimer: I'm new to Tailscale having only set it up yesterday, and only somewhat familiar with Wireguard.

I wasn't able to get it working either with the above configurations. However, I did get it working by following the instructions here: https://tailscale.com/kb/1019/subnets/#enable-ip-forwarding

And then going to the Tailscale UI and enabling the following flags under "Edit Route Settings" for the router/node.

• Connect machines you can’t install Tailscale on.
• Allow your network to route internet traffic through this machine.

[kevinsegal]

https://tailscale.com/kb/1019/subnets/#enable-ip-forwarding

This config worked like a charm. Should add this to the main documentation, this is something a lot of people (including myself) have been trying to do for a long time, and tail scale just managed to do what zerotier couldn't do for me.

[roydufek]

May I ask what exact steps you took to make this work? I can route to the networks inside my UDM from my android phone but I can't then route to other advertised routes outside my UDM. Not sure what step I missing... I did add the ip forwarding via the 99-tailscale.conf file but not sure how to finish it off with the firewall step as the UDM doesn't have the "firewalld". I'm pretty newbie when it comes to all this but just trying to get the UDM to act like any other linux router type installation would but don't understand userspace networking all that much. Were these instructions adding to the changes suggested above or did you install and only follow enable IP forwarding? Thank you for any help you can provide! I've spent hours banging my head on wall trying to make this work... I think I am missing the masquerade step but not sure how to add that for the UDM and tailscale setup. Thanks again- :)

[FearNaBoinne]

I am on UnifyOS v3.0.20.

I got a message that the route already existed (RTNETLINK answers: File exists) and had to first delete it before I could add it to table 52.
When I did that, I could add the rule and have access over both the TailScale SSH console and the local SSH.

So my script (quick and dirty) looks like:

ROUTES_TO_ADD=$( ip route | grep "dev br" )
ROUTES_TO_DELETE=$( ip route | grep "dev br" |  cut -d " " -f 1)
echo "${ROUTES_TO_DELETE}" | while read -r route; do /sbin/ip route del ${route} table 52; done
echo "${ROUTES_TO_ADD}" | while read -r route; do /sbin/ip route add ${route} table 52; done

That got the connection up (together with the priority rule to allow tailscale to route out of the correct WAN port), however it doesn't resolve all my problems.

I can ping remote tailscale IPs and the networks behind them from the UDM Pro shell, I can ping the local tailscale IP from my hosts, but I cannot ping the remote tailscale IPs from my hosts, let alone the networks behind them.

PS: I am not certain the ROUTES_TO_DELETE part works correctly if there are multiple IP subnets to route locally, because I am pretty certain this will only give me the first one, but I haven't been able to test it, because I only have the one subnet right now!

[jhavens12]

This doesn't work for me - nothing happens when I follow the instructions. I can't even ping any of the tailscale machine IPs from UDM. What am I doing wrong?

[ksc98]

Same. At this point whoever wants a fancy cup of coffee, hit me up! :)

[jasonwbarnett]

@tomvoss this worked wonderfully. Thank you

I made one modification to the script though.

getDefaultRouteTable() {
    /sbin/ip rule list priority 32766 | cut -d " " -f 4 | cut -d "." -f 1
}

The output of /sbin/ip rule list priority 32766 | cut -d " " -f 4 on my UDM-Pro was:

$ /sbin/ip rule list priority 32766 | cut -d " " -f 4
202.eth9

[jasonwbarnett]

I also forked tailscale to eliminate the script completely. Though, my fork doesn't automate failover. This is my home network and I only have a single WAN connection.

https://github.com/jasonwbarnett/tailscale/tree/jwb/add-support-for-udm-pro

[jasonwbarnett]

I also overrode the systemd ExecStart so that arguments weren't being double up. I'm not sure if it has any actual impact, but it seemed cleaner to me.

$ EDITOR=vim systemctl edit tailscaled.service

### Editing /etc/systemd/system/tailscaled.service.d/override.conf
### Anything between here and the comment below will become the new contents of the file

[Service]
ExecStart=
ExecStart=/usr/sbin/tailscaled --port=${PORT} $FLAGS

### Lines below this comment will be discarded

### /lib/systemd/system/tailscaled.service
# [Unit]
# Description=Tailscale node agent
# Documentation=https://tailscale.com/kb/
# Wants=network-pre.target
# After=network-pre.target NetworkManager.service systemd-resolved.service
#
# [Service]
# EnvironmentFile=/etc/default/tailscaled
# ExecStartPre=/usr/sbin/tailscaled --cleanup
# ExecStart=/usr/sbin/tailscaled --state=/var/lib/tailscale/tailscaled.state --socket=/run/tailscale/tailscaled.sock --port=${PORT} $FLAGS
# ExecStopPost=/usr/sbin/tailscaled --cleanup
#
# Restart=on-failure
#
# RuntimeDirectory=tailscale
# RuntimeDirectoryMode=0755
# StateDirectory=tailscale
# StateDirectoryMode=0700
# CacheDirectory=tailscale
# CacheDirectoryMode=0750
# Type=notify
#
# [Install]
# WantedBy=multi-user.target

$ cat /etc/default/tailscaled
# Set the port to listen on for incoming VPN packets.
# Remote nodes will automatically be informed about the new port number,
# but you might want to configure this in order to set external firewall
# settings.
PORT="41641"

# Extra flags you might want to pass to tailscaled.
FLAGS="--state=/data/tailscale/tailscaled.state --socket=/var/run/tailscale/tailscaled.sock"

$ ps -ef | grep tailscale[d]
root     2496726       1  0 Dec05 ?        02:15:39 /usr/sbin/tailscaled --port=41641 --state=/data/tailscale/tailscaled.state --socket=/var/run/tailscale/tailscaled.sock

[LP0101]

I'm currently thinking about buying a UDM-pro to replace my firewall. What's the current status on this on the latest UnifyOS with only one WAN connection? Does following the instructions here work? Is there more that needs to be done?

[jasonwbarnett]

@LP0101 You still need to use this project to install tailscale on a UDM-Pro and if you don't want to install tailscale on each device then you would need to follow the directions in this discussion outlined by @tomvoss

[CaptInsano]

Sorry to ask this again but there has been discussion in this thread about improvements to the excellent work by @tomvoss at the start of this thread by other users so I am confused:

After installing tailscale as per this repo, is the first post of this thread still the best option to follow in order to allow devices on my LAN without tailscale to access other devices/subnets on my tailnet? Or do I just install tailscale and then follow the instructions of https://tailscale.com/kb/1019/subnets?tab=linux ?

@jasonwbarnett You had mentioned improvements to script & even removal of the need for the script in your own fork but I cannot see how to install same on my UDM.

Thanks to everyone of their work on this!

[jasonwbarnett]

@CaptInsano I don't hold myself as the authority here, so take what I say with a grain of salt.

After installing tailscale as per this repo, is the first post of this thread still the best option to follow in order to allow devices on my LAN without tailscale to access other devices/subnets on my tailnet? Or do I just install tailscale and then follow the instructions of https://tailscale.com/kb/1019/subnets?tab=linux ?

Yes, this discussion is still the most up-to-date information for configuring a UDM-Pro as a subnet router.

@jasonwbarnett You had mentioned improvements to script & even removal of the need for the script in your own fork but I cannot see how to install same on my UDM.

1. I never tried following @austin-millan comment.
2. The code I wrote in my fork is really hacky and probably won't ever be merged until I work with someone over at tailscale to figure out a good way to do what I did that doesn't interfere with other tailscale use cases. I need to pick this up, so thanks for the nudge.

I don't know how technical you are, but if you're familiar with go it's pretty straight forward to compile tailscale off of my branch and upload it, replacing the binaries with the ones compiled.

A high level looks like this (this should be done after following the standard SierraSoftworks/tailscale-udm installation):

# Clone Repo
git clone -b jwb/add-support-for-udm-pro https://github.com/jasonwbarnett/tailscale.git

# Compile binaries
cd tailscale
./build_udm_pro.sh

# Upload binaries to UDM-Pro
scp ./tailscale{,d} root@<ip-of-udm-pro>:~/

# Login via SSH to the UDM-Pro
ssh root@<ip-of-udm-pro>

# Overwrite original binaries with new ones
mv ./tailscaled /usr/sbin/tailscaled
mv ./tailscale /usr/bin/tailscale

# Restart tailscaled
systemctl restart tailscaled

You can confirm everything worked by visiting https://login.tailscale.com/admin/machines and the version reported for the UDM-Pro should be: 1.56.1-udm-pro

[jasonwbarnett]

tailscale PR opened: tailscale/tailscale#10828

[mickg10]

@jasonwbarnett Just to check - as that PR is still pending - does this still require user-space networking? Somehow, after upgrading to 1.64, not having use-space networking broke with tailscaled outputting:
Received error: fetch control key: Get "https://controlplane.tailscale.com/key?v=90": dial tcp [2a05:d014:386:202:4d:f871:20f3:6602]:443: connect: network is unreachable

I do wonder if they broke something...

[jasonhollis]

This all worked perfectly for me but I'm still getting:

tailscale status
100.122.23.76 ktp-newport jason@ linux -
100.84.226.19 brainassistant jason@ linux idle; offers exit node; offline
100.119.51.50 hanewport jason@ linux idle; offers exit node; offline
100.76.248.120 haosnewport jason@ linux idle; offers exit node; offline
100.127.227.41 hapi5-1 jason@ linux idle; offers exit node; offline
100.120.49.71 homeassistant jason@ linux idle; offers exit node
100.104.22.108 jfh14m2max-2 jason@ macOS offline
100.119.177.51 jfh16m1max-2 jason@ macOS offline
100.87.136.8 jfhipadpro jason@ iOS offline
100.95.146.87 jfhiphone14promax jason@ iOS offline
100.79.125.130 mabels-macbook-pro mabel@ macOS offline

Health check:

- Update available: 1.56.1 -> 1.62.1, run tailscale update or tailscale set --auto-update to update

- dns-os: getting OS base config is not supported

- dns: getting OS base config is not supported

root@KTP-Newport:~# ping 100.120.49.71
PING 100.120.49.71 (100.120.49.71) 56(84) bytes of data.

--- 100.120.49.71 ping statistics ---
10 packets transmitted, 0 received, 100% packet loss, time 9332ms

root@KTP-Newport:~# tailscale ping 100.120.49.71
pong from homeassistant (100.120.49.71) via 103.214.220.33:45494 in 11ms

It's not working! It seems totally crazy to me to have to put Tailscale in other places when I have a perfect UDMP SE sitting there were it should be running!