Tailscale Exit node breaks internet

[atjbramley]

Searches bring up a couple of similar posts which require users to paste bug reports for resolution - so I'm jumping straight to that point.

Symptoms

• • Internet works fine before connecting to exit node
• • No internet functions work while exit node is selected
• • Internet immediately restored once disconnected from exit node

I've got a really simple network - a macOS client and a Linux server:

• macbook - 100.106.79.66
• linux - 100.83.183.74

Everything green in the admin panel, exit node selected and enabled there too.

Bugreport:

BUG-23b73c81b568d9e981010d2c7ba03f24903bf923f0961cf54c49b8b15e8420ee-20210814171751Z-f19c109b201be180

Thanks in advance!

Front conversations

[DentonGentry]

The domain in question does not have DNS settings configured in Tailscale, so they will use whatever DNS settings the OS had. Could I ask what DNS server the Macbook is using?

If it is the local router or a DNS server on your local LAN: exit nodes are designed for the use case of being out and about, say on a public Wi-Fi network which is not entirely trusted. By default when an exit node is activated, access to the local LAN is blocked. If the DNS server is on the local LAN, this will look like the entire Internet is broken because name resolution doesn't work.

Clicking on the Tailscale icon in the Mac menu bar and selecting "Exit node", there is a "Allow local network access" selection. Selecting that (it will show a checkmark) would be one way to fix the immediate problem. The Macbook will be allowed to reach a DNS server on the local LAN.

---

Assuming that is the issue, the next problem may be that when you leave the home and really are out and about, DNS will break because the home LAN is no longer reachable. Switching to a public DNS server like 1.1.1.1 or 8.8.8.8 would be one way to solve this.

[atjbramley]

Thanks for the suggestion. I've tried the following:

• configuring a DNS server manually in the macOS network settings (CloudFlare)
• ticking and unticking the 'allow local network access'

I've still got the same access - the internet breaks as soon as I apply the exit node setting. I've also got no access to LAN resources once I do this.

Maybe it would help to take a step back - I'm aiming to use TailScale for 2 purposes:

• Access network resources (mainly a NAS) without having to expose the NAS to the internet
• Function as a private VPN so that I can display a different public IP when it suits me

Am I doing the right thing at all by using a TailScale exit mode? Should I instead use a TailScale as a Subnet Router and run say OpenVPN in a docker container for my VPN needs?

Thanks!

[DentonGentry]

Using an exit node would display a different public IP (the IP of the exit node), and would still allow access to the Tailscale IP address of the NAS.

Mixing multiple VPNs at the same time tends not to work well. You're welcome to try it if you'd like, but the amount of support we can provide when setting up routes through a Docker container running OpenVPN is limited.

---

The behavior described is what we'd expect if "Allow local network access" were disabled, which is odd.

There is one off thing in the telemetry: it reports Endpoints of the public IP addresses it found on the other side of the firewall, and its local 192.168.1.x address, but when it tries to contact the Coordination server the message is:

2021-08-14 18:15:47.624334 +0100 +0100: Received error: PollNetMap: Post "https://controlplane.tailscale.com/machine/REDACTED/map": read tcp 172.20.10.9:55390->18.193.255.254:443: read: can't assign requested address

The 172.20.10.9 address does not make sense. That isn't any of the IP addresses it reported as endpoints, I don't see where it is coming from. Is Docker running, and is that IP address associated with any running container?

18.193.255.254 is controlplane.tailscale.com, so that makes sense.

[atjbramley]

Regarding the expected behaviour - that's exactly what I'm after, except that the NAS doesn't have a Tailscale IP (it's an old Lenovo NAS with an obselete app store). My aim is to use the Tailscale network to allow me to access this NAS remotely, rather than exposing it directly.

If I can take the IP of the host then that's also perfect - no need for OpenVPN containers at that point as TailScale will be doing everything I wanted OpenVPN for.

---

The Linux machine serving as the exit node did indeed have Docker running and a few containers that may have caused conflicts. For the moment, I've pulled all of these and purged everything - the only networks docker now has running as the 3 default ones (bridge, host, none)

These are configured as follows:

Network	Subnet	Gateway
bridge	172.17.0.0/16	172.17.0.1
host	N/A	N/A
none	N/A	N/A

Nothing's changed with regards to behaviour - connecting to the exit mode immediately kills internet.

I've generated a new bugreport having taken the docker images down:

BUG-23b73c81b568d9e981010d2c7ba03f24903bf923f0961cf54c49b8b15e8420ee-20210815070918Z-0c57be00663b2605

[DentonGentry]

I've left the various private IP addresses in, but replaced the public IP address with w.x.y.z.
What we see now, the Macbook is trying to connect to the Linux box:

2021-08-15 08:09:13.580261 +0100 +0100: magicsock: disco: send, starting discovery for [cvbgo] (d:0828567cb7473207)
2021-08-15 08:09:13.580778 +0100 +0100: magicsock: disco: d:6a0e9a570c1efbf8->d:0828567cb7473207 ([cvbgo], 172.27.230.1:41641) sent ping tx=137f30227d34
2021-08-15 08:09:13.581135 +0100 +0100: magicsock: disco: d:6a0e9a570c1efbf8->d:0828567cb7473207 ([cvbgo], 172.27.232.1:41641) sent ping tx=004b1207c39a
2021-08-15 08:09:13.581174 +0100 +0100: magicsock: disco: d:6a0e9a570c1efbf8->d:0828567cb7473207 ([cvbgo], 172.27.234.1:41641) sent ping tx=391e68c047b6
2021-08-15 08:09:13.581203 +0100 +0100: magicsock: disco: d:6a0e9a570c1efbf8->d:0828567cb7473207 ([cvbgo], 192.168.1.42:41641) sent ping tx=1eda353564ce
2021-08-15 08:09:13.581244 +0100 +0100: magicsock: disco: d:6a0e9a570c1efbf8->d:0828567cb7473207 ([cvbgo], 172.27.228.1:41641) sent ping tx=47f0fd6e77ac
2021-08-15 08:09:13.581276 +0100 +0100: magicsock: disco: d:6a0e9a570c1efbf8->d:0828567cb7473207 ([cvbgo], 172.27.226.1:41641) sent ping tx=746e55f0a277
2021-08-15 08:09:13.581314 +0100 +0100: magicsock: disco: d:6a0e9a570c1efbf8->d:0828567cb7473207 ([cvbgo], 172.17.0.1:41641) sent ping tx=7ac14a17c873
2021-08-15 08:09:13.581342 +0100 +0100: magicsock: disco: d:6a0e9a570c1efbf8->d:0828567cb7473207 ([cvbgo], 172.27.238.1:41641) sent ping tx=f8f91dfdd561
2021-08-15 08:09:13.581364 +0100 +0100: magicsock: disco: d:6a0e9a570c1efbf8->d:0828567cb7473207 ([cvbgo], derp-8) sentcall-me-maybe
2021-08-15 08:09:13.581387 +0100 +0100: magicsock: disco: d:6a0e9a570c1efbf8->d:0828567cb7473207 ([cvbgo], 172.27.224.1:41641) sent ping tx=bf0441cb15e5
2021-08-15 08:09:13.581409 +0100 +0100: magicsock: disco: d:6a0e9a570c1efbf8->d:0828567cb7473207 ([cvbgo], 172.27.236.1:41641) sent ping tx=6f4d560830b8
2021-08-15 08:09:13.605157 +0100 +0100: magicsock: disco: d:6a0e9a570c1efbf8<-d:0828567cb7473207 ([cvbgo], w.x.y.z:41641)  got ping tx=b971be3598dd
2021-08-15 08:09:14.576805 +0100 +0100: magicsock: disco: timeout waiting for pong 8808b600c46b from w.x.y.z:41641 ([cvbgo], d:0828567cb7473207)
2021-08-15 08:09:16.57981 +0100 +0100: magicsock: disco: timeout waiting for pong 95b7e08ee907 from w.x.y.z:41641 ([cvbgo], d:0828567cb7473207)
2021-08-15 08:09:18.39529 +0100 +0100: user bugreport: BUG-23b73c81b568d9e981010d2c7ba03f24903bf923f0961cf54c49b8b15e8420ee-20210815070918Z-0c57be00663b2605
2021-08-15 08:09:18.581223 +0100 +0100: magicsock: disco: timeout waiting for pong f8f91dfdd561 from 172.27.238.1:41641 ([cvbgo], d:0828567cb7473207)
2021-08-15 08:09:18.581433 +0100 +0100: magicsock: disco: timeout waiting for pong 7ac14a17c873 from 172.17.0.1:41641 ([cvbgo], d:0828567cb7473207)
2021-08-15 08:09:18.582395 +0100 +0100: magicsock: disco: timeout waiting for pong 746e55f0a277 from 172.27.226.1:41641 ([cvbgo], d:0828567cb7473207)
2021-08-15 08:09:18.582475 +0100 +0100: magicsock: disco: timeout waiting for pong 137f30227d34 from 172.27.230.1:41641([cvbgo], d:0828567cb7473207)
2021-08-15 08:09:18.582726 +0100 +0100: magicsock: disco: timeout waiting for pong 004b1207c39a from 172.27.232.1:41641([cvbgo], d:0828567cb7473207)
2021-08-15 08:09:18.582835 +0100 +0100: magicsock: disco: timeout waiting for pong 391e68c047b6 from 172.27.234.1:41641([cvbgo], d:0828567cb7473207)
2021-08-15 08:09:18.582891 +0100 +0100: magicsock: disco: timeout waiting for pong 1eda353564ce from 192.168.1.42:41641([cvbgo], d:0828567cb7473207)
2021-08-15 08:09:18.582954 +0100 +0100: magicsock: disco: timeout waiting for pong bf0441cb15e5 from 172.27.224.1:41641([cvbgo], d:0828567cb7473207)
2021-08-15 08:09:18.583003 +0100 +0100: magicsock: disco: timeout waiting for pong 47f0fd6e77ac from 172.27.228.1:41641([cvbgo], d:0828567cb7473207)
2021-08-15 08:09:18.583065 +0100 +0100: magicsock: disco: timeout waiting for pong 6f4d560830b8 from 172.27.236.1:41641([cvbgo], d:0828567cb7473207)
2021-08-15 08:09:18.583113 +0100 +0100: magicsock: disco: timeout waiting for pong 7570da130645 from w.x.y.z:41641 ([cvbgo], d:0828567cb7473207)

The Macbook tries to contact the Linux system on every IP Endpoint. Since the Linux system is reporting a bunch of Docker IPs, the Macbook tries all of them — which is harmless, it just won't get a response.

The one that is weird is 192.168.1.x. The two systems are sitting on the same LAN right now? They should be able to connect that way, but the Macbook just gets a timeout waiting for the pong.

[scottcwang]

Hello,

I'm experiencing the same issue! Is there anything in the bug report (BUG-09e98eb3b3e8059328d671089bb98c7beba41f9aabc33c297dfb393a54746c26-20210815191157Z-e9d47a5e768554fd) that will help us figure out what's wrong?

Steps to reproduce

1. Turn forwarding on: net.ipv4.ip_forward = 1, net.ipv6.conf.all.forwarding = 1 in /etc/sysctl.conf
2. tailscale up --advertise-exit-node on the host (IP 100.126.115.2 aka 192.168.0.25)
3. Enable the exit node in the web UI
4. Select exit node (IP 100.86.43.60)
5. ping 1.1.1.1
6. ping 100.126.115.2
7. ping 192.168.0.25

Observed
5. Request timed out
6. ICMP ping response received
7. Request timed out

Expected
5. ICMP ping response received
6. ICMP ping response received
7. ICMP ping response received

[DentonGentry]

BUG-09e98eb3b3e8059328d671089bb98c7beba41f9aabc33c297dfb393a54746c26-20210815191157Z-e9d47a5e768554fd looks very similar. 192.168.0.0/24 is the surprising one, the nodes are on the local LAN together.
Please make sure that "Allow local network access" is enabled in the exit node menu, since the node is there on the local network.

2021-08-15 14:11:56.1434866 -0500 -0500: 2021/08/15 14:11:56 magicsock: disco: send, starting discovery for [gQYLt] (d:4864e80e283467df)
2021-08-15 14:11:56.1441704 -0500 -0500: 2021/08/15 14:11:56 magicsock: disco: d:e5838ba3db8619c4->d:4864e80e283467df ([gQYLt], derp-9) sent call-me-maybe
2021-08-15 14:11:56.1447363 -0500 -0500: 2021/08/15 14:11:56 magicsock: disco: d:e5838ba3db8619c4->d:4864e80e283467df ([gQYLt], w.x.y.z:41641) sent ping tx=44a39ecbdae6
2021-08-15 14:11:56.1459375 -0500 -0500: 2021/08/15 14:11:56 magicsock: disco: d:e5838ba3db8619c4->d:4864e80e283467df ([gQYLt], 192.168.255.1:41641) sent ping tx=dd5b2b6c6810
2021-08-15 14:11:56.1459375 -0500 -0500: 2021/08/15 14:11:56 magicsock: disco: d:e5838ba3db8619c4->d:4864e80e283467df ([gQYLt], 10.89.0.1:41641) sent ping tx=c6fd7d662d10
2021-08-15 14:11:56.1459375 -0500 -0500: 2021/08/15 14:11:56 magicsock: disco: d:e5838ba3db8619c4->d:4864e80e283467df ([gQYLt], 10.88.0.1:41641) sent ping tx=8285613d5c69
2021-08-15 14:11:56.1459375 -0500 -0500: 2021/08/15 14:11:56 magicsock: disco: d:e5838ba3db8619c4->d:4864e80e283467df ([gQYLt], 10.89.1.1:41641) sent ping tx=3a1dccbecdfb
2021-08-15 14:11:56.6999499 -0500 -0500: 2021/08/15 14:11:56 v1.12.3-td91ea7286-ge1bbbd90c peers: 26972/8996
2021-08-15 14:11:57.2476416 -0500 -0500: 2021/08/15 14:11:57 user bugreport: BUG-09e98eb3b3e8059328d671089bb98c7beba41f9aabc33c297dfb393a54746c26-20210815191157Z-e9d47a5e768554fd
2021-08-15 14:11:57.5153814 -0500 -0500: 2021/08/15 14:11:57 magicsock: disco: timeout waiting for pong a8c1d0441c7f from 192.168.0.25:41641 ([gQYLt], d:4864e80e283467df)
2021-08-15 14:11:58.6897596 -0500 -0500: 2021/08/15 14:11:58 v1.12.3-td91ea7286-ge1bbbd90c peers: 29324/9332
2021-08-15 14:11:59.5161707 -0500 -0500: 2021/08/15 14:11:59 magicsock: disco: timeout waiting for pong a64cd0872a50 from 192.168.0.25:41641 ([gQYLt], d:4864e80e283467df)
2021-08-15 14:12:01.1467162 -0500 -0500: 2021/08/15 14:12:01 magicsock: disco: timeout waiting for pong 44a39ecbdae6 from w.x.y.z:41641 ([gQYLt], d:4864e80e283467df)
2021-08-15 14:12:01.1469489 -0500 -0500: 2021/08/15 14:12:01 magicsock: disco: timeout waiting for pong 3a1dccbecdfb from 10.89.1.1:41641 ([gQYLt], d:4864e80e283467df)
2021-08-15 14:12:01.1469489 -0500 -0500: 2021/08/15 14:12:01 magicsock: disco: timeout waiting for pong 8285613d5c69 from 10.88.0.1:41641 ([gQYLt], d:4864e80e283467df)
2021-08-15 14:12:01.1469489 -0500 -0500: 2021/08/15 14:12:01 magicsock: disco: timeout waiting for pong c6fd7d662d10 from 10.89.0.1:41641 ([gQYLt], d:4864e80e283467df)
2021-08-15 14:12:01.1469489 -0500 -0500: 2021/08/15 14:12:01 magicsock: disco: timeout waiting for pong dd5b2b6c6810 from 192.168.255.1:41641 ([gQYLt], d:4864e80e283467df)

[DentonGentry]

The issues which most commonly come up with exit nodes and Internet breakage:

• use of a DNS server on the local LAN, and having that DNS server be blocked when the exit node is activated. Both reports in this bug use a globally reachable DNS server (1.1.1.1, 8.8.8.8, etc)
• ACLs needing to use autogroup:internet ACLs have no way to represent "allow user to use this exit node" without completely opening all traffic #1742 Neither report is using ACLs, both have the default.

[scottcwang]

Thanks for taking a look. I enabled "Allow local network access", which makes ping 192.168.0.25 work, but ping 1.1.1.1 still times out.