DNS resolution not working after turning exit node #3842
Labels
bug
Bug
dns
exit-node
Exit node related
L2 Few
Likelihood
P2 Aggravating
Priority level
T5 Usability
Issue type
Comments
|
I had the same issue and it was a firewall issue. After setting
on the Exit node, DNS works again. |
|
Well, I have no firewall running on exit node at all. There is also nothing in the logs of exit node indicating any kind of connection attempt. Ok, looked at logs at both sides and there are this log on the client side (tailscale on ubuntu on raspberry pi) while trying to make a lookup: Strangely, pinging that ip address works just fine: Trace also works: However, telnet shows: Never encountered anything like that, honestly. No route to host for telnet and pinging just fine, what? |
|
There is another interesting piece of information in logs: |
|
Hopefully I'll find time to investigate soon. |
bradfitz
added a commit
that referenced
this issue
Jan 31, 2022
We're finding a bunch of host operating systems/firewalls interact poorly with peerapi. We either get ICMP errors from the host or users need to run commands to allow the peerapi port: #3842 (comment) ... even though the peerapi should be an internal implementation detail. Rather than fight the host OS & firewalls, this change handles the server side of peerapi entirely in netstack (except on iOS), so it never makes its way to the host OS where it might be messed with. Two main downsides are: 1) netstack isn't as fast, but we don't really need speed for peerapi. And actually, with fewer trips to/from the kernel, we might actually make up for some of the netstack performance loss by staying in userspace. 2) tcpdump / Wireshark etc packet captures will no longer see the peerapi traffic. Oh well. Crawshaw's been wanting to add packet capture server support to tailscaled, so we'll probably do that sooner now. A future change might also then use peerapi for the client-side (except on iOS). Updates #3842 (probably fixes, as well as many exit node issues I bet) Change-Id: Ibc25edbb895dc083d1f07bd3cab614134705aa39 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
|
@JayWStapleton was investigating https://forum.tailscale.com/t/exit-node-on-oracle-oci/1662/7 and could reproduce the It turned out to be ICMP errors from the exit node: We can just handle peerapi entirely in netstack, though: I sent #3851. |
bradfitz
added a commit
that referenced
this issue
Jan 31, 2022
We're finding a bunch of host operating systems/firewalls interact poorly with peerapi. We either get ICMP errors from the host or users need to run commands to allow the peerapi port: #3842 (comment) ... even though the peerapi should be an internal implementation detail. Rather than fight the host OS & firewalls, this change handles the server side of peerapi entirely in netstack (except on iOS), so it never makes its way to the host OS where it might be messed with. Two main downsides are: 1) netstack isn't as fast, but we don't really need speed for peerapi. And actually, with fewer trips to/from the kernel, we might actually make up for some of the netstack performance loss by staying in userspace. 2) tcpdump / Wireshark etc packet captures will no longer see the peerapi traffic. Oh well. Crawshaw's been wanting to add packet capture server support to tailscaled, so we'll probably do that sooner now. A future change might also then use peerapi for the client-side (except on iOS). Updates #3842 (probably fixes, as well as many exit node issues I bet) Change-Id: Ibc25edbb895dc083d1f07bd3cab614134705aa39 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
|
Great, I use Ubuntu on OCI too. |
bradfitz
added a commit
that referenced
this issue
Jan 31, 2022
We're finding a bunch of host operating systems/firewalls interact poorly with peerapi. We either get ICMP errors from the host or users need to run commands to allow the peerapi port: #3842 (comment) ... even though the peerapi should be an internal implementation detail. Rather than fight the host OS & firewalls, this change handles the server side of peerapi entirely in netstack (except on iOS), so it never makes its way to the host OS where it might be messed with. Two main downsides are: 1) netstack isn't as fast, but we don't really need speed for peerapi. And actually, with fewer trips to/from the kernel, we might actually make up for some of the netstack performance loss by staying in userspace. 2) tcpdump / Wireshark etc packet captures will no longer see the peerapi traffic. Oh well. Crawshaw's been wanting to add packet capture server support to tailscaled, so we'll probably do that sooner now. A future change might also then use peerapi for the client-side (except on iOS). Updates #3842 (probably fixes, as well as many exit node issues I bet) Change-Id: Ibc25edbb895dc083d1f07bd3cab614134705aa39 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
|
Okay, the fix is in the latest unstable build, in version 1.21.43 or later. Can somebody try it out? @cepera-ang? We're not sure yet whether we'll backport it to the 1.20.x branch yet. First we want to see how many people's problems it fixes. |
|
Yep, quick testing show that it's working now (updated only the exit node) |
|
Same here. Updating to that unstable build on the exit node fixed the problem. |
|
Linux (ArchLinux to be precise) |
This was referenced Feb 1, 2022
|
Can confirm that the ufw rules are unnecessary in the current unstable build. |
bradfitz
added a commit
that referenced
this issue
Feb 4, 2022
We're finding a bunch of host operating systems/firewalls interact poorly with peerapi. We either get ICMP errors from the host or users need to run commands to allow the peerapi port: #3842 (comment) ... even though the peerapi should be an internal implementation detail. Rather than fight the host OS & firewalls, this change handles the server side of peerapi entirely in netstack (except on iOS), so it never makes its way to the host OS where it might be messed with. Two main downsides are: 1) netstack isn't as fast, but we don't really need speed for peerapi. And actually, with fewer trips to/from the kernel, we might actually make up for some of the netstack performance loss by staying in userspace. 2) tcpdump / Wireshark etc packet captures will no longer see the peerapi traffic. Oh well. Crawshaw's been wanting to add packet capture server support to tailscaled, so we'll probably do that sooner now. A future change might also then use peerapi for the client-side (except on iOS). Updates #3842 (probably fixes, as well as many exit node issues I bet) Change-Id: Ibc25edbb895dc083d1f07bd3cab614134705aa39 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com> (cherry picked from commit bd90781) + edits (and cherry picked part of commit f3c0023)
bradfitz
added a commit
that referenced
this issue
Feb 4, 2022
We're finding a bunch of host operating systems/firewalls interact poorly with peerapi. We either get ICMP errors from the host or users need to run commands to allow the peerapi port: #3842 (comment) ... even though the peerapi should be an internal implementation detail. Rather than fight the host OS & firewalls, this change handles the server side of peerapi entirely in netstack (except on iOS), so it never makes its way to the host OS where it might be messed with. Two main downsides are: 1) netstack isn't as fast, but we don't really need speed for peerapi. And actually, with fewer trips to/from the kernel, we might actually make up for some of the netstack performance loss by staying in userspace. 2) tcpdump / Wireshark etc packet captures will no longer see the peerapi traffic. Oh well. Crawshaw's been wanting to add packet capture server support to tailscaled, so we'll probably do that sooner now. A future change might also then use peerapi for the client-side (except on iOS). Updates #3842 (probably fixes, as well as many exit node issues I bet) Change-Id: Ibc25edbb895dc083d1f07bd3cab614134705aa39 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com> (cherry picked from commit bd90781) + edits (and cherry picked part of commit f3c0023)
DentonGentry
added
L2 Few
bradfitz
added a commit
that referenced
this issue
Feb 7, 2022
We're finding a bunch of host operating systems/firewalls interact poorly with peerapi. We either get ICMP errors from the host or users need to run commands to allow the peerapi port: #3842 (comment) ... even though the peerapi should be an internal implementation detail. Rather than fight the host OS & firewalls, this change handles the server side of peerapi entirely in netstack (except on iOS), so it never makes its way to the host OS where it might be messed with. Two main downsides are: 1) netstack isn't as fast, but we don't really need speed for peerapi. And actually, with fewer trips to/from the kernel, we might actually make up for some of the netstack performance loss by staying in userspace. 2) tcpdump / Wireshark etc packet captures will no longer see the peerapi traffic. Oh well. Crawshaw's been wanting to add packet capture server support to tailscaled, so we'll probably do that sooner now. A future change might also then use peerapi for the client-side (except on iOS). Updates #3842 (probably fixes, as well as many exit node issues I bet) Change-Id: Ibc25edbb895dc083d1f07bd3cab614134705aa39 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com> (cherry picked from commit bd90781) + edits (and cherry picked part of commit f3c0023)
|
I have another connectivity issue. I tried to install pi-hole to my vpn server and I unable to connect to its web-interface, nor to tcp DNS resolver. Interestingly, I can connect to ssh and tailscaled simple web-server. I tried from windows and linux machines, same story. Logs on client BUG-eafc8fc920530480e584258f1a87593e4f3a50ef100996de93f63d2df5d6a318-20220213143122Z-e9516f2deb31fbe1: Log on the server BUG-497beb7817e8ae7ece3867e0ee1b898b86d2248109879b958ca2faeb2d1d1d82-20220213143146Z-a23a7c7bfafcd8ea |
|
@cepera-ang I moved that last comment into a new issue, tailscale/tailscale-www#975 |
|
Fixed in 1.22 |
|
It seems I'm see'ing this return. I'm on v1.30.0 MacOS. |
|
Please open a new bug, with details. It is unlikely you are experiencing the same root cause. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
What is the issue?
After updating from Tailscale 1.18 to Tailscale 1.20.2 I no longer can use exit node functionality. I have ubuntu cloud machine as exit node (named vpn) and a windows machine. After enabling exit node on windows I get all DNS requests going to 100.100.100.100 and dying in timeout. The same requests from older version work flawlessly.
For ex, windows, 1.20.2, exit node off:
Windows, 1.20.2, exit node on:
Linux, 1.18.2, exit node on or off, whatever (same result) :
Linux, 1.20.2, exit node on:
Well, anyway, it seems like 100.100.100.100 not working anywhere in 1.20.2 for me.
I see some changes related to DNS and exit nodes in release notes. Is there some configuration I have to do, in order to get this working again?
Steps to reproduce
No response
Are there any recent changes that introduced the issue?
Updated Tailscale everywhere to the latest version.
OS
Linux, Windows
OS version
Ubuntu 20.04.3 LTS (GNU/Linux 5.11.0-1027-oracle aarch64), Microsoft Windows [Version 10.0.19044.1466]
Tailscale version
1.20.2
Bug report
BUG-c2f835af9713719097081eaf7976601903d023065d119901ad8e2e1799922664-20220130093427Z-bd88e452804a0817
The text was updated successfully, but these errors were encountered: