zerotier api work with localhost but not 127.0.0.1 #842

ghost · 2018-08-29T13:53:45Z

$ sudo zerotier-cli info
Error connecting to the ZeroTier service:

Please check that the service is running and that TCP port 9993 can be contacted via 127.0.0.1.

$ curl -H 'X-ZT1-Auth: xxxxxxxxxxxxxxxxx' http://127.0.0.1:9993/status
curl: (52) Empty reply from server

$ curl -H 'X-ZT1-Auth: xxxxxxxxxxxxxxxxx' http://localhost:9993/status
{
 "address": "xxxxxxxxxxx",
 "config": {
  "physical": null,
  "settings": {
   "allowTcpFallbackRelay": true,
   "portMappingEnabled": true,
   "primaryPort": 9993,
   "softwareUpdate": "disable",
   "softwareUpdateChannel": "release"
  }
 },
 "online": true,
 "publicIdentity": "",
 "tcpFallbackActive": false,
 "version": "1.2.12",
 "versionBuild": 0,
 "versionMajor": 1,
 "versionMinor": 2,
 "versionRev": 12
}

and /etc/hosts:

127.0.0.1       localhost
127.0.1.1       vps vps.xxxxxxx.com

x.x.x.x         vps.xxxxxx.com

# The following lines are desirable for IPv6 capable hosts
::1     localhost ip6-localhost ip6-loopback
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

and ifconfig lo

$ ifconfig lo
lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 316190  bytes 17294152 (17.2 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 316190  bytes 17294152 (17.2 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

The text was updated successfully, but these errors were encountered:

laduke · 2018-08-29T16:32:34Z

hi, it seems like zerotier-cli wouldn't work for anybody if that was the case?

Check the local firewall.

ghost · 2018-08-29T23:16:39Z

This is ubuntu linux box on Linode. No firewall on this box.

$ telnet localhost 9993
Trying ::1...
Connected to localhost.
Escape character is '^]'.
^]

telnet> q
Connection closed.
$ telnet 127.0.0.1 9993
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
^]

telnet> q
Connection closed.

It seems ipv6 related. localhost currently resolves to ::1 not 127.0.0.1

I tried to add precedence ::ffff:0:0/96 100 to /etc/gai.conf. After that, localhost resolves to 127.0.0.1. And then REST api request sent to localhost stopped working. But [::1] still works

$ curl -H 'X-ZT1-Auth: xxxxxxxxx' http://[::1]:9993/status
{
 "address": "xxxxxxx",
 "config": {
  "physical": null,
  "settings": {
   "allowTcpFallbackRelay": true,
   "portMappingEnabled": true,
   "primaryPort": 9993,
   "softwareUpdate": "disable",
   "softwareUpdateChannel": "release"
  }
 },
 "online": true,
....
 "tcpFallbackActive": false,
 "version": "1.2.12",
 "versionBuild": 0,
 "versionMajor": 1,
 "versionMinor": 2,
 "versionRev": 12
}

So the issue is that zerotier-one only accepts requests from ::1 not 127.0.01.

$ netstat -nlp|grep 9993
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
tcp        0      0 x.x.x.x:9993            0.0.0.0:*               LISTEN      -
tcp        0      0 127.0.0.1:9993          0.0.0.0:*               LISTEN      -
tcp6       0      0 x:x:x:x:x:x:9993        :::*                    LISTEN      -
tcp6       0      0 ::1:9993                :::*                    LISTEN      -
udp        0      0 x.x.x.x:9993            0.0.0.0:*                           -
udp6       0      0 x:x:x:x:x:x:9993        :::*                                -

laduke · 2018-08-29T23:34:06Z

I can't reproduce on mac or ubuntu 16.04. Any idea how to reproduce?

ghost · 2018-08-29T23:40:35Z

I have 3 ubuntu Linux boxes. 2 of them have the same issue.

kernel versions:

# Ubuntu 18.04.1 (KVM Linode VPS, with issue)
$ uname -r
4.17.17-x86_64-linode116

# Ubuntu 16.04.5 (OpenVZ BWH VPS, with issue)
$ uname -r
2.6.32-042stab127.2

# Ubuntu 18.04.1 (KVM BWH VPS, only this one is good)
$ uname -r 
4.15.0-23-generic

ghost · 2018-08-29T23:43:22Z

The other box (OpenVZ VPS) with the same issue

$ ifconfig lo
lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:32773952 errors:0 dropped:0 overruns:0 frame:0
          TX packets:32773952 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:28537104198 (28.5 GB)  TX bytes:28537104198 (28.5 GB)

$ cat /etc/hosts
ff02::1         ip6-allnodes
ff02::2         ip6-allrouters

127.0.0.1 localhost.localdomain localhost
# Auto-generated hostname. Please do not remove this comment.
x.x.x.x vps1.xxxx.com  vps1
::1             localhost ip6-localhost ip6-loopback

$ netstat -nlp|grep 9993
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
tcp        0      0 x.x.x.x:9993       0.0.0.0:*               LISTEN      -
tcp        0      0 127.0.0.1:9993          0.0.0.0:*               LISTEN      -
tcp6       0      0 ::1:9993                :::*                    LISTEN      -
udp        0      0 x.x.x.x:9993       0.0.0.0:*                           -

Zerotier works fine on these two boxes if I send api requests to [::1] to join networks instead of zerotier-cli

laduke · 2018-08-30T16:24:46Z

What do those boxes have in common?

glimberg · 2018-08-30T16:25:32Z

what's the output of iptables -L and ip6tables -L on those machines?

ghost · 2018-08-30T16:37:57Z

$ sudo iptables -L -n
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
ACCEPT     tcp  --  0.0.0.0/0            x.x.x.x(public ip)        tcp dpt:993

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
DROP       tcp  --  x.x.x.x(public ip)        0.0.0.0/0            tcp spt:993 flags:0x04/0x04

$ sudo ip6tables -L -n
Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

ghost · 2018-08-30T16:43:27Z

I tried to completely disable ipv6, the issue was still there.

net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1

ghost · 2018-08-30T16:50:37Z

i have figured out why they don't work:

iptables -A POSTROUTING -t nat -j MASQUERADE

mvonweis · 2018-10-22T21:57:42Z

Hi, I got bitten by this too. There were some iptables rules remaining from a prior attempt to install a VPN. It took me a long while to figure out that Http::GET simply returns an empty responseBody and that's why the error message given by zerotier-cli is also empty. It would be great if this situation could be identified and reported to the user in a better way.

bennytheshap · 2019-05-21T04:21:11Z

I don't have any active iptables rules, but am experiencing this issue on an up-to-date Pi. Any suggestions?

root@sheetbroker:/var/lib/zerotier-one# curl -H 'X-ZT1-Auth: xxxxxxxxxxxxxxxxx' http://127.0.0.1:9993/status
curl: (52) Empty reply from server
root@sheetbroker:/var/lib/zerotier-one# curl -H 'X-ZT1-Auth: xxxxxxxxx' http://[::1]:9993/status
{}

SmiVan · 2019-08-13T14:15:38Z

Having exactly the same issue as @bennytheshap on Raspberry Pi:

pi@raspberry:~ $ curl -H 'X-ZT1-Auth: ...' http://127.0.0.1:9993/status
curl: (52) Empty reply from server
pi@raspberry:~ $ curl -H 'X-ZT1-Auth: ...' http://[::1]:9993/status
{
 ...
 "config": {
  "physical": null,
  "settings": {
   "allowTcpFallbackRelay": true,
   "portMappingEnabled": true,
   "primaryPort": 9993,
   "softwareUpdate": "disable",
   "softwareUpdateChannel": "release"
  }
 },
 "online": true,
 ...
 "tcpFallbackActive": false,
 "version": "1.4.2",
 "versionBuild": 0,
 "versionMajor": 1,
 "versionMinor": 4,
 "versionRev": 2
}

Should a separate issue be opened for this?

OzuYatamutsu · 2019-08-27T20:44:10Z

I also have the same problem.

jinhai@pir2 ~/Downloads/ZeroTierOne-1.4.4> curl http://127.0.0.1:9993
curl: (52) Empty reply from server
jinhai@pir2 ~/Downloads/ZeroTierOne-1.4.4> curl http://[::1]:9993
jinhai@pir2 ~/Downloads/ZeroTierOne-1.4.4>

OzuYatamutsu · 2019-08-27T21:52:24Z

Ah, I found the solution. What @chnnnkk suggested is true, but the fix is to delete these rules, i.e.:

sudo iptables -D POSTROUTING -t nat -j MASQUERADE

(repeat until there are no more rules to delete)

jinhai@smol ~> sudo iptables -D POSTROUTING -t nat -j MASQUERADE
jinhai@smol ~> sudo iptables -D POSTROUTING -t nat -j MASQUERADE
jinhai@smol ~> sudo iptables -D POSTROUTING -t nat -j MASQUERADE
iptables: No chain/target/match by that name.
jinhai@smol ~> sudo zerotier-cli listnetworks
200 listnetworks <nwid> <name> <mac> <status> <type> <dev> <ZT assigned ips>
<snip>

dreamer2q · 2020-08-14T13:24:28Z

@OzuYatamutsu you really helped me. However, I do not know why it works...

laduke added the Type: Question General questions about the product, plans, etc. label Aug 29, 2018

ghost closed this as completed Aug 30, 2018

This issue was closed.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

zerotier api work with localhost but not 127.0.0.1 #842

zerotier api work with localhost but not 127.0.0.1 #842

ghost commented Aug 29, 2018 •

edited by ghost

Loading

laduke commented Aug 29, 2018

ghost commented Aug 29, 2018 •

edited by ghost

Loading

laduke commented Aug 29, 2018

ghost commented Aug 29, 2018 •

edited by ghost

Loading

ghost commented Aug 29, 2018 •

edited by ghost

Loading

laduke commented Aug 30, 2018 •

edited

Loading

glimberg commented Aug 30, 2018

ghost commented Aug 30, 2018 •

edited by ghost

Loading

ghost commented Aug 30, 2018

ghost commented Aug 30, 2018

mvonweis commented Oct 22, 2018

bennytheshap commented May 21, 2019 •

edited

Loading

SmiVan commented Aug 13, 2019 •

edited

Loading

OzuYatamutsu commented Aug 27, 2019

OzuYatamutsu commented Aug 27, 2019

dreamer2q commented Aug 14, 2020

zerotier api work with localhost but not 127.0.0.1 #842

zerotier api work with localhost but not 127.0.0.1 #842

Comments

ghost commented Aug 29, 2018 • edited by ghost Loading

laduke commented Aug 29, 2018

ghost commented Aug 29, 2018 • edited by ghost Loading

laduke commented Aug 29, 2018

ghost commented Aug 29, 2018 • edited by ghost Loading

ghost commented Aug 29, 2018 • edited by ghost Loading

laduke commented Aug 30, 2018 • edited Loading

glimberg commented Aug 30, 2018

ghost commented Aug 30, 2018 • edited by ghost Loading

ghost commented Aug 30, 2018

ghost commented Aug 30, 2018

mvonweis commented Oct 22, 2018

bennytheshap commented May 21, 2019 • edited Loading

SmiVan commented Aug 13, 2019 • edited Loading

OzuYatamutsu commented Aug 27, 2019

OzuYatamutsu commented Aug 27, 2019

dreamer2q commented Aug 14, 2020

ghost commented Aug 29, 2018 •

edited by ghost

Loading

ghost commented Aug 29, 2018 •

edited by ghost

Loading

ghost commented Aug 29, 2018 •

edited by ghost

Loading

ghost commented Aug 29, 2018 •

edited by ghost

Loading

laduke commented Aug 30, 2018 •

edited

Loading

ghost commented Aug 30, 2018 •

edited by ghost

Loading

bennytheshap commented May 21, 2019 •

edited

Loading

SmiVan commented Aug 13, 2019 •

edited

Loading