ZeroTier versions after 1.4.6 do NOT work in ASUS RT-AC68U

[Marty4RedNetGit]

[NOTE: Perhaps the following issue may be related to previous issue #1314]

I have set up 2 separate Home Networks which are about 20 miles apart.

HOME_NETWORK_#1

ASUS RT-AC86U Router
HW: aarch64
AsusWRT with Linux Kernel: 4.1.27
ZeroTier package installed on the router via Entware repository.

RESULTS:
ZeroTier 1.8.3 version works without any problems.
ZeroTier 1.6.5 version works without any problems.
I can access the router as well as some IoT devices on the LAN.

HOME_NETWORK_#2

ASUS RT-AC68U Router
HW: armv7l
AsusWRT with Linux Kernel: 2.6.36
ZeroTier package installed on the router via Entware repository.

RESULTS:
ZeroTier 1.8.3 version does NOT work.
ZeroTier 1.6.5 version does NOT work.
ZeroTier 1.6.3 version does NOT work.
ZeroTier 1.4.6 version works OK.

For every version that does NOT work, I get the following message after the command:
zerotier-one -d

WARNING: ioctl() failed setting up Linux tap device (set MAC)

After getting this warning message, it looks like ZeroTier is "working" since the router can join my ZeroTier network, and the router node ID shows up as an active, authorized member in my network shown in the ZeroTier Central dashboard (https://my.zerotier.com/), so everything appears to be fine. However, the router does not respond at all when I try to connect to it via its assigned ZeroTier IP address. The configuration setup on this router is pretty much the same as the 1st router (in HOME_NETWORK_#1) which works very well. For some reason, the RT-AC68U Router works only when the 1.4.6 version is installed, and nothing else in the setup & configuration of the network is changed.

I really want to use the most recent ZeroTier version available to take advantage of all bug fixes as well as any security improvements.

From searching and reading other threads/posts, it seems that the above issue might be related to previous issue #1314 WRT the order in which the MAC address is set during the TAP device configuration. Currently, the MAC address is set AFTER the TAP device state is set to UP.

Is there anything else that I can do to figure out what the problem is when using more recent versions after the 1.4.6 release?

Thank you.

[glimberg]

AsusWRT with Linux Kernel: 2.6.36

That's a really old Linux kernel version (released over 11 years ago). Any way to upgrade it? If not, you may be stuck on ZeroTier v1.4.6 on that system.

[eugenezh]

Got the same problem with ASUS RT-AC66U_B1 router with asus-merlin firmware (hardware and firmware identical to RT-AC68U). The reason for the problem seems to be described in this issue:
#1314

Unfortunately, kernel upgrade for this router model does not seem to be possible.

[dartraiden]

Seems like it's a duplicate of #1314 (there are also comments from people with old kernels and mention of 1.4.6)

Try to build ZT with this patch #1389. It has already been merged into the dev branch but did not arrive into the release.

This problem is typical for all routers with old kernels, for example, using Padavan firmware.

[glimberg]

I've added a PR #1559 that might fix this, but would require you to build & test it for your router and/or old kernel versions.

[Marty4RedNetGit]

@glimberg,

Thank you for trying to fix the issue found with old Linux kernel versions when setting up the interface MAC address. Unfortunately at this point, I don't have the hardware, resources & spare time to set up a Linux development environment to build from sources. I'll try to set aside some time in mid-March to see what I need to be able to build from sources and then test your potential fix.

Thanks again. Much appreciated.

[dartraiden]

btw kernel 3.4 is affected too

[glimberg]

btw kernel 3.4 is affected too

Can't reproduce your claim here. The OS/Distro with the oldest kernel that's still supported by the vendor we have available is CentOS 7.0 with kernel 3.10 and ZeroTier's 1.6.x and 1.8.x releases all work fine on there.

[glimberg]

Trying to get a CentOS 6 VM up and running to test a patch on kernel 2.6.x, but seeing as it's not even supported by CentOS/RedHat anymore, that is proving difficult.

[san3Xian]

Similar here and ZeroTier 1.5.0 version works OK.
HW: mipsel
Linux Kernel version: 3.4.113

With version 1.8.4, I see that the mac value of the zerotier interface has been successfully set to the correctly address, but I still got a warning message and cannot communicate with other members of the network.
Also I compiled PR #1559 and it doesn't seem to fix it , based on #1559 and the above conversation, I commented out the following snippet of code in tag 1.8.4 and zerotier-one worked without any warnings and was able to communicate successfully with other members of the network.

ZeroTierOne/osdep/LinuxEthernetTap.cpp

Lines 230 to 234 in 269501e

	if (ioctl(sock,SIOCSIFHWADDR,(void *)&ifr) < 0) {
	::close(sock);
	printf("WARNING: ioctl() failed setting up Linux tap device (set MAC)\n");
	return;
	}

[Smallthing]

Similar here and ZeroTier 1.5.0 version works OK. HW: mipsel Linux Kernel version: 3.4.113

With version 1.8.4, I see that the mac value of the zerotier interface has been successfully set to the correctly address, but I still got a warning message and cannot communicate with other members of the network. Also I compiled PR #1559 and it doesn't seem to fix it , based on #1559 and the above conversation, I commented out the following snippet of code in tag 1.8.4 and zerotier-one worked without any warnings and was able to communicate successfully with other members of the network.

ZeroTierOne/osdep/LinuxEthernetTap.cpp

Lines 230 to 234 in 269501e

	if (ioctl(sock,SIOCSIFHWADDR,(void *)&ifr) < 0) {
	::close(sock);
	printf("WARNING: ioctl() failed setting up Linux tap device (set MAC)\n");
	return;
	}

hi can you tech me how to make new version zerotier on ac68u merlin ,thank you

[san3Xian]

Similar here and ZeroTier 1.5.0 version works OK. HW: mipsel Linux Kernel version: 3.4.113
With version 1.8.4, I see that the mac value of the zerotier interface has been successfully set to the correctly address, but I still got a warning message and cannot communicate with other members of the network. Also I compiled PR #1559 and it doesn't seem to fix it , based on #1559 and the above conversation, I commented out the following snippet of code in tag 1.8.4 and zerotier-one worked without any warnings and was able to communicate successfully with other members of the network.

ZeroTierOne/osdep/LinuxEthernetTap.cpp

Lines 230 to 234 in 269501e

	if (ioctl(sock,SIOCSIFHWADDR,(void *)&ifr) < 0) {
	::close(sock);
	printf("WARNING: ioctl() failed setting up Linux tap device (set MAC)\n");
	return;
	}

hi can you tech me how to make new version zerotier on ac68u merlin ,thank you

Just clone the code locally, comment out the code I mentioned above, and recompile the corresponding architecture binary with golang,considering that some router firmware is only writable in the /opt directory,maybe you need to modify the zerotier's data path.

ZeroTierOne/osdep/OSUtils.cpp

Line 420 in eac56a2

return std::string("/var/lib/zerotier-one");

[Smallthing]

so you mean this project can compile with golang ? thank you I'll try it

[glimberg]

@Smallthing no, this project is C, C++ and Rust. No Go at all.

[san3Xian]

@Smallthing sorry for the confusion, I compiled it with mipsel-linux-gnu-gcc and /usr/bin/mipsel-linux-gnu-g++

➤ git diff
diff --git a/make-linux.mk b/make-linux.mk
index a375884e..f669c88a 100644
--- a/make-linux.mk
+++ b/make-linux.mk
@@ -1,13 +1,18 @@
 # Automagically pick CLANG or RH/CentOS newer GCC if present
 # This is only done if we have not overridden these with an environment or CLI variable
-ifeq ($(origin CC),default)
-        CC:=$(shell if [ -e /usr/bin/clang ]; then echo clang; else echo gcc; fi)
-        CC:=$(shell if [ -e /opt/rh/devtoolset-8/root/usr/bin/gcc ]; then echo /opt/rh/devtoolset-8/root/usr/bin/gcc; else echo $(CC); fi)
-endif
-ifeq ($(origin CXX),default)
-        CXX:=$(shell if [ -e /usr/bin/clang++ ]; then echo clang++; else echo g++; fi)
-        CXX:=$(shell if [ -e /opt/rh/devtoolset-8/root/usr/bin/g++ ]; then echo /opt/rh/devtoolset-8/root/usr/bin/g++; else echo $(CXX); fi)
-endif
+#ifeq ($(origin CC),default)
+#        CC:=$(shell if [ -e /usr/bin/clang ]; then echo clang; else echo gcc; fi)
+#        CC:=$(shell if [ -e /opt/rh/devtoolset-8/root/usr/bin/gcc ]; then echo /opt/rh/devtoolset-8/root/usr/bin/gcc; else echo $(CC); fi)
+#endif
+#ifeq ($(origin CXX),default)
+#        CXX:=$(shell if [ -e /usr/bin/clang++ ]; then echo clang++; else echo g++; fi)
+#        CXX:=$(shell if [ -e /opt/rh/devtoolset-8/root/usr/bin/g++ ]; then echo /opt/rh/devtoolset-8/root/usr/bin/g++; else echo $(CXX); fi)
+#endif
+#CC:=/opt/rt-n56u/toolchain-mipsel/toolchain-3.4.x/bin/mipsel-linux-uclibc-gcc
+CC:=/usr/bin/mipsel-linux-gnu-gcc
+CXX:=/usr/bin/mipsel-linux-gnu-g++
+#CXX:=/opt/rt-n56u/toolchain-mipsel/toolchain-3.4.x/bin/mipsel-linux-uclibc-g++
+ZT_STATIC=1
 
 INCLUDES?=
 DEFS?=
@@ -73,9 +78,11 @@ ifeq ($(ZT_DEBUG),1)
        # C25519 in particular is almost UNUSABLE in -O0 even on a 3ghz box!
 node/Salsa20.o node/SHA512.o node/C25519.o node/Poly1305.o: CXXFLAGS=-Wall -O2 -g -pthread $(INCLUDES) $(DEFS)
 else
-       CFLAGS?=-O3 -fstack-protector -fPIE
+       #CFLAGS?=-O3 -fstack-protector -fPIE
+       CFLAGS?=-O3 -fPIE
        override CFLAGS+=-Wall -Wno-deprecated -pthread $(INCLUDES) -DNDEBUG $(DEFS)
-       CXXFLAGS?=-O3 -fstack-protector -fPIE
+       #CXXFLAGS?=-O3 -fstack-protector -fPIE
+       CXXFLAGS?=-O3 -fPIE
        override CXXFLAGS+=-Wall -Wno-deprecated -std=c++11 -pthread $(INCLUDES) -DNDEBUG $(DEFS)
        LDFLAGS=-pie -Wl,-z,relro,-z,now
        STRIP?=strip
@@ -311,7 +318,8 @@ zerotier-cli: zerotier-one
        ln -sf zerotier-one zerotier-cli
 
 libzerotiercore.a:     FORCE
-       make CFLAGS="-O3 -fstack-protector -fPIC" CXXFLAGS="-O3 -std=c++11 -fstack-protector -fPIC" $(CORE_OBJS)
+       #make CFLAGS="-O3 -fstack-protector -fPIC" CXXFLAGS="-O3 -std=c++11 -fstack-protector -fPIC" $(CORE_OBJS)
+       make CFLAGS="-O3 -fPIC" CXXFLAGS="-O3 -std=c++11 -fPIC" $(CORE_OBJS)
        ar rcs libzerotiercore.a $(CORE_OBJS)
        ranlib libzerotiercore.a
 
diff --git a/osdep/OSUtils.cpp b/osdep/OSUtils.cpp
index 7af1a1b7..7bde9399 100644
--- a/osdep/OSUtils.cpp
+++ b/osdep/OSUtils.cpp
@@ -417,7 +417,8 @@ std::string OSUtils::platformDefaultHomePath()
        return std::string("/var/db/zerotier-one");
 #else
        // Use /var/lib for Linux and other *nix
-       return std::string("/var/lib/zerotier-one");
+       //return std::string("/var/lib/zerotier-one");
+       return std::string("/opt/var/lib/zerotier-one");
 #endif
 
 #endif

[zcmhi]

Similar here and ZeroTier 1.5.0 version works OK. HW: mipsel Linux Kernel version: 3.4.113

With version 1.8.4, I see that the mac value of the zerotier interface has been successfully set to the correctly address, but I still got a warning message and cannot communicate with other members of the network. Also I compiled PR #1559 and it doesn't seem to fix it , based on #1559 and the above conversation, I commented out the following snippet of code in tag 1.8.4 and zerotier-one worked without any warnings and was able to communicate successfully with other members of the network.

ZeroTierOne/osdep/LinuxEthernetTap.cpp

Lines 230 to 234 in 269501e

	if (ioctl(sock,SIOCSIFHWADDR,(void *)&ifr) < 0) {
	::close(sock);
	printf("WARNING: ioctl() failed setting up Linux tap device (set MAC)\n");
	return;
	}

Yes, Great! Before I ran the zerotier version 1.8.4 and 1.10.1 on Linux Kernel 3.4 Padavan, but couldn't connect to other zerotier network members anyway, saw your method to comment the five lines, then it worked normally, and also tested good for latest zerotier 1.10.2.

[csl0918]

与此处类似，ZeroTier 1.5.0 版本工作正常。硬件：mipsel Linux 内核版本：3.4.113
在 1.8.4 版本中，我看到 zerotier 接口的 mac 值已成功设置为正确的地址，但我仍然收到警告消息，无法与网络的其他成员通信。我还编译了 PR #1559但它似乎没有修复它，基于#1559和上面的对话，我在标签 1.8.4 中注释了以下代码片段并且 zerotier-one 在没有任何警告的情况下工作并且能够与网络的其他成员成功沟通。

ZeroTierOne/osdep/LinuxEthernetTap.cpp

Lines 230 to 234 in 269501e

	if (ioctl(sock,SIOCSIFHWADDR,(void *)&ifr) < 0) {
	::close(sock);
	printf("WARNING: ioctl() failed setting up Linux tap device (set MAC)\n");
	return;
	}

对，很好！之前我在Linux Kernel 3.4 Padavan上运行了zerotier 1.8.4和1.10.1版本，但是无论如何都无法连接到其他zerotier网络成员，看到你的方法注释了五行，然后它正常工作，并且测试良好最新的 zerotier 1.10.2。

Similar here and ZeroTier 1.5.0 version works OK. HW: mipsel Linux Kernel version: 3.4.113
With version 1.8.4, I see that the mac value of the zerotier interface has been successfully set to the correctly address, but I still got a warning message and cannot communicate with other members of the network. Also I compiled PR #1559 and it doesn't seem to fix it , based on #1559 and the above conversation, I commented out the following snippet of code in tag 1.8.4 and zerotier-one worked without any warnings and was able to communicate successfully with other members of the network.

ZeroTierOne/osdep/LinuxEthernetTap.cpp

Lines 230 to 234 in 269501e

	if (ioctl(sock,SIOCSIFHWADDR,(void *)&ifr) < 0) {
	::close(sock);
	printf("WARNING: ioctl() failed setting up Linux tap device (set MAC)\n");
	return;
	}

Yes, Great! Before I ran the zerotier version 1.8.4 and 1.10.1 on Linux Kernel 3.4 Padavan, but couldn't connect to other zerotier network members anyway, saw your method to comment the five lines, then it worked normally, and also tested good for latest zerotier 1.10.2.

Hello, can you send me a modified IPK file of the latest version of zerotier of mipselsf-k3.4? I don't understand compilation, thank you very much! 39427441@qq.com

[ciro-mota]

Please, could you provide a step by step on how to compile the ipk file with these suggested modifications? Thank you!