error: host vs container / private vs shared / tun/tap vs ethertap

[Andrei-Pozolotin]

Guus:

1. on linux:

# uname -a
Linux mail-serv 4.12.8-1-ec2 #1 SMP Mon Aug 21 22:43:04 PDT 2017 x86_64 GNU/Linux

2. using package:

https://www.archlinux.org/packages/community/x86_64/tinc/

3. this simple tinc configuration:

# cat /etc/tinc/mail/tinc.conf
AddressFamily = ipv4
Device = /dev/net/tun 
DeviceType = tun
Name = mail_1
ConnectTo = mail_2

4. starts fine when running on the host:

# journalctl -u tinc@mail
Oct 10 20:08:34 serv-1 systemd[1]: Started Tinc net mail.
Oct 10 20:08:34 serv-1 tincd[9842]: tincd 1.0.32 starting, debug level 0
Oct 10 20:08:34 serv-1 tincd[9842]: /dev/net/tun is a Linux tun/tap device (tun mode)
Oct 10 20:08:34 serv-1 tincd[9842]: Ready

5. it also starts fine in container with nspawn private network

systemd-nspawn ... --private-network

6. however when running in container with nspawn shared host network,

systemd-nspawn ... # remove option: --private-network 

tinc produces error:

# journalctl -M mail-serv -u tinc@mail
Oct 10 20:16:28 mail-serv tincd[735]: tincd 1.0.32 starting, debug level 0
Oct 10 20:16:28 mail-serv tincd[735]: /dev/net/tun is a Linux ethertap device
Oct 10 20:16:28 mail-serv tincd[735]: Ready
Oct 10 20:16:28 mail-serv tincd[735]: Error while reading from Linux ethertap device /dev/net/tun: File descriptor in bad state
...
Oct 10 20:16:31 mail-serv tincd[735]: Too many errors from /dev/net/tun, exiting!
Oct 10 20:16:31 mail-serv tincd[735]: Terminating

7. so in the last scenario same device /dev/net/tun is somehow detected as ethertap, not tun/tap

for reference: systemd-nspawn

https://www.freedesktop.org/software/systemd/man/systemd-nspawn.html

Question: are there any quick pointers on how to resove this?

Thank you.

[Andrei-Pozolotin]

attached is log from

# strace tincd -D -d 3 -n mail &> tinc.log

tinc.log.zip

[Andrei-Pozolotin]

log shows apparent problem:

ioctl(3, TUNSETIFF, 0x7fffb3906320)     = -1 EPERM (Operation not permitted)
ioctl(3, _IOC(0, 0x54, 0xca, 0), 0x7fffb3906320) = -1 EBADFD (File descriptor in bad state)

and the workaround is

systemd-nspawn ... --capability=CAP_NET_ADMIN

which is also cofirmed by
https://www.freedesktop.org/software/systemd/man/systemd-nspawn.html

--capability=
   ... Also CAP_NET_ADMIN is retained if --private-network is specified. ...

then these requests remain:
A) can tinc work with less capabilities then CAP_NET_ADMIN?
B) please report Operation not permitted instead of File descriptor in bad state?

[ghost]

A) CAP_NET_ADMIN is necessary to create a tun or tap device:
https://github.com/torvalds/linux/blob/master/Documentation/networking/tuntap.txt#L44
Other device types might work well without CAP_NET_ADMIN.

[gsliepen]

Tinc can work without CAP_NET_ADMIN. In fact, it can even be started as a non-root user, even with a tun device: one needs to set up a persistent tun device beforehand, and ensure the non-root user has permissions to use it. To do this, use the following command:

ip tuntap add dev $NETNAME mode tun user $USER

Where you have to fill in $NETNAME and $USER yourself of course. You also have to configure the tun/tap interface before you start tinc, since if it doesn't have CAP_NET_ADMIN, the tinc-up script will most likely fail too.

[Andrei-Pozolotin]

thank you