Bug: write /dev/net/tun: invalid argument on Ubuntu 16.04.1

[eliezedeck]

Attempting to write the default tunnel will cause this error.

Work-around: Update syscalls_linux.go, the function newTUN to not use the flag cIFF_NO_PI. It will work this way.

That is, the line:

name, err := createInterface(file.Fd(), ifName, cIFF_TUN|cIFF_NO_PI)

will be replaced by:

name, err := createInterface(file.Fd(), ifName, cIFF_TUN)

I remember this situation back in the days I was playing with Linux TUN feature. And this also happened back then. This looks like a bug in the Kernel itself. But I guess, since it hasn't been addressed for years in the mainstream and people have got ways around it, it has been left as is.

Thus, I'm guessing it should be addressed at the library level as well. I have this problem on Ubuntu 16.04.1, but I'm thinking it also happens on other systems.

[eliezedeck]

Just for clarification: this happens only for TUN, and not TAP.

[songgao]

Hi @eliezedeck, what did you write into the TUN interface?

The doc says IFF_NO_PI is "No packet information". If the flag is not set, there would be 4 bytes of header for each packet. Did you assume this 4 bytes header, or did you mean IFF_NO_PI doesn't work as expected?

[eliezedeck]

@songgao Yes, 4 bytes of extra header, as already mentioned in the documentation.

But I think it was because I forwarded packet data from macOS and I fed it to a Ubuntu. This is what caused the issue.

Still, I still consider this a bug. Why?

• On macOS, these 4 bytes header are present
• Default behavior on Linux is using IFF_NO_PI, so, none of these 4 bytes
• Exchanging data packets as is between macOS and Linux will not work reliably. Note that exchanging the packets as is, is quite important for uniformity of use.

Possible fix:

• Wrap the Read() of the Interface: for "darwin" platform, skip 4 bytes for every-read
• Wrap the Write() of the Interface: for "darwin" platform also, prefix every-write with a []byte{0, 0, 0, 2}. That byte sequence is always present on macOS.

I've made myself a simple wrapper to incorporate these behavior. It can be used for reference:

package transport

import (
	"runtime"
	"sync"

	"github.com/songgao/water"
)

type TunnelIO struct {
	Tun       *water.Interface
	readMutex sync.Mutex
	writMutex sync.Mutex
}

func (tr *TunnelIO) Read(b []byte) (n int, err error) {
	tr.readMutex.Lock()
	defer tr.readMutex.Unlock()

	var readDirect [2000]byte

	switch runtime.GOOS {
	case "darwin":

		// Skip the 4 bytes header. See the Write() method below for more information.

		n, err = tr.Tun.Read(readDirect[:])
		if n > 0 {
			copy(b, readDirect[4:n])
		}
		return

	default:
		return tr.Tun.Read(b)
	}
}

func (tr *TunnelIO) Write(p []byte) (n int, err error) {
	tr.writMutex.Lock()
	defer tr.writMutex.Unlock()

	switch runtime.GOOS {
	case "darwin":

		// macOS always adds the 4 bytes packet header, which is not yet the IP packet. It causes confusion because the
		// default behavior on Linux is IFF_NO_PI (which means, none of these 4 bytes header). So, exchanging packets
		// between TUN on macOS and Linux will not be compatible with each other during write, if these are not present.

		var writePrepare [2000]byte

		copy(writePrepare[:], []byte{0, 0, 0, 2})
		n = copy(writePrepare[4:], p)
		_, err = tr.Tun.Write(writePrepare[:4+n])
		return

	default:
		return tr.Tun.Write(p)
	}
}

[songgao]

On macOS, these 4 bytes header are present

That's a good point. I didn't realize this difference. Thanks for pointing that out!

The Linux version of this library has been using IFF_NO_PI since beginning, so simply removing the flag would be a breaking change for software using the library. The macOS support was introduced recently. I'll see if I can get macOS to do the same by default so that forwarding packet between the two would work. In addition, I think Config can have a field indicating whether IFF_NO_PI should be supplied.

Sorry for the inconvenience! Will try to get a solution soon ...

[eliezedeck]

You're welcome @songgao. Thanks for the nice library :)

[cuonglm]

I face the same issue when sending packet from Windows to Linux.

I'm finding windows documentation for this but no luck. Where should I lookup?

[cuonglm]

@songgao @eliezedeck any suggestion?

[songgao]

@Gnouc could you please file a new GH issue with sample code on both platforms? It's probably a different issue than this.

[cuonglm]

@songgao I will try to make a sample code to re-produce the issue. But I guess it's maybe the same. I use govpn, with Linux as server and client connect from a Windows machine. The client code from windows:

package govpn

import (
	"io"
	"strings"

	"github.com/pkg/errors"
	"github.com/songgao/water"
)

func newTAPer(ifaceName *string) (io.ReadWriteCloser, error) {
	if strings.HasPrefix(*ifaceName, interfaceTun) {
		return nil, errors.Wrap(errUnsupportedInterface, *ifaceName)
	}
	config := water.Config{}
	config.DeviceType = water.TAP
	config.PlatformSpecificParams.ComponentID = *ifaceName
	config.PlatformSpecificParams.Network = "192.168.1.10/24"
	output, err := water.New(config)
	return output, errors.Wrap(err, "water.NewTAP")
}

The TAP interface is up normally, I observe log in server and see that it reported the same issue, the govpn t.dev.Write reports:

{"data":67,"error":"t.dev.Write 42: write /dev/net/tun: invalid argument"}

The client can send packet to server but server failed to response.