proposal: x/sys: permit custom socket implementations

[x1unix]

Brief intro

Hello.

I'm working on VmWare vSockets / VMCI library port from C to Go.

vSockets (previously known as VMCI) are vendor-specific sockets (similar to vsock in Linux) that allow hypervisor and guest communication.
On host side, these sockets behave like regular sockets. The only difference is different socket family number and special sockaddr structure.
socket(), accept() and other methods behave the same.

The problem

I tried to utilize standard methods from syscall package like syscall.Bind and syscall.Accept which behave like eponymous methods from libc but have some extra glue code.

The simplest solution was to import vmci_sockets.h header use cgo but I remember how cgo painful was from previous experience with it, so I decided to move a Go way.

I considered syscall package as the closest low-level alternative to socket-related functions from libc but stumbled upon some limitations.

syscall.Bind

The syscall.Bind method accepts an abstract sockaddr interface which should implement a private Sockaddr.sockaddr method to return raw pointer to a struct and struct size.

As Sockaddr.sockaddr interface cannot be implemented outside of syscall package, currently it's impossible to use this function for vendor-specific sockets.
I believe that this is an artificial constraint because I was able to use an inner private syscall.bind method with some hacks for vSockets socket:

//go:linkname syscallBind syscall.bind
func syscallBind(s syscall.Handle, name unsafe.Pointer, namelen int32) (err error)

afVmci, _ := VMCISock_GetAFValue()
sockFd, _ := syscall.Socket(afVmci, syscall.SOCK_STREAM, 0)
addr := newSockAddr(saFamily(afVmci), uint32(port), VMAddrCIDAny)
addrPtr, ptrSize := addr.sockaddr()

// call to unexported private func
if err := syscallBind(sockFd, addrPtr, ptrSize); err != nil {
    panic(err)
}

if err := syscall.Listen(sockFd, listenBacklogSize); err != nil {
    panic(err)
}

syscall.Accept

Unlike the previous method, syscall.Accept method is more close and only accepts a single parameter - socket file descriptor.
That means, it's impossible to use it for custom sockets (unlike accept() function from libc`)

I tried to take a look how it's implemented in net package and seems like the package uses internal/poll/FD.Accept structure method.

Proposed Actions

I kindly ask the Go contributors and Go team to review my proposal or consider (or recommend) a better solution.

Solution 1

• Make syscall.Sockaddr.sockaddr method public to make possible implementing it for different sockets.
• Add alternative syscall.Accept-like method that also accepts custom sockaddr struct (something like FD.accept described above).

Solution 2

Extend net package to make it able to work with custom sockets.
Currently all low-level socket operations are bound to a poll.FD god object-like structure

Thank you.

[ianlancetaylor]

The syscall package is frozen. I think this suggestion should be for the golang.org/x/sys/unix package instead.

You mention the net package and you also mention syscall.Bind. If you are using syscall.Bind then you are presumably not using the net package. I'm not sure how the net package is affected here.

Can you describe the address family and socket address structures in more detail? Thanks.

[ianlancetaylor]

CC @tklauser @mdlayher

[mdlayher]

Have you seen https://github.com/mdlayher/vsock?

This library might already suit your needs. If not, the changes to implement this specific type would belong in x/sys and a third party wrapper library, rather than the frozen syscall package.

[x1unix]

@mdlayher vsock and vSockets are different things. vsock is Linux related guest-side sockets and vSockets (previously VMCI) are VmWare hypervisor host-side sockets.

I'll change issue title to x/sys, but maybe it can be more suitable for net package which also works with sockets but limited only to unix and tcp/udp sockets.

I saw that you implemented an interesting sockets package but seems it only supports Linux right now.

Do you accept contributions? I would like to add Windows support for my project's needs.

[mdlayher]

vsock and vSockets are different things. vsock is Linux related guest-side sockets and vSockets (previously VMCI) are VmWare hypervisor host-side sockets.

Understood, thanks for clarifying.

I'll change issue title to x/sys, but maybe it can be more suitable for net package which also works with sockets but limited only to unix and tcp/udp sockets.

The general philosophy has been that the standard library should provide extension points but there's no reason to incorporate exotic socket types into the stdlib. My netlink, vsock, and packet sockets packages have lived outside of stdlib for a long time since we can plug into the runtime network poller and implement the typical interfaces without a tighter integration. This means that I can develop independently and not be bound to Go release cycles or compatibility concerns as well.

I saw that you implemented an interesting sockets package but seems it only supports Linux right now.

Do you accept contributions? I would like to add Windows support for my project's needs.

I do accept contributions, but this package is focused exclusively on the BSD sockets API and syscalls found on UNIX-like systems. I don't think Windows APIs would be a good fit. Thanks for offering.

[x1unix]

@ianlancetaylor

You mention the net package and you also mention syscall.Bind. If you are using syscall.Bind then you are presumably not using the net package. I'm not sure how the net package is affected here.

The net package also allows working with sockets, but it's support is strictly limited to TCP, UDP and UNIX sockets.
It's already integrated with runtime network poller but it cannot work with arbitrary socket types.

The net package already does a lot of dirty work of abstracting os-specific sockets API.

VMCI behaves like regular unix sockets but currently it's impossible to just call net.Accept for custom socket types.

Can you describe the address family and socket address structures in more detail? Thanks.

According to the vmci_sockets.h header from open-vm-tools, VMCI is available on Windows, Mac and Linux (as a VmWare hypervisor itself).

I will describe the process for Windows hosts, but I believe that it should be similar on UNIX-like systems (except os-specific calls).

Obtaining an address family

Looks like VMCI doesn't have a specific constant address family and it needs to be obtained from special VMCI kernel object using a special control code.

So, the hypervisor exposes a special kernel object \\.\VMCI that can be queried to obtain hypervisor version, current CID (the same term exists in vsock on Linux) and the address family.

See VMCISock_GetAFValue in C or a Go version of the code.

Here is a full example

Socket address structure

Socket address structure is almost common in Windows, Apple, Linux and FreeBSD operating systems.
The only difference is that on Apple and FreeBSD there is an extra property:

/** \cond PRIVATE */
#if defined(_WIN32) || defined(VMKERNEL)
   typedef unsigned short sa_family_t;
#endif // _WIN32

struct sockaddr_vm {
#if defined(__APPLE__) || defined(__FreeBSD__)
   unsigned char svm_len;
#endif // __APPLE__ || __FreeBSD__

   /** \brief Address family. \see VMCISock_GetAFValueFd() */
   sa_family_t svm_family;

   /** \cond PRIVATE */
   unsigned short svm_reserved1;
   /** \endcond */

   /** \brief Port.  \see VMADDR_PORT_ANY */
   unsigned int svm_port;

   /** \brief Context ID.  \see VMADDR_CID_ANY */
   unsigned int svm_cid;

   /** \cond PRIVATE */
   unsigned char svm_zero[sizeof(struct sockaddr) -
#if defined(__APPLE__)
                             sizeof(unsigned char) -     // svm_len
#endif // __APPLE__
                             sizeof(sa_family_t) -         // svm_family
                             sizeof(unsigned short) -   // svm_reserved1
                             sizeof(unsigned int) -       // svm_port
                             sizeof(unsigned int)];       // svm_cid
   /** \endcond */
};