Vpnkit reporting too many open files

[aemengo]

• Operating System (Platform and Version): macOS High Sierra, version: 10.13.3
• vpnkit: version: v0.2.0

When we look into the logs of our vpnkit configuration, we routinely see the following (too many open files error):

==> .../vpnkit.stderr.log <==
...
vpnkit: [ERROR] Hostnet_udp.input: bind raised Too many open files
vpnkit: [ERROR] Hostnet_udp.input: bind raised Too many open files
vpnkit: [ERROR] Hostnet_udp.input: bind raised Too many open files
vpnkit: [ERROR] Hostnet_udp.input: bind raised Too many open files
vpnkit: [ERROR] Hostnet_udp.input: bind raised Too many open files
vpnkit: [ERROR] Hostnet_udp.input: bind raised Too many open files
vpnkit: [ERROR] Hostnet_udp.input: bind raised Too many open files
vpnkit: [ERROR] Hostnet_udp.input: bind raised Too many open files
vpnkit: [ERROR] Hostnet_udp.input: bind raised Too many open files
vpnkit: [ERROR] Hostnet_udp.input: bind raised Too many open files
vpnkit: [ERROR] Hostnet_udp.input: bind raised Too many open files
vpnkit: [ERROR] Hostnet_udp.input: bind raised Too many open files
vpnkit: [ERROR] Hostnet_udp.input: bind raised Too many open files
vpnkit: [ERROR] Hostnet_udp.input: bind raised Too many open files

This manifests in extremely spotty network connectivity to the VM that we are trying to prop up. The simple fix would be to bump our file descriptor limit (currently at 4864) , but we cannot ask our end users to do that.

Are there any straightforward reasons why we are breaching this limit for vpnkit? Many thanks

@sigilite

[djs55]

@aemengo there was recently a file descriptor leak in vpnkit triggered by things like haproxy (but possibly other things too). Take a look here: docker/for-mac#1132 (comment) -- perhaps try with this experimental build to see if anything changes.

Failing that, could you provide some reproduction steps so I can investigate locally? Perhaps a Dockerfile / docker-compose.yml which works with Docker for Mac?

[aemengo]

@djs55 It's proving time-consuming to get a valid example. But something what we believe to be an issue is the following line

vpnkit: [INFO] UDP interface connected on 8.8.8.8

It shows up frequently. If we can set up our own dns server, is there a way to route specific connections or containers inside of a linuxkit built VM running -networking vpnkit to our dns server instead of vpnkit?

[aemengo]

Apologies for the issue. Our team has finally understood the issue.

For future inquiries:

Our custom Linuxkit VM was spawning numerous containers, each with the value of nameserver 8.8.8.8 in their /etc/resolv.conf. This meant that with each container being spawned, when configured with vpnkit, dns resolutions would go through to google but through vpnkit. Changing that nameserver value to nameserver 192.168.65.1 (the default gateway of vpnkit) resolved the issue as well as the suffered network performance that we were experiencing.