prototype minikube qemu driver with dedicated network

[medyagh]

What Happened?

https://github.com/lima-vm/vde_vmnet

Attach the log file

n/a

Operating System

No response

Driver

No response

[afbjorklund]

There are some similar solutions to this, that might also be used.

For mac, there is "vde" and "socket". For linux, there is "tap" and "bridge"

(for reference, the default networking is called "user" and uses "slirp" lib)

More details on: https://wiki.qemu.org/Documentation/Networking

---

The main difference is that with "user", your IP is not accessible from host (like Docker Desktop)

This means that every node will be "10.0.2.15", and we need to use tunneling to reach it from host.

But when using "vde" (or the others), you get an IP address reachable from host (like Docker Engine)

This is more similar to the traditional minikube setup (before kic), with things like minikube ip etc

[afbjorklund]

In the qemu driver, this is chosen with the "network" parameters:

 docker-machine create --driver qemu --help | grep net
   --qemu-network "user"										Name of network to connect to (user, tap, bridge)
   --qemu-network-address 										IP of the network adress to be used for networking (for tap)
   --qemu-network-bridge "br0"										Name of the network bridge to be used for networking (for bridge)
   --qemu-network-interface "tap0"									Name of the network interface to be used for networking (for tap)
   --qemu-network-socket "tap0"										Path of the network socket to be used for networking (for vde)

In the driver code, it's in Network and NetworkSocket

afbjorklund/docker-machine-driver-qemu@837a91f

[AkihiroSuda]

vde_vmnet will be deprecated by https://github.com/lima-vm/socket_vmnet

[afbjorklund]

vde_vmnet will be deprecated by https://github.com/lima-vm/socket_vmnet

I think it will work the same, just replace Network: "vde" with Network: "socket"

-netdev vde,id=net0,sock=/var/run/vde.ctl

-netdev socket,id=net0,fd=3 # file descriptor

However, it might require some more code to do the prefix command wrapper ?

/opt/socket_vmnet/bin/socket_vmnet_client /var/run/socket_vmnet ...

There is no such execution parameter, in the current qemu driver at least.

   --qemu-program "qemu-system-x86_64"									Name of program to run

But it shouldn't be a big change, later on.

https://github.com/machine-drivers/docker-machine-driver-qemu (and qemu2)

[AkihiroSuda]

I think it will work the same,

Yes, but socket_vmnet is more efficient (321 Mbps vs 686 Mbps)
lima-vm/lima#851 (comment)

However, it might require some more code to do the prefix command wrapper ?

The command wrapper is not needed if you pass the FD by yourself.

[afbjorklund]

It would need some code to pass the fd, either way :-) And for the network type

[afbjorklund]

If I understand this prototype correctly, the user will be required to set up the network:

sudo vde_vmnet --vmnet-gateway=192.168.105.1 /tmp/vde.ctl

sudo socket_vmnet --vmnet-gateway=192.168.105.1 /tmp/socket_vmnet

And then the socket location will be passed to the minikube start as a qemu parameter

---

In the future, one might imagine the driver managing its own private networks.

That requires a similar admin privileges setup, as other drivers (e.g. libvirt)

It will probably be out of scope for the libmachine driver, but minikube might do it.

There was some similar evolution (of networks), in the container drivers (docker/podman)

[AkihiroSuda]

Btw QEMU 7.1 will add the native support for vmnet, but that requires running the entire QEMU as the root (So I'm not going to use it for Lima)

[afbjorklund]

Sounds good! I don't know how slow slirp ("user") is, compared to the other two, but it will probably stay as the default.