Skip to content

Working with virtual machines in bhyve for USB passthrough

Jump to bottom
nainapatrascu edited this page Sep 3, 2022 · 20 revisions

Creating and running a virtual machine

In order to create a virtual machine on the FreeBSD host using the bhyve hypervisor, first you need to create a guest.img file using the following command:

truncate -s 16G guest.img

Then, you need to download the iso file with the desired guest OS (FreeBSD or Linux) and to rename it into disk.iso.

To create the virtual machine, use the create_vm.sh script in order to install the guest OS in the <guest.img> virtual file system.

create_vm.sh

#!/bin/sh

CRTSCRIPT=`readlink -f $0`
BASEDIR=${CRTSCRIPT%/*}

if [ $# -lt 3 ]
then
    echo "Usage: sh $0 <guest.img> <file.iso> <machine>"
    exit 1
fi

kldload vmm
${BASEDIR}/tap.sh

bhyve \
    -A -H -P \
    -W \
    -c 1 \
    -m 1G \
    -w \
    -s 0:0,hostbridge \
    -s 31,lpc \
    -l com1,stdio \
    -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
    -s 29,fbuf,tcp=0.0.0.0:5901,w=800,h=600,wait \
    -s 4,ahci-hd,"$1"  \
    -s 5,virtio-net,tap0  \
    -s 6,ahci-cd,"$2" \
    $3

Create the virtual machine having the name "vm0":

./create_vm.sh guest.img disk.iso vm0

In this script core's number is hardcoded to 2 (-c 2) and VM's RAM memory is limited to 1G (-m 1G). When using a virtual machine, first we need to load the kernel module (kldload vmm) and then to run the virtual machine. We suppose that tap interface is tap0, created using tap.sh script.

tap.sh

#!/bin/sh

ifconfig tap0 destroy
ifconfig bridge0 destroy

ifconfig tap0 create
sysctl net.link.tap.up_on_open=1
ifconfig bridge0 create
ifconfig bridge0 addm re0 addm tap0
ifconfig bridge0 up

After creating the guest virtual machine, you can start it and specify the USB device to be attached to it.

On the FreeBSD host, list the available USB devices and save the information about the desired USB device:

# usbconfig 
ugen2.1: <Intel UHCI root HUB> at usbus2, cfg=0 md=HOST spd=FULL (12Mbps) pwr=SAVE (0mA)
ugen3.1: <Intel EHCI root HUB> at usbus3, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=SAVE (0mA)
ugen0.1: <Intel UHCI root HUB> at usbus0, cfg=0 md=HOST spd=FULL (12Mbps) pwr=SAVE (0mA)
ugen1.1: <Intel UHCI root HUB> at usbus1, cfg=0 md=HOST spd=FULL (12Mbps) pwr=SAVE (0mA)
ugen3.2: <ALCOR Intenso Rainbow Line> at usbus3, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=ON (200mA)

Considering "ALCOR Intenso Rainbow Line" as the USB device that we want to attach to the guest, we should save the bus and address numbers. As the device is identified through ugen3.2, the bus number is 3 and the device address is 2. These two numbers need to be specified to the bhyve command when starting again the virtual machine. In the run_vm.sh script, the line -s 30,xhci,bus=1,devaddr=2 indicates that we want to attach the USB devices identified though the bus number 3 and device address 2 to the slot 30 of the virtual machine. The xhci parameter specifies that we want to use the emulation of the xHCI controller.

run_vm.sh

#!/bin/sh

CRTSCRIPT=`readlink -f $0`
BASEDIR=${CRTSCRIPT%/*}

if [ $# -lt 2 ]
then
    echo "Usage: sh $0 <guest.img> <machine>"
    exit 1
fi

# We assure that we have the tap0 interface created
${BASEDIR}/tap.sh

# Load vm module
kldload vmm

# Load kernel
#bhyveload -c stdio -m 512M -d $1 $2

# Run virtual machine
(bhyve \
    -A -H -P \
    -W \
    -c 1 \
    -m 1G \
    -w \
    -s 0:0,hostbridge \
    -s 31,lpc \
    -l com1,stdio \
    -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
    -s 29,fbuf,tcp=0.0.0.0:5901,w=800,h=600,wait \
    -s 4,ahci-hd,"$1"  \
    -s 5,virtio-net,tap0  \
    -s 30,xhci,bus=1,devaddr=2 \
    $2 | tee log.stdout) 3>&1 1>&2 2>&3 | tee log.stderr

The scripts throws stdout logs to log.stdout file and stderr logs to log.stderr file. In order to monitor the files in real time, use:

tail -f log.stdout

Bhyve starts a vnc server and you need to connect with a vnc client to port 5901, so you can connect to the virtual machine:

vncviewer <IP_OF_HOST_FREEBSD>:5901

To destroy a virtual machine use the following command:

bhyvectl --vm=<vm_name> --destroy

NOTE: If the following error is thrown while running create_vm.sh for the first time: Error opening bootrom "/usr/local/share/uefi-firmware/BHYVE_UEFI.fd": No such file or directory then run the following command: pkg install bhyve-firmware

USB keyboard VS USB storage device

  1. USB keyboard passthrough

    When running a bhyve guest with passthrough of a USB keyboard, no other configurations are required once the virtual machine is started. You should be able to use the USB device directly: press the keys of the USB keyboard and follow the display in the console of the virtual machine.

  2. USB storage device passthrough

    When attaching a USB stick to a guest machine, the device cannot be directly used inside the guest, it has to be mounted.

    Example: Mounting a USB device inside a FreeBSD guest

    • Check the presence of the USB device:
    usbconfig
    • Use lsblk command to identify the disk that has been assigned for the USB device by the guest:
    # lsblk
DEVICE         MAJ:MIN SIZE TYPE                                          LABEL MOUNT
ada0             0:97   16G GPT                                               - -
  <FREE>         -:-    12K -                                                 - -
  ada0p1         0:98  260M efi                                               - /boot/efi
  ada0p2         0:99   15G freebsd-ufs                                       - /
  ada0p3         0:100 819M freebsd-swap                                      - SWAP
  <FREE>         -:-   205M -                                                 - -
da0              0:114  15G MBR                                               - -
  <FREE>         -:-   993K -                                                 - -
  da0s1          0:115  15G linux-data                                        - -

    da0s1 is the disk assigned for my 15G storage device.

    • Mount the USB device in a desired location (for example, /media/usb):
    mount -t msdosfs /dev/da0s1 /media/usb

    Then, you should be able to access the content of the USB device in the /media/usb directory.

NOTE: The detaching mechanism of a USB device is not currently implemented. So, when powering off the guest machine, it is required to reboot the FreeBSD host in order to completely detach the USB device from the guest and to be able to use it again.

Clone this wiki locally