Skip to content

Virtual Machine Migration using bhyve

Elena Mihailescu edited this page Mar 5, 2019 · 3 revisions

NOTE: The project is under development and may not be completed.

Setup - Clone, build, install

Clone project

root@host # git clone
root@host # cd freebsd
root@host # git checkout projects/bhyve_migration 

Build and install world

root@host # make buildworld -j9
root@host # make installworld -j9
root@host # reboot

Build and install kernel

root@host # make buildkernel -j9
root@host # make installkernel -j9
root@host # reboot

Note: The project should be build and installed on both source and destination hosts. Alternatively, the project can be cloned on the shared storage (set using NFS and described in the next step) and build & installed from there on each system.

Setup - Shared Storage using NFS (Network File System)

To work, the migration process needs that both (source and destination) hosts to share the same image. You can setup a shared storage using NFS following the steps described here. You can setup one of the hosts to be server and the other one to be client.

Setup - Create a FreeBSD virtual machine using bhyve

To create a virtual machine using bhyve, you can use the tutorial from here.

NOTE: It is important the ''guest.img'' created to be shared between the two host systems.

Run the virtual machine on the source host

Run the previous created virtual machine using bhyve.

# load the OS
root@source_host # bhyveload -c stdio -m 512M -d guest.img guest_vm

# start the virtual machine
root@source_host # bhyve -c 1 -m 512M -H -A -P -s 0:0,hostbridge -s 1:0,lpc -s 3:0,virtio-net,tap0 -s 4:0,virtio-blk,guest.img -l com1,stdio guest_vm 
root@guest #

Start the virtual machine for migration on the destination host

In order to migrate a virtual machine, it is necessarily to start an empty virtual machine on the destination system which will wait for connections from certain host on a certain port. To start the virtual machine for migration on the destination host, the ''-R <hostname_or_IP>,[]'' option shall be added to the bhyve command. The default port is 24983 (from "bhyve").

# load the OS
root@destination_host # bhyveload -c stdio -m 512M -d guest.img guest_vm_dst
# start the virtual machine for migration
# note the -R <hostname_or_IP>,[<PORT>] option
root@destination_host # bhyve -c 1 -m 512M -H -A -P -s 0:0,hostbridge -s 1:0,lpc -s 3:0,virtio-net,tap0 -s 4:0,virtio-blk,guest.img -l com1,stdio -R source_host,24983 guest_vm_dst

Start the migration process (from source host)

root@source_host # bhyvectl --migrate=destination_host,24983 --vm=guest_vm

On the source host, the virtual machine will be closed(destroyed). On the destination host, after the migration process is completed, the access to the migrated virtual machine will be granted and a console will start:

root@guest #
You can’t perform that action at this time.