-
Notifications
You must be signed in to change notification settings - Fork 0
Virtual Machine Migration using bhyve
NOTE: The project is under development and may not be completed.
root@host # git clone https://github.com/FreeBSD-UPB/freebsd-src
root@host # cd freebsd
Note: The project should be built and installed on both source and destination hosts. Alternatively, the project can be cloned on the shared storage (using NFS as described in the following step) then built and installed from there on each system.
root@host # git checkout projects/bhyve_live_migration
root@host # make world -DWITH_BHYVE_SNAPSHOT -j`sysctl -n hw.ncpu`
root@host # reboot
# Make sure the BHYVE_SNAPSHOT option is present in the GENERIC file.
root@host # egrep "^options\s+BHYVE_SNAPSHOT" sys/amd64/conf/GENERIC
# Add it:
root@host # echo "options BHYVE_SNAPSHOT" >> sys/amd64/conf/GENERIC
root@host # make kernel -j`sysctl -n hw.ncpu`
root@host # reboot
To work, the migration process needs both (source and destination) hosts to share the same disk image. You can set up a shared storage using NFS following the steps described here. You can set up one of the hosts to be the server and the other one to be the client.
To create a virtual machine using bhyve, you can follow the tutorial from here.
NOTE: It is mandatory for the created guest.img to be shared between the two host systems.
Run the previously created virtual machine using bhyve.
# start the virtual machine
root@source_host # bhyve -c 1 -m 1024M -H -A -P -s 0:0,hostbridge -s 1:0,lpc -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd -s 3:0,ahci,guest.img -l com1,stdio guest_vm
....
root@guest #
In order to migrate a virtual machine (be it warm or live), an empty virtual machine should be started on the destination. It will wait for a connection from the specified host on the specified port.
To start the virtual machine for migration on the destination host, the -R host[:port] option should be added to the bhyve command. The default port is 24983 (from "bhyve").
# start the virtual machine for migration
# note the -R host[:port] option
root@destination_host # bhyve -c 1 -m 1024M -H -A -P -s 0:0,hostbridge -s 1:0,lpc -s 3:0,ahci,guest.img -l com1,stdio -R source_host:24983 guest_vm_dst
To migrate a running virtual machine, run bhyvectl with either --migrate=host[:port] for warm migration or --migrate-live=host[:port] for live migration. The flags are mutually exclusive.
root@source_host # bhyvectl --migrate=destination_host:24983 --vm=guest_vm
root@source_host # bhyvectl --migrate-live=destination_host:24983 --vm=guest_vm
On the source host, the virtual machine will be closed and destroyed. On the destination host, after the migration process is completed, the virtual machine will resume execution from where it left off.