Skip to content

Latest commit

 

History

History
166 lines (112 loc) · 4.22 KB

qemu.md

File metadata and controls

166 lines (112 loc) · 4.22 KB
Raw

Local image testing with QEMU / libvirt

To create local testing clusters using QEMU, some prerequisites have to be met:

Deploying the VMs requires libvirt to be installed and configured correctly. You may either use your local libvirt setup if it meets the requirements, or use a containerized libvirt in docker.

Containerized libvirt

Constellation will automatically deploy a containerized libvirt instance, if no connection URI is defined in the Constellation config file. Follow the steps in our libvirt readme if you wish to build your own image.

Local libvirt setup

Ubuntu

Install required packages

General reference

sudo apt install qemu-kvm libvirt-daemon-system xsltproc
sudo systemctl enable libvirtd
sudo usermod -a -G libvirt $USER
# reboot

Setup emulated TPM

Using a virtual TPM (vTPM) with QEMU only works if swtpm is version 0.7 or newer! Ubuntu 22.04 currently ships swtpm 0.6.3, so you need to install swtpm from launchpad.

  1. Uninstall current version of swtpm (if installed)

    sudo apt remove swtpm swtpm-tools

  2. Add ppa (this command shows the ppa for Ubuntu 22.04 jammy but others are available)

    sudo add-apt-repository ppa:stefanberger/swtpm-jammy
sudo apt update

  3. Install swtpm

    sudo apt install swtpm swtpm-tools

  4. Patch configuration under /etc/swtpm_setup.conf

    # Program invoked for creating certificates
create_certs_tool = /usr/bin/swtpm_localca

  5. Patch ownership of /var/lib/swtpm-localca

    sudo chown -R swtpm:root /var/lib/swtpm-localca
Fedora 
sudo dnf install -y dnf-plugins-core
sudo dnf -y install qemu-kvm libvirt-daemon-config-network libvirt-daemon-kvm xsltproc swtpm
sudo usermod -a -G libvirt $USER
# reboot

Update libvirt settings

Open /etc/libvirt/qemu.conf and change the following settings:

security_driver = "none"

Then restart libvirt

sudo systemctl restart libvirtd

Connecting to a console

Once you created a cluster via constellation you can connect to the virtual machines using the following commands. First, run

virsh -c qemu+tcp://localhost:16599/system

which gives you a new virsh shell. In this shell, you can run

list

to see the created virtual machines, as well as their status. This should look something like this:

Id   Name                       State
------------------------------------------
1    constell-control-plane-0   running
2    constell-worker-0          running

If you want to connect to one of those machines, run

console <ID>

where <ID> would be the id of your virtual machine as seen above (e.g. 1). After that, press Enter another time and you should drop into a shell on the virtual machine.

Please note that connecting via qemu+tcp is not encrypted and should not be used in a security relevant environment.

Troubleshooting

VMs are not properly cleaned up after a failed constellation create command

Terraform may fail to remove your VMs, in which case you need to do so manually.

  • List all domains: virsh list --all
  • Destroy domains with nvram: virsh undefine --nvram <name>

VMs have no internet access

iptables rules may prevent your VMs form properly accessing the internet. Make sure your rules are not dropping forwarded packages.

List your rules:

sudo iptables -S

The output may look similar to the following:

-P INPUT ACCEPT
-P FORWARD DROP
-P OUTPUT ACCEPT
-N DOCKER
-N DOCKER-ISOLATION-STAGE-1
-N DOCKER-ISOLATION-STAGE-2
-N DOCKER-USER

If your FORWARD chain is set to DROP, you will need to update your rules:

sudo iptables -P FORWARD ACCEPT