Repository navigation
VM Testing
Automated QEMU VMs for testing cockpit-compose across three distros × three runtime scenarios,
plus two extra Fedora-only scenarios for rootless/rootful-specific bugs.
The VM harness is provided by @rxtx4816/cockpit-plugin-base-react and invoked via npm run vm <command>. Plugin-specific config (VM names, ports, distro packages) lives in scripts/test-vm.config.sh in this repo.
The harness downloads official cloud images, provisions them with cloud-init, and mounts your local src/ folder live into the VM — so npm run watch changes are visible in the browser without restarting anything.
| Scenario | What's installed | Tests |
|---|---|---|
podman |
podman + podman-compose + docker-compose (standalone) | §7.2 external provider, §7.3 Python compose |
docker |
Docker CE + docker compose v2 plugin | All Docker scenarios |
both |
Both runtimes side by side | Runtime toggle (§6.21), §4 side-by-side |
podman-rootful (Fedora only) |
podman + podman-compose, rootful socket only — no rootless socket, no linger | Regression test for issue #242 (empty project list under system-wide Podman) |
full (Fedora only) |
Docker CE (rootful and rootless) + Podman (rootful and rootless), all four running at once | The Rootless/Rootful toggle itself — switching modes without switching runtime |
VM identifiers are <distro>-<scenario>, e.g. debian-podman, arch-both, fedora-podman-rootful, fedora-full.
Shortcuts — expand to all matching VMs:
-
arch/debian/fedora→ all scenarios for that distro (fedoranow includespodman-rootfulandfulltoo) -
podman/docker/both→ that scenario across all three distros -
all→ all VMs
fedora-fullis experimental. Rootless Docker requires its own separate daemon (dockerd-rootless-setuptool.sh install), unlike Podman where one binary serves both modes. The cloud-init recipe follows Docker's documented rootless install but hasn't been verified end-to-end by actually booting it. If first boot fails, checknpm run vm logs fedora-full, then SSH in and checkjournalctl --user -u dockeras thetestuser (rootless Docker's own logs) — and please report what you find.
Install QEMU and the cloud image tools (Arch Linux):
sudo pacman -S qemu-full cloud-image-utils wgetKVM must be accessible (/dev/kvm) for reasonable boot speed.
The script falls back to software emulation automatically if it isn't, but it will be slow.
# 1. Download cloud base images (~500-700 MB each, one per distro)
npm run vm download debian # just debian
npm run vm download # all three distros
# 2. Build the plugin so src/main.js exists
npm run buildImages are saved to .vms/<distro>/base.qcow2 (gitignored) and shared across all
scenarios for that distro — downloading once covers all three scenarios.
npm run vm start debian-podman # specific VM
npm run vm start debian # all three debian scenarios
npm run vm start podman # podman scenario on all three distros
npm run vm start # all 9 VMs (needs ~18 GB RAM)The VM boots in the background. Block until cloud-init finishes:
npm run vm wait debian-podmanwait polls SSH then runs cloud-init status --wait inside the VM. It warns if
cockpit doesn't come up. Docker VMs take longer (~5 min first boot) because
Docker CE is downloaded from the internet during provisioning.
Then open the URL and accept the self-signed certificate.
| VM | Cockpit | SSH |
|---|---|---|
| arch-podman | https://localhost:9090 | ssh -p 2220 test@localhost |
| arch-docker | https://localhost:9091 | ssh -p 2221 test@localhost |
| arch-both | https://localhost:9092 | ssh -p 2222 test@localhost |
| debian-podman | https://localhost:9093 | ssh -p 2223 test@localhost |
| debian-docker | https://localhost:9094 | ssh -p 2224 test@localhost |
| debian-both | https://localhost:9095 | ssh -p 2225 test@localhost |
| fedora-podman | https://localhost:9096 | ssh -p 2226 test@localhost |
| fedora-docker | https://localhost:9097 | ssh -p 2227 test@localhost |
| fedora-both | https://localhost:9098 | ssh -p 2228 test@localhost |
| fedora-podman-rootful | https://localhost:9099 | ssh -p 2229 test@localhost |
Login: test / test (your ~/.ssh/id_*.pub is also injected automatically if found)
| Command | What it does |
|---|---|
npm run vm status |
Show all 9 VMs with state and ports |
npm run vm download [distro|all] |
Download base cloud images |
npm run vm build |
npm run build shortcut |
npm run vm start <vm|shortcut> |
Start VM(s) in background |
npm run vm wait <vm> |
Block until cloud-init fully finishes |
npm run vm stop <vm|shortcut> |
Stop VM(s) |
npm run vm ssh <vm> |
Open SSH session |
npm run vm logs <vm> |
Tail serial console output |
npm run vm clean <vm|shortcut> |
Wipe disk + re-provision on next start |
npm run vm reset <distro> |
Remove all files including base image |
The src/ directory on your host is mounted read-only into the VM at
/usr/share/cockpit/cockpit-compose via 9p virtfs. Start watch mode on the host
and just refresh the browser in the VM:
npm run watchNo file copying or VM restarts needed.
| Package | podman | docker | both |
|---|---|---|---|
| cockpit | ✓ | ✓ | ✓ |
| passt | ✓ | — | ✓ |
| podman | ✓ | — | ✓ |
| podman-compose | ✓ | — | ✓ |
| docker-compose (standalone, Arch/Debian) | ✓ | — | ✓ |
| Docker CE + compose v2 plugin | — | ✓ | ✓ |
podman scenario notes:
-
docker-composestandalone (v1 on Debian, v2 on Arch) enables thepodman composeexternal provider path even without the Docker daemon — testing guide §7.2 -
podman-compose(Python) covers the no-compose lsfallback — testing guide §7.3 - Fedora gets only
podman-composesincedocker-composestandalone isn't in its default repos
docker scenario notes:
- Docker CE is installed from Docker's official repository during first boot (not distro repos), so first boot takes ~5 min instead of ~2 min
- Adds
testuser to thedockergroup
both scenario notes:
- Both runtimes installed; use the runtime toggle in the UI to switch between them
- Tests testing guide §4 (side by side) and §6.21 (runtime toggle)
podman-rootful scenario notes (Fedora only):
- Same packages as
podman, but the VM deliberately has no rootless Podman socket — nologinctl enable-linger, no user systemd session started, no/run/user/<uid>/podman/podman.sock. Only the system-wide/run/podman/podman.sockexists. - This reproduces issue #242: a
Compose project started with
sudo podman compose up -d(rootful) was invisible in cockpit-compose's project list, because discovery ran unprivileged with no way to reach the root-owned socket. - To test manually:
Then open https://localhost:9099, enable Cockpit "Administrative access", and confirm the stack now appears in the Docker Compose page and can be managed (logs, exec, scale, etc.).
npm run vm start fedora-podman-rootful npm run vm wait fedora-podman-rootful npm run vm ssh fedora-podman-rootful # inside the VM: cd ~/testcompose/gotify # or any other pre-staged fixture under ~/testcompose sudo podman compose up -d
-
Related fix: this scenario also caught a second bug, since it's the first setup where a
Fedora rootful stack actually became visible. Fedora's
podman composedelegates to the standalone Pythonpodman-compose, which records whatever path was passed to-fin thecom.docker.compose.project.config_filescontainer label without resolving it to an absolute path — e.g. a baredocker-compose.ymlif youcd'd into the stack directory first. Every subsequent action (up,down,logs, etc.) then failed withmissing files: ['docker-compose.yml'], since the plugin's own spawned commands don't share that directory as their working directory. Fixed by resolving relativeconfig_filesentries against the (always-absolute)com.docker.compose.project.working_dirlabel at discovery time — seegroupPodmanContainers()insrc/api/stacks/query.ts.
full scenario notes (Fedora only, experimental):
- All four modes running simultaneously: Docker rootful (
systemctl enable --now docker), Docker rootless (dockerd-rootless-setuptool.sh install, running as thetestuser's own systemd service), Podman rootful (podman.socket), Podman rootless (per-user socket). - Intended for exercising the Rootless/Rootful toggle itself — switch Docker or Podman modes without needing to reprovision a different VM. See Podman Compatibility.
- See the experimental-status callout above the scenario table before relying on this one.
-
Known first-boot quirk: on the first VM built this way, Podman's rootless user socket
came up enabled but not actually started (
systemctl --user is-active podman.socketreportedinactive), and rootless Docker's own setup (dockerd-rootless-setuptool.sh install) hadn't produced a usabledocker.servicein the user's systemd session either — only rootful Docker and rootful Podman were confirmed working. If rootless mode doesn't show up as available in the toggle, SSH in and checksystemctl --user status podman.socket/systemctl --user status docker, start them manually if needed, then click Recheck. - Cockpit's Administrative access must be turned on (in the Cockpit page header, not inside the plugin) for anything rootful to work at all on this VM — see Troubleshooting.md's "Stacks don't appear" section, causes 6–7, for what happens if you forget and how the plugin now fails loudly instead of silently misbehaving.
The rootful Podman socket (/run/podman/podman.sock) is enabled and
pasta is configured as the default rootless network backend to avoid
the nftables error common on newer kernels.
VMs use an overlay disk over the base image. The base is never modified.
# Wipe one VM's disk and cloud-init state; next start re-provisions from scratch
npm run vm clean debian-podman
# Wipe all debian VMs at once (shortcut)
npm run vm clean debian
# Remove everything for a distro including the shared base image
npm run vm reset debianclean is the go-to fix when cloud-init fails or you want to test a fresh install
without re-downloading the base image. Each scenario gets its own overlay disk, so
cleaning debian-podman doesn't affect debian-docker.
Note that the Arch image is a rolling release — if you downloaded it a while ago
and want a fresh snapshot, use reset arch then download arch.
VM_MEM=4096 VM_CPUS=4 npm run vm start fedora| Variable | Default | Description |
|---|---|---|
VM_MEM |
1024 |
RAM in MB |
VM_CPUS |
2 |
vCPU count |
VM_DISK_SIZE |
12G |
Overlay disk size |
Starting all 9 VMs at once (npm run vm start) needs ~9 GB of RAM at the
default VM_MEM=1024, plus 18 vCPUs — enough to bog down most dev machines.
There's no built-in concurrency limiter (VMs launch back-to-back as
daemonized QEMU processes), so manage this by controlling what you start:
-
Day-to-day development and most feature tests: use a single distro's
three scenarios instead of the full matrix — behavior for app-level logic
(stack CRUD, YAML editor, etc.) doesn't depend on the distro.
npm run vm start arch # arch-podman, arch-docker, arch-both — ~3 GB RAM npm run vm wait arch npm run test:e2e -- --project=arch-podman --project=arch-docker --project=arch-both npm run vm stop arch
-
Full 9-VM matrix (needed for rootless/rootful and distro-specific
quirks — see testing guide §7): run it in batches, one distro at a time,
rather than starting all 9 concurrently:
This caps peak usage to one distro's VMs instead of all 9 at once.
for distro in arch debian fedora; do npm run vm start "$distro" npm run vm wait "$distro" npm run test:e2e -- --project="$distro-podman" --project="$distro-docker" --project="$distro-both" npm run vm stop "$distro" done
Don't go below the VM_MEM=1024 default — go above it for the heavier
specs. Earlier guidance here suggested VM_MEM=768 was safe since the
fixture stacks are individually lightweight, but live testing during the
issue #227 e2e expansion found the default 1024 MB itself is already too
tight once a spec brings up something like volumes-test (runs Postgres)
or drives a real exec/PTY session (exec.spec.ts) — free -h inside the
guest showed it swapping internally (only ~950 MB total RAM, actively
using swap) under that load, which manifests as flaky timeouts that look
like app bugs but aren't. If you're running logs.spec.ts, exec.spec.ts,
prune.spec.ts, or scale.spec.ts — or the full suite — start those VMs
with more headroom:
VM_MEM=2048 npm run vm start arch-dockerlogs.spec.ts went from consistently failing to consistently passing after
bumping a VM from 1024 → 2048 MB with no code changes. See "Known flaky
behavior" in E2E Test Reference for the full
writeup.
Once a VM is up, run the Playwright E2E suite against it. Each VM is a Playwright project named after its VM identifier — use --project to select which VM(s) to test:
# Start and wait for a VM
npm run vm start debian-podman
npm run vm wait debian-podman
# Run against that specific VM
npm run test:e2e -- --project=debian-podman
# Run against a subset (all must be running)
npm run test:e2e -- --project=arch-podman --project=debian-docker
# Run against all 9 VMs (requires all to be running)
npm run test:e2e
# Visual runner — shows every step, ideal for debugging failures
npm run test:e2e:ui
# Record a new test interactively
BASE_URL=https://localhost:9093 npm run test:e2e:codegenCheck which VMs are currently running before selecting projects:
npm run vm statusTests live in e2e/ and cover the login flow, stack list, status badges, YAML editor, and more. All test stacks are pre-staged during VM provisioning — no manual setup required.
For full documentation including how to write new tests, see the Testing Guide.
wait exits but Cockpit page is not accessible
SSH is available before cloud-init finishes installing packages. Use wait (not just
start) — it runs cloud-init status --wait inside the VM to block until everything
is done. If wait reports a warning, SSH in and check:
npm run vm ssh debian
sudo cloud-init status --long
sudo journalctl -u cloud-init --no-pager -n 50cloud-init reports a package install error (exit code 100)
A package name was wrong or unavailable. The most common cause on Debian is using
docker-compose-v2 (not in default repos) instead of docker-compose. Clean and retry:
npm run vm clean debian
npm run vm start debian
npm run vm wait debianVM won't start / QEMU error about virtfs
qemu-base doesn't include virtfs support. Install qemu-full:
sudo pacman -S qemu-fullCockpit loads but cockpit-compose is missing The 9p mount failed. SSH in and check:
mount | grep cockpit
ls /usr/share/cockpit/cockpit-compose/
# If missing:
sudo modprobe 9p 9pnet 9pnet_virtio
sudo mount /usr/share/cockpit/cockpit-composeFirst boot is very slow
Expected without KVM. Make sure your user is in the kvm group:
sudo usermod -aG kvm $USER # then log out and back in"crun: sd-bus call: Interactive authentication required / OCI permission denied" when starting containers
Rootless Podman needs a systemd user session, but the test user doesn't have one when accessed
via Cockpit without lingering enabled. Fix for a running VM:
npm run vm ssh debian
sudo loginctl enable-linger test
mkdir -p /home/test/.config/containers
echo -e '[network]\ndefault_rootless_network_cmd = "pasta"' > /home/test/.config/containers/containers.confenable-linger gives the user a persistent session. Switching to pasta for networking
removes the dependency on the systemd user.slice entirely (pasta handles everything in userspace).
Both are set automatically on newly provisioned VMs.
"short-name did not resolve to an alias" when pulling images
Podman on Debian and Fedora doesn't search Docker Hub by default. Fix by adding docker.io
as an unqualified search registry in the running VM:
npm run vm ssh debian
echo 'unqualified-search-registries = ["docker.io"]' | sudo tee /etc/containers/registries.conf.d/docker-io.confThis is set automatically on newly provisioned VMs. For an existing VM that's missing it, the one-liner above is enough — no reboot required.
Port already in use
Change the port constants in scripts/test-vm.config.sh:
DEBIAN_SSH_PORT=2221
DEBIAN_COCKPIT_PORT=9091Cockpit Compose
Using the UI
- Stacks Dashboard
- Managing Stacks
- Bulk Actions
- Background Tasks
- Viewing Logs
- Editing Configuration
- Stack Info
- Pulling Images
- Events
- Process Viewer
- Shell Access
- Running Commands
- Prune Resources
- Scaling Services
Stack Management
Runtime Support
Help
Development