Can not access VMs on external usb device - QEMU/KVM - Virtual Machine Manager #4524

sanmue · 2026-04-25T08:02:09Z

sanmue
Apr 25, 2026

Hi,

before switching to bluefin (developer mode enabled), i dumped config for my vms, networks, snapshots (via virsh dumpxml/net-dumpxml/snapshot-dumpxml), which are on an external usb device (/run/media/myUserID/1TB-M2NVME/01_VM/images/).

I was able to restore (except for the ones with uefi secureboot #4550 but thats another thing).

But when tried to start a VM I get the error message, e.g.:

Error starting domain: Cannot access storage file '/run/media/myUserID/1TB-M2NVME/01_VM/images/debianLAMP.qcow2' (as uid:107, gid:107): Permission denied

Traceback (most recent call last):
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 67, in cb_wrapper
    callback(asyncjob, *args, **kwargs)
    ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 101, in tmpcb
    callback(*args, **kwargs)
    ~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/usr/share/virt-manager/virtManager/object/libvirtobject.py", line 57, in newfn
    ret = fn(self, *args, **kwargs)
  File "/usr/share/virt-manager/virtManager/object/domain.py", line 1446, in startup
    self._backend.create()
    ~~~~~~~~~~~~~~~~~~~~^^
  File "/usr/lib64/python3.14/site-packages/libvirt.py", line 1390, in create
    raise libvirtError('virDomainCreate() failed')
libvirt.libvirtError: Cannot access storage file '/run/media/myUserID/1TB-M2NVME/01_VM/images/debianLAMP.qcow2' (as uid:107, gid:107): Permission denied

So I tried changing ownership (root, qemu, libvirt, kvm) and access rights, currently:
sudo ls -lad /run/media/myUserID/1TB-M2NVME/01_VM/images

drwxrwxrwx+ 2 qemu qemu 4096 21. Apr 10:17 /run/media/myUserID/1TB-M2NVME/01_VM/images

sudo ls -la /run/media/myUserID/1TB-M2NVME/01_VM/images

-rwxrwxrwx  1 qemu qemu   8689418240 17. Apr 01:15 debianLAMP.qcow2

mount | grep 1TB-M2NVME

/dev/sdb1 on /run/media/myUserID/1TB-M2NVME type ext4 (rw,nosuid,nodev,relatime,seclabel,errors=remount-ro,uhelper=udisks2)

id

uid=1000(myUserID) gid=1000(myUserID) groups=1000(myUserID),10(wheel),18(dialout),952(docker),954(incus-admin),957(libvirt)

~~sudo usermod -aG qemu,kvm myUserID does not seem to have worked / survived the reboot.~~

EDIT: ok, should have read: https://docs.fedoraproject.org/en-US/atomic-desktops/troubleshooting/#_unable_to_add_user_to_group

When creating a new VM with Virtual Machine Manager in standard path /var/lib/libvirt/images, it works fine.
Copying the qcow img to /var/lib/libvirt/images and adjusting path to /var/lib/libvirt/images/debianLAMP.qcow2 in config of VM, I can start VMs (but not the ones with a current snapshot set, but that's also another step).

In https://docs.projectbluefin.io/bluefin-dx/#virtualization-and-container-runtimes i could not find a tip.

~~SELinux Context Issues~~
Disabled SELinux temp. but no effect.

ls -laZ /run/media/myUserID/1TB-M2NVME/01_VM/images
~~

...
.rwxrwxrwx@   81G qemu qemu ?                                2026-04-19 03:54  bluefin.qcow2
...
.rwxrwxrwx   8,7G qemu qemu system_u:object_r:unlabeled_t:s0 2026-04-17 01:15  debianLAMP.qcow2
...

~~

remark: but before i tried without effect:

adding file context rule:
sudo semanage fcontext -a -t virt_image_t "/run/media/myUserID/1TB-M2NVME/01_VM/images(/.*)?"
and applied context:
sudo restorecon -Rv /run/media/myUserID/1TB-M2NVME/01_VM/images
all files had the virt_image_t context

Looking for problems regarding access rights / mount options next...

klmcwhirter · 2026-04-26T23:21:52Z

klmcwhirter
Apr 26, 2026

When using libvirt VMs (via virsh or virt-manager) I created custom volumes (e.g., from within virt-manager) that live in my ~/.local/virt dir. I have an iso sub-dir (for install media) and vm sub-dir (for qcow2 files). It is a little fiddly to keep the file ownership but it does work. This is primarily because virt-manager wants sudo auth to access the default volume locations. But I didn't dig into it further.

By keeping the dirs separate I can easily choose which of the large files I need on my local hd at any given time - e.g., the iso's got removed once not needed; and so did the qcow2 files when my work on that project was done for the moment. And ~/.local/virt is backed up as part of my standard $HOME backup regimen.

I can only assume you can do something similar with USB external drives as well. But may need some udev and/or automount config to get that custom volume to work reliably. Personally, I would just copy the files to/from the drive as needed if they do not need to be accessed from the ext drive. But I do realize that there are use cases where leaving them on the ext drive might be preferred. I have not tried that though.

Good luck. But do try the custom volume approach to see if that gets you farther.

1 reply

sanmue Apr 27, 2026
Author

Thanks for your reply.

I also have my iso in a seperate dir, but do not use custom vulumes for that so far (mostly only as "shares"). I have 2 other external HD for my backup purposes.

I would like to avoid copy files to/from the drive.

I experimented a bit and found a way (so far except for the ones with UEFI SecureBoot).
I will post the current standing / steps here, also as a documentation for myself :-)

sanmue · 2026-04-27T17:19:28Z

sanmue
Apr 27, 2026
Author

Summary

Workaround: mounting usb drive via fstab to a folder in my home directory
Works so far, except for the VMs with uefi secureboot (TODO)

Sources

Steps

qemu.conf

decided to stay with shipped default /etc/libvirt/qemu.conf
- meaning NOT setting:
  - user or group for QEMU processes run by system instance
  - dynamic_ownership
  - remember_owner

Local dir for mounting usb device via fstab entry

usb device is formatted with ext4
make a directory in my home folder to mount my usb device to
- mkdir -p ~/mnt/VM
get UUID of usb device with sudo blkid
make fstab entry for the usb device and mount it
- sudo vim /etc/fstab
  - fstab entry: see further below
- sudo systemctl daemon-reload
- sudo mount --all
- lsblk to see what we've got
- relevant mount options:
  - rw (read write)
  - exec (allows execution)
  - seclabel (activating SELinux-labeling)
  - nofail (dont stop booting if mounting usb device fails)

# /etc/fstab
# LABEL=1TB-M2NVME # external USB drive for my VMs
UUID=dca13319-4c13-4ecb-af1a-ac82b3c1aca7 /home/myUserID/mnt/VM ext4 rw,exec,seclabel,nosuid,nodev,relatime,errors=remount-ro,uhelper=udisks2,nofail 0 0

Setting access rights

set access rights
- sudo chown -R root:qemu ~/mnt/VM
- sudo chmod -R 770 ~/mnt/VM
- at least setting group to qemu and giving execution-rights (not only rw) was necessary get it running

SELinux context

SELinux part:
- sudo semanage fcontext -a -t svirt_image_t "~/mnt/VM(/.*)?"
  - defines SELinux context rule (persistent): all files and folders in "~/mnt/VM" to get context type "svirt_image_t"
- sudo restorecon -Rv ~/mnt/VM
  - to actually apply the context to the files recursively
  - execute again if files created in the future dont have correct context
- verify:
  - sudo ls -laZ ~/mnt/VM
  - sudo ls -laZ ~/mnt/VM/images

sudo ls -laZ ~/mnt/VM
drwxrwx---@ - root   qemu   system_u:object_r:svirt_image_t:s0 2026-04-25 19:32 images
drwxrwx---@ - root   qemu   system_u:object_r:svirt_image_t:s0 2023-06-07 17:44 qemu

sudo ls -laZ ~/mnt/VM/images
...
-rwxrwx---. 1 root   qemu   system_u:object_r:svirt_image_t:s0      53695545344 26. Apr 11:24 debian.qcow2
...

Adjusting path in the xml files for the VM

adjusting path in the xml files for the VM
- backup files in /etc/libvirt/qemu/
- sudo find /etc/libvirt/qemu/ -name '*.xml' -exec sed -i 's|/old/path/to/images/|/home/myUserID/mnt/VM/images/|g' {} \;
extract of an xml file:

<domain type="kvm">
  <name>debian</name>
  ..
  <devices>
    ...
    <disk type="file" device="disk">
      ...
      <source file="/home/myUserID/mnt/VM/images/debian.qcow2"/>
      ...
    </disk>
    ...
  </devices>
</domain>

Adjusting path for the VM snapshots in their xml files

adjusting path for the VM snapshots too (assuming exported before with virsh snapshot-dumpxml ...):
- find /path/to/xml-snapshotfiles/ -maxdepth 1 -name '<snapshotfile>.xml' -exec sed -i 's|/old/path/to/images/|/home/myUserID/mnt/VM/images/|g' {} \;

Create virtual machines

create virtual machines from xml files (exported before with virsh dumpxml ...)
- virsh define /path/to/xml/<vm>.xml

Redefining snapshots

redefining snapshots
- virsh snapshot-create --redefine <vm-name> /path/to/snapshot-xml/<snapshot>.xml
sudo systemctl restart libvirtd

Adjusting backing file paths in snapshot files

I have external snapshots.

Example

-rwxrwx---. 1 root   qemu      214171648  1. Feb 11:02 debian-102.20260123_2016
-rwxrwx---. 1 root   qemu     1679294464 17. Apr 01:04 debian-102.20260201_1102
-rwxrwx---. 1 root   qemu    10613358592  1. Feb 11:02 debian-102.qcow2

sudo qemu-img info ~/mnt/VM/images/debian-102.20260201_1102

image: /var/home/myUserID/mnt/VM/images/debian-102.20260201_1102
file format: qcow2
...
backing file: /run/media/myUserID/1TB-M2NVME/01_VM/images/debian-102.20260123_2016
backing file format: qcow2
...
Child node '/file':
    filename: /var/home/myUserID/mnt/VM/images/debian-102.20260201_1102
    ...

sudo qemu-img info ~/mnt/VM/images/debian-102.20260123_2016
image: /var/home/myUserID/mnt/VM/images/debian-102.20260123_2016
file format: qcow2
...
backing file: /run/media/myUserID/1TB-M2NVME/01_VM/images/debian-102.qcow2
backing file format: qcow2
...
Child node '/file':
    filename: /var/home/myUserID/mnt/VM/images/debian-102.20260123_2016
    ...

Script to set backing file paths in snapshot files

To change the backing file path in the snapshot files, i had to temporary create symlinks for the old paths, since qemu-img rebase ... checks for the old paths.
But they do not exist anymore, since my external usb drive is now mounted via fstab in a different path.

Script (re)created / optimized by KI.

#!/usr/bin/env bash

VM_IMAGES_DIR_CURRENT="/home/${USER}/mnt/VM/images/" # folder containing qcow2 files, including the (external) snapshots
VM_IMAGES_DIR_OLD="/run/media/${USER}/1TB-M2NVME/01_VM/images/" # old backing file path

# all VM qcow2 files incl. snaphosts are in VM_IMAGES_DIR_CURRENT

# check if path ${VM_IMAGES_DIR_CURRENT} exists:
if [[ ! -d "${VM_IMAGES_DIR_CURRENT}" ]]; then
    echo "Error: Folder '${VM_IMAGES_DIR_CURRENT}' does not exist, exit."
    exit 1
fi

# 1. recreate old paths
# - To change the backing file path in the snapshot files, i had to temporary create symlinks for the old paths, since `qemu-img rebase ...` checks for the old paths.
# - But they do not exist anymore, since my external usb drive is now mounted via fstab in a different path.
sudo mkdir -p "${VM_IMAGES_DIR_OLD}"
sudo chown "${USER}:${USER}" "${VM_IMAGES_DIR_OLD}"

# 2. array for the backing file names, which have to be "created" as symlinks
declare -a backing_files_to_link=()

# 3. collect all backing file names # search all files in folder ${VM_IMAGES_DIR_CURRENT}:
while IFS= read -r -d $'\0' file; do
    backing_file=$(sudo qemu-img info "$file" | grep "backing file:" | sed "s/backing file: //")
    if [ -n "$backing_file" ]; then
        backing_file_name=$(basename "$backing_file")
        if [[ ! " ${backing_files_to_link[*]} " =~ ${backing_file_name} ]]; then
            backing_files_to_link+=("$backing_file_name")
        fi
    fi
done < <(sudo find "${VM_IMAGES_DIR_CURRENT}" -maxdepth 1 -type f -print0)

# 4. temporary create symlinks for all backing file names to the old path
for file in "${backing_files_to_link[@]}"; do
    target="${VM_IMAGES_DIR_CURRENT}${file}"
    link="${VM_IMAGES_DIR_OLD}${file}"

    if sudo test -f "${target}"; then
        echo "Creating symlink: ${link} -> ${target}"
        sudo ln -sf "${target}" "${link}"
    else
        echo "ERROR: Backing file '${target}' does NOT exist at new path!"
        exit 1
    fi
done

# 5. adjust backing file path in snapshot file
while IFS= read -r -d $'\0' file; do
    backing_file=$(sudo qemu-img info "$file" | grep "backing file:" | sed "s/backing file: //")

	# check if backing file exists and if it is not the backing file itself:
    if [ -n "$backing_file" ] && [ "$backing_file" != "$file" ]; then
		# adjust backing file path to the new one (all occurances of VM_IMAGES_DIR_OLD):
        new_backing_file="${backing_file//${VM_IMAGES_DIR_OLD}/${VM_IMAGES_DIR_CURRENT}}"

        # show which file will be adjusted and execute rebase:
        echo "Adjust: ${file} -> Backing File: $(basename "$new_backing_file")"
        if ! sudo qemu-img rebase -F qcow2 -b "${new_backing_file}" "${file}"; then
            echo "Failed to adjust backing file for '${file}'"
            continue
        fi
    else
        echo "Skipping file '${file}': contains no 'backing file' or it is the backing file itself."
    fi
done < <(sudo find "${VM_IMAGES_DIR_CURRENT}" -maxdepth 1 -type f -print0)

# 6. delete temporary created symlinks
for file in "${backing_files_to_link[@]}"; do
    echo "Deleting temporary symlink '${VM_IMAGES_DIR_OLD}${file}'"
    sudo rm -f "${VM_IMAGES_DIR_OLD}${file}"
done

sudo systemctl restart libvirtd

0 replies

Navigation Menu

Can not access VMs on external usb device - QEMU/KVM - Virtual Machine Manager #4524

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Replies: 2 comments 1 reply

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Select a reply

Uh oh!

Can not access VMs on external usb device - QEMU/KVM - Virtual Machine Manager #4524

Uh oh!

Uh oh!

sanmue Apr 25, 2026

Replies: 2 comments · 1 reply

Uh oh!

Uh oh!

klmcwhirter Apr 26, 2026

Uh oh!

sanmue Apr 27, 2026 Author

Uh oh!

Uh oh!

sanmue Apr 27, 2026 Author

Summary

Sources

Steps

qemu.conf

Local dir for mounting usb device via fstab entry

Setting access rights

SELinux context

Adjusting path in the xml files for the VM

Adjusting path for the VM snapshots in their xml files

Create virtual machines

Redefining snapshots

Adjusting backing file paths in snapshot files

Example

Script to set backing file paths in snapshot files

sanmue
Apr 25, 2026

Replies: 2 comments 1 reply

klmcwhirter
Apr 26, 2026

sanmue Apr 27, 2026
Author

sanmue
Apr 27, 2026
Author