"udevadm trigger" fails in containers with ro /sys

[mbiebl]

systemd version the issue has been seen with

v243

Used distribution

Debian sid

Filed as downstream bug report: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=944860

Since v243, udevadm trigger fails in a container with read-only /sys. I tested both LXC and systemd-nspawn:

v242:

root@buster:~# udevadm trigger --subsystem-match=block --action=change
root@buster:~# echo $?
0

v243:

root@sid:~# udevadm trigger --subsystem-match=block --action=change
Failed to write 'change' to '/sys/devices/pci0000:00/0000:00:1c.6/0000:0e:00.0/usb4/4-1/4-1:1.0/host6/target6:0:0/6:0:0:0/block/sdb/uevent': Read-only file system
Failed to write 'change' to '/sys/devices/pci0000:00/0000:00:1f.2/ata1/host0/target0:0:0/0:0:0:0/block/sda/uevent': Read-only file system
Failed to write 'change' to '/sys/devices/pci0000:00/0000:00:1f.2/ata1/host0/target0:0:0/0:0:0:0/block/sda/sda1/uevent': Read-only file system
Failed to write 'change' to '/sys/devices/pci0000:00/0000:00:1f.2/ata1/host0/target0:0:0/0:0:0:0/block/sda/sda10/uevent': Read-only file system
Failed to write 'change' to '/sys/devices/pci0000:00/0000:00:1f.2/ata1/host0/target0:0:0/0:0:0:0/block/sda/sda11/uevent': Read-only file system
Failed to write 'change' to '/sys/devices/pci0000:00/0000:00:1f.2/ata1/host0/target0:0:0/0:0:0:0/block/sda/sda2/uevent': Read-only file system
Failed to write 'change' to '/sys/devices/pci0000:00/0000:00:1f.2/ata1/host0/target0:0:0/0:0:0:0/block/sda/sda3/uevent': Read-only file system
Failed to write 'change' to '/sys/devices/pci0000:00/0000:00:1f.2/ata1/host0/target0:0:0/0:0:0:0/block/sda/sda4/uevent': Read-only file system
Failed to write 'change' to '/sys/devices/pci0000:00/0000:00:1f.2/ata1/host0/target0:0:0/0:0:0:0/block/sda/sda5/uevent': Read-only file system
Failed to write 'change' to '/sys/devices/pci0000:00/0000:00:1f.2/ata1/host0/target0:0:0/0:0:0:0/block/sda/sda6/uevent': Read-only file system
Failed to write 'change' to '/sys/devices/pci0000:00/0000:00:1f.2/ata1/host0/target0:0:0/0:0:0:0/block/sda/sda7/uevent': Read-only file system
Failed to write 'change' to '/sys/devices/pci0000:00/0000:00:1f.2/ata1/host0/target0:0:0/0:0:0:0/block/sda/sda8/uevent': Read-only file system
Failed to write 'change' to '/sys/devices/pci0000:00/0000:00:1f.2/ata1/host0/target0:0:0/0:0:0:0/block/sda/sda9/uevent': Read-only file system
Failed to write 'change' to '/sys/devices/pci0000:00/0000:00:1f.2/ata2/host1/target1:0:0/1:0:0:0/block/sr0/uevent': Read-only file system
root@sid:~# echo $?
1

This is a result of commit 97afc03

commit 97afc0351a96e0daa83964df33937967c75c644f
Author: Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
Date:   Wed Jun 5 09:54:54 2019 +0200

    udevadm trigger: log errors and return first failure
    
    When udevadm trigger is called, the list of devices to trigger is always
    generated through enumeration, and devices can come and go, so we should not
    treat -ENOENT as a failure. But other types of failure should be logged.
    It seems they were logged until baa30fbc2c04b23209d0b8fb3c86cd15ef9ea81a.
    
    Also, return the first error. (I'm not sure if there are other failure modes
    which we want to ignore. If they are, they'll need to be whitelisted like
    -ENOENT.).

@keszybz this change in behaviour is problematic, as we now have packages failing to install inside containers. We have quite a few packages which ship udev rules and trigger udev after installation.

[mbiebl]

@keszybz
we could update all those packages to guard the invocation of udevadm trigger with systemd-detect-virt. But this will mean a lot of busywork and boilerplate code.
I'd prefer if udevadm trigger would behave sensibly ootb with a ro /sys (in a similar way how it skips execution if run in a chroot)

[poettering]

I figure we should just exit early in udevadm trigger when we detect we are run in a container

[serfreeman1337]

Can we re-open this ?
Or maybe not. I think it's ubuntu issue, not systemd.

88da55e commit reverted required change.

Ubuntu's netplan fails to run inside systemd-nspawn container:

$ netplan apply
host0: Failed to write 'change' to '/sys/devices/virtual/net/host0/uevent': Read-only file system
subprocess.CalledProcessError: Command '['udevadm', 'trigger', '--attr-match=subsystem=net']' returned non-zero exit status 1.

[scramblr]

I am having the same issue on Debian. It's a systemd issue.