Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

logind TakeDevice fails for Xen Virtual Keyboard #22944

Open
jandryuk opened this issue Apr 1, 2022 · 18 comments
Open

logind TakeDevice fails for Xen Virtual Keyboard #22944

jandryuk opened this issue Apr 1, 2022 · 18 comments

Comments

@jandryuk
Copy link

jandryuk commented Apr 1, 2022

systemd version the issue has been seen with

systemd-249.9

Used distribution

Fedora 35

Linux kernel version used (uname -a)

Linux fedora 5.16.16-200.fc35.x86_64 #1 SMP PREEMPT Sat Mar 19 13:52:41 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

CPU architecture issue was seen on

x86-64

Expected behaviour you didn't see

TakeDevice fails when gnome-shell calls it for the Xen Virtual Keyboard (driver xen-kbdfront). This means there is no keyboard in the VM. This may be more of a udev labeling issue than a logind issue.

Apr 01 12:55:09 fedora gnome-shell[937]: Could not open device /dev/input/event5: GDBus.Error:System.Error.ENODEV: No such device

The "Xen Virtual Pointer" and "Xen Virtual Multi-touch" devices are "taken" properly. Only mouse (Xen Virtual Pointer) is actually transmitting input events though since the multi-touch backend is not connected. "Xen Virtual Keyboard" is sending input properly according to libinput debug-events.

Adding SYSTEMD_LOG_LEVEL=debug to logind does not add any useful information for why TakeDevice fails.

event5 has parent input6. strace on logind shows failure to read udev data before sending the ENODEV error:

683   openat(AT_FDCWD, "/run/udev/data/+input:input6", O_RDONLY|O_CLOEXEC) = -1 
ENOENT (No such file or directory)
683   openat(AT_FDCWD, "/run/udev/data/+input:input6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
683   sendmsg(13, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="l\3\1\1\23\0\0\0\305\1\0\0?\0\0\0\5\1u\0/\0\0\0\6\1s\0\6\0\0\0:1.150\0\0\4\1s\0\23\0\0\0System.Error.ENODEV\0\0\0\0\0\10\1g\0\1s\0\0", iov_len=80}, {iov_base="\16\0\0\0No such device\0", iov_len=19}], msg_iovlen=2, msg_controllen=0, msg_flags=0}, MSG_DONTWAIT|MSG_NOSIGNAL) = 99

There is data in the major:minor, /run/udev/data/c13:69, but not for the parent. The Xen Pointer has both major:minor and parent (input7) entries in /run/udev/data.

The strace failure seems to be session_device_verify calling manager_process_seat_device which returns from here

/* Ignore non-master devices for unknown seats */
if (!master && !seat)
return 0;

That means the keyboard isn't registered so it returns ENODEV here

/* if it's still not available, then the device is invalid */
sd->device = hashmap_get(sd->session->manager->devices, sp);
if (!sd->device)
return -ENODEV;

The Xen Virtual Keyboard driver, xen-kbdfront, is unusual as it registers as being able to generate every input. Partially this is problematic as it is therefore detected as a joystick as seen in the udevadm info output below.

I tried hacking up the udev rules to prevent the joystick and uaccess labeling, but that was insufficient. TakeDevice still failed.

Any pointers appreciated.

Unexpected behaviour you saw

TakeDevice failing for /dev/input/event5

Steps to reproduce the problem

Run gnome-shell in a Xen VM with a PV keyboard (vkbd device). Setting that up is non-trivial. I am happy to help debug.

Additional program output to the terminal or log subsystem illustrating the issue

[root@fedora ~]# udevadm info /dev/input/event5
P: /devices/virtual/input/input6/event5
N: input/event5
L: 0
E: DEVPATH=/devices/virtual/input/input6/event5
E: DEVNAME=/dev/input/event5
E: MAJOR=13
E: MINOR=69
E: SUBSYSTEM=input
E: USEC_INITIALIZED=6824812
E: ID_INPUT=1
E: ID_INPUT_JOYSTICK=1
E: ID_INPUT_KEY=1
E: ID_INPUT_KEYBOARD=1
E: ID_SERIAL=noserial
E: LIBINPUT_DEVICE_GROUP=0/0/0:xenbus/device/vkbd/0
E: TAGS=:power-switch:seat:uaccess:
E: CURRENT_TAGS=:power-switch:seat:uaccess:
[root@fedora ~]# cat /run/udev/data/c13\:69 
I:6824812
E:ID_INPUT=1
E:ID_INPUT_JOYSTICK=1
E:ID_INPUT_KEY=1
E:ID_INPUT_KEYBOARD=1
E:ID_SERIAL=noserial
E:LIBINPUT_DEVICE_GROUP=0/0/0:xenbus/device/vkbd/0
G:uaccess
G:seat
G:power-switch
Q:uaccess
Q:seat
Q:power-switch
V:1
@jandryuk
Copy link
Author

jandryuk commented Apr 1, 2022

Full strace:
systemd-logind.strace.txt

udevadm test output:
udevadm-test-event5.txt

@poettering
Copy link
Member

my educated guess is that you don#t actually have a working seat, and that's the precondition for devices to be able to be taken.

What does "loginctl seat-status seat0" say?

@poettering poettering added the needs-reporter-feedback ❓ There's an unanswered question, the reporter needs to answer label Apr 26, 2022
@jandryuk
Copy link
Author

Here is the output. Thank you for taking a look.

seat0
        Sessions: *2
         Devices:
                  ├─/sys/devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
                  │ input:input0 "Power Button"
                  ├─/sys/devices/LNXSYSTM:00/LNXSLPBN:00/input/input1
                  │ input:input1 "Sleep Button"
                  ├─/sys/devices/pci0000:00/0000:00:01.1/ata2/host1/target1:0:0/1:0:0:0/block/sr0
                  │ block:sr0
                  ├─/sys/devices/pci0000:00/0000:00:01.1/ata2/host1/target1:0:0/1:0:0:0/scsi_generic/sg0
                  │ scsi_generic:sg0
                  ├─/sys/devices/pci0000:00/0000:00:03.0/sound/card0
                  │ sound:card0 "I82801AAICH"
                  ├─/sys/devices/pci0000:00/0000:00:04.0/drm/card0
                  │ [MASTER] drm:card0
                  │ └─/sys/devices/pci0000:00/0000:00:04.0/drm/card0/card0-Virtual-1
                  │   [MASTER] drm:card0-Virtual-1
                  ├─/sys/devices/pci0000:00/0000:00:04.0/graphics/fb0
                  │ graphics:fb0 "bochs-drmdrmfb"
                  ├─/sys/devices/platform/i8042/serio0/input/input2
                  │ input:input2 "AT Translated Set 2 keyboard"
                  │ ├─/sys/devices/platform/i8042/serio0/input/input2/input2::capslock
                  │ │ leds:input2::capslock
                  │ ├─/sys/devices/platform/i8042/serio0/input/input2/input2::numlock
                  │ │ leds:input2::numlock
                  │ └─/sys/devices/platform/i8042/serio0/input/input2/input2::scrolllock
                  │   leds:input2::scrolllock
                  ├─/sys/devices/platform/i8042/serio1/input/input4
                  │ input:input4 "ImExPS/2 Generic Explorer Mouse"
                  ├─/sys/devices/platform/pcspkr/input/input5
                  │ input:input5 "PC Speaker"
                  ├─/sys/devices/virtual/input/input6/event5
                  │ input:event5
                  ├─/sys/devices/virtual/input/input6/js0
                  │ input:js0
                  ├─/sys/devices/virtual/input/input7
                  │ input:input7 "Xen Virtual Pointer"
                  ├─/sys/devices/virtual/input/input8
                  │ input:input8 "Xen Virtual Multi-touch"
                  └─/sys/devices/virtual/misc/rfkill
                    misc:rfkill

@poettering
Copy link
Member

hmm, interesting.

but i think you are right, this has something to do with the fact that the parent input device is not matched somehow and thus doesn#t get a udev input device. That's something to look into.

what does "udevadm info /sys/class/input/input6" say? i.e. on the parent input device? It needs to have the "seat" tag assigned, and I figure this is missing for you, probably because it is on such a weird bus.

what happens if you do udevadm test /sys/class/input/input6?

@jandryuk
Copy link
Author

Here you go:

[root@fedora ~]# udevadm info /sys/class/input/input6 
P: /devices/virtual/input/input6
L: 0
E: DEVPATH=/devices/virtual/input/input6
E: SUBSYSTEM=input
E: USEC_INITIALIZED=3589803915
E: ID_INPUT=1
E: ID_INPUT_JOYSTICK=1
E: ID_INPUT_KEY=1
E: ID_INPUT_KEYBOARD=1
E: ID_SERIAL=noserial
E: TAGS=:seat:
E: CURRENT_TAGS=:seat:
[root@fedora ~]$ udevadm test /sys/class/input/input6 > input6-test 2>&1
[root@fedora ~]$ less input6-test 
Trying to open "/etc/systemd/hwdb/hwdb.bin"...
Trying to open "/etc/udev/hwdb.bin"...
=== trie on-disk ===
tool version:          249
file size:        11631390 bytes
header size             80 bytes
strings            2383374 bytes
nodes              9247936 bytes
Load module index
Found cgroup2 on /sys/fs/cgroup/, full unified hierarchy
Found container virtualization none.
Loaded timestamp for '/etc/systemd/network'.
Loaded timestamp for '/usr/lib/systemd/network'.
Parsed configuration file /usr/lib/systemd/network/99-default.link
Created link configuration context.
Loaded timestamp for '/etc/udev/rules.d'.
Loaded timestamp for '/usr/lib/udev/rules.d'.
Reading rules file: /usr/lib/udev/rules.d/01-md-raid-creating.rules
Reading rules file: /usr/lib/udev/rules.d/10-dm.rules
Reading rules file: /usr/lib/udev/rules.d/11-dm-lvm.rules
Reading rules file: /usr/lib/udev/rules.d/11-dm-mpath.rules
Reading rules file: /usr/lib/udev/rules.d/11-dm-parts.rules
Reading rules file: /usr/lib/udev/rules.d/13-dm-disk.rules
Reading rules file: /usr/lib/udev/rules.d/39-usbmuxd.rules
Reading rules file: /usr/lib/udev/rules.d/40-libgphoto2.rules
Reading rules file: /usr/lib/udev/rules.d/40-usb-media-players.rules
Reading rules file: /usr/lib/udev/rules.d/40-usb_modeswitch.rules
Reading rules file: /usr/lib/udev/rules.d/50-udev-default.rules
Reading rules file: /usr/lib/udev/rules.d/56-hpmud.rules
Reading rules file: /usr/lib/udev/rules.d/60-autosuspend.rules
Reading rules file: /usr/lib/udev/rules.d/60-block-scheduler.rules
Reading rules file: /usr/lib/udev/rules.d/60-block.rules
Reading rules file: /usr/lib/udev/rules.d/60-cdrom_id.rules
Reading rules file: /usr/lib/udev/rules.d/60-drm.rules
Reading rules file: /usr/lib/udev/rules.d/60-evdev.rules
Reading rules file: /usr/lib/udev/rules.d/60-fido-id.rules
Reading rules file: /usr/lib/udev/rules.d/60-input-id.rules
Reading rules file: /usr/lib/udev/rules.d/60-libjaylink.rules
Reading rules file: /usr/lib/udev/rules.d/60-mdevctl.rules
Reading rules file: /usr/lib/udev/rules.d/60-net.rules
Reading rules file: /usr/lib/udev/rules.d/60-persistent-alsa.rules
Reading rules file: /usr/lib/udev/rules.d/60-persistent-input.rules
Reading rules file: /usr/lib/udev/rules.d/60-persistent-storage-tape.rules
Reading rules file: /usr/lib/udev/rules.d/60-persistent-storage.rules
Reading rules file: /usr/lib/udev/rules.d/60-persistent-v4l.rules
Reading rules file: /usr/lib/udev/rules.d/60-sensor.rules
Reading rules file: /usr/lib/udev/rules.d/60-serial.rules
Reading rules file: /usr/lib/udev/rules.d/60-tpm-udev.rules
Reading rules file: /usr/lib/udev/rules.d/60-vboxguest.rules
Reading rules file: /usr/lib/udev/rules.d/60_flashrom.rules
Reading rules file: /usr/lib/udev/rules.d/61-gdm.rules
Reading rules file: /usr/lib/udev/rules.d/61-gnome-bluetooth-rfkill.rules
Reading rules file: /usr/lib/udev/rules.d/61-gnome-settings-daemon-rfkill.rules
Reading rules file: /usr/lib/udev/rules.d/61-mutter.rules
Reading rules file: /usr/lib/udev/rules.d/62-multipath.rules
Reading rules file: /usr/lib/udev/rules.d/63-md-raid-arrays.rules
Reading rules file: /usr/lib/udev/rules.d/64-btrfs-dm.rules
Reading rules file: /usr/lib/udev/rules.d/64-btrfs-zoned.rules
Reading rules file: /usr/lib/udev/rules.d/64-btrfs.rules
Reading rules file: /usr/lib/udev/rules.d/64-md-raid-assembly.rules
Reading rules file: /usr/lib/udev/rules.d/65-libwacom.rules
Reading rules file: /usr/lib/udev/rules.d/65-md-incremental.rules
Reading rules file: /usr/lib/udev/rules.d/65-sane-backends.rules
Reading rules file: /usr/lib/udev/rules.d/66-kpartx.rules
Reading rules file: /usr/lib/udev/rules.d/68-del-part-nodes.rules
Reading rules file: /usr/lib/udev/rules.d/69-bcache.rules
Reading rules file: /usr/lib/udev/rules.d/69-cd-sensors.rules
Reading rules file: /usr/lib/udev/rules.d/69-dm-lvm-metad.rules
Reading rules file: /usr/lib/udev/rules.d/69-libftdi.rules
Reading rules file: /usr/lib/udev/rules.d/69-libmtp.rules
Reading rules file: /usr/lib/udev/rules.d/69-md-clustered-confirm-device.rules
Reading rules file: /usr/lib/udev/rules.d/70-hypervfcopy.rules
Reading rules file: /usr/lib/udev/rules.d/70-hypervkvp.rules
Reading rules file: /usr/lib/udev/rules.d/70-hypervvss.rules
Reading rules file: /usr/lib/udev/rules.d/70-joystick.rules
Reading rules file: /usr/lib/udev/rules.d/70-libfprint-2.rules
Reading rules file: /usr/lib/udev/rules.d/70-memory.rules
Reading rules file: /usr/lib/udev/rules.d/70-mouse.rules
Reading rules file: /usr/lib/udev/rules.d/70-power-switch.rules
Reading rules file: /usr/lib/udev/rules.d/70-printers.rules
Reading rules file: /usr/lib/udev/rules.d/70-spice-vdagentd.rules
Reading rules file: /usr/lib/udev/rules.d/70-touchpad.rules
Reading rules file: /usr/lib/udev/rules.d/70-uaccess.rules
Reading rules file: /usr/lib/udev/rules.d/70-wacom.rules
Reading rules file: /usr/lib/udev/rules.d/71-seat.rules
Reading rules file: /usr/lib/udev/rules.d/73-seat-late.rules
Reading rules file: /usr/lib/udev/rules.d/75-net-description.rules
Reading rules file: /usr/lib/udev/rules.d/75-probe_mtd.rules
Reading rules file: /usr/lib/udev/rules.d/77-mm-broadmobi-port-types.rules
Reading rules file: /usr/lib/udev/rules.d/77-mm-cinterion-port-types.rules
Reading rules file: /usr/lib/udev/rules.d/77-mm-dell-port-types.rules
Reading rules file: /usr/lib/udev/rules.d/77-mm-dlink-port-types.rules
Reading rules file: /usr/lib/udev/rules.d/77-mm-ericsson-mbm.rules
Reading rules file: /usr/lib/udev/rules.d/77-mm-fibocom-port-types.rules
Reading rules file: /usr/lib/udev/rules.d/77-mm-foxconn-port-types.rules
Reading rules file: /usr/lib/udev/rules.d/77-mm-gosuncn-port-types.rules
Reading rules file: /usr/lib/udev/rules.d/77-mm-haier-port-types.rules
Reading rules file: /usr/lib/udev/rules.d/77-mm-huawei-net-port-types.rules
Reading rules file: /usr/lib/udev/rules.d/77-mm-longcheer-port-types.rules
Reading rules file: /usr/lib/udev/rules.d/77-mm-mtk-port-types.rules
Reading rules file: /usr/lib/udev/rules.d/77-mm-nokia-port-types.rules
Reading rules file: /usr/lib/udev/rules.d/77-mm-quectel-port-types.rules
Reading rules file: /usr/lib/udev/rules.d/77-mm-sierra.rules
Reading rules file: /usr/lib/udev/rules.d/77-mm-simtech-port-types.rules
Reading rules file: /usr/lib/udev/rules.d/77-mm-telit-port-types.rules
Reading rules file: /usr/lib/udev/rules.d/77-mm-tplink-port-types.rules
Reading rules file: /usr/lib/udev/rules.d/77-mm-ublox-port-types.rules
Reading rules file: /usr/lib/udev/rules.d/77-mm-x22x-port-types.rules
Reading rules file: /usr/lib/udev/rules.d/77-mm-zte-port-types.rules
Reading rules file: /usr/lib/udev/rules.d/78-sound-card.rules
Reading rules file: /usr/lib/udev/rules.d/80-drivers.rules
Reading rules file: /usr/lib/udev/rules.d/80-iio-sensor-proxy.rules
Reading rules file: /usr/lib/udev/rules.d/80-libinput-device-groups.rules
Reading rules file: /usr/lib/udev/rules.d/80-mm-candidate.rules
Reading rules file: /usr/lib/udev/rules.d/80-net-setup-link.rules
Reading rules file: /usr/lib/udev/rules.d/80-udisks2.rules
Reading rules file: /usr/lib/udev/rules.d/81-net-dhcp.rules
Reading rules file: /usr/lib/udev/rules.d/84-nm-drivers.rules
Reading rules file: /usr/lib/udev/rules.d/85-nm-unmanaged.rules
Reading rules file: /usr/lib/udev/rules.d/85-regulatory.rules
Reading rules file: /usr/lib/udev/rules.d/90-alsa-restore.rules
Reading rules file: /usr/lib/udev/rules.d/90-bolt.rules
Reading rules file: /usr/lib/udev/rules.d/90-fwupd-devices.rules
Reading rules file: /usr/lib/udev/rules.d/90-libgpod.rules
Reading rules file: /usr/lib/udev/rules.d/90-libinput-fuzz-override.rules
Reading rules file: /usr/lib/udev/rules.d/90-nm-thunderbolt.rules
Reading rules file: /usr/lib/udev/rules.d/90-pipewire-alsa.rules
Reading rules file: /usr/lib/udev/rules.d/90-vconsole.rules
Reading rules file: /usr/lib/udev/rules.d/91-drm-modeset.rules
Reading rules file: /usr/lib/udev/rules.d/95-cd-devices.rules
Reading rules file: /usr/lib/udev/rules.d/95-dm-notify.rules
Reading rules file: /usr/lib/udev/rules.d/95-upower-hid.rules
Reading rules file: /usr/lib/udev/rules.d/95-upower-wup.rules
Reading rules file: /usr/lib/udev/rules.d/98-kexec.rules
Reading rules file: /usr/lib/udev/rules.d/99-qemu-guest-agent.rules
Reading rules file: /usr/lib/udev/rules.d/99-systemd.rules
Reading rules file: /usr/lib/udev/rules.d/99-vmware-scsi-udev.rules
input6: /usr/lib/udev/rules.d/60-input-id.rules:5 Importing properties from results of builtin command 'input_id'
input6: capabilities/ev raw kernel attribute: 3
input6: capabilities/abs raw kernel attribute: 0
input6: capabilities/rel raw kernel attribute: 0
input6: capabilities/key raw kernel attribute: 7fffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffff00000000 0 ffffffffffff ffffffffffffffff ffffffffffffffff fffffffffffffffe
input6: properties raw kernel attribute: 0
input6: test_key: checking bit block 0 for any keys; found=1
input6: test_key: checking bit block 64 for any keys; found=1
input6: test_key: checking bit block 128 for any keys; found=1
input6: test_key: checking bit block 192 for any keys; found=1
input6: /usr/lib/udev/rules.d/60-input-id.rules:6 Importing properties from results of builtin command 'hwdb --subsystem=input --lookup-prefix=id-input:modalias:'
input6: No entry found from hwdb.
input6: /usr/lib/udev/rules.d/60-input-id.rules:6 Failed to run builtin 'hwdb --subsystem=input --lookup-prefix=id-input:modalias:': No data available
input6: /usr/lib/udev/rules.d/71-seat.rules:74 Importing properties from results of builtin command 'path_id'
input6: /usr/lib/udev/rules.d/71-seat.rules:74 Failed to run builtin 'path_id': No such file or directory
input6: sd-device: Created db file '/run/udev/data/+input:input6' for '/devices/virtual/input/input6'
Unload module index
Unloaded link configuration context.
This program is for debugging only, it does not run any program
specified by a RUN key. It may show incorrect results, because
some values may be different, or not available at a simulation run.

DEVPATH=/devices/virtual/input/input6
ACTION=add
SUBSYSTEM=input
TAGS=:seat:
ID_INPUT=1
ID_INPUT_JOYSTICK=1
ID_INPUT_KEY=1
ID_INPUT_KEYBOARD=1
.INPUT_CLASS=joystick
ID_SERIAL=noserial
CURRENT_TAGS=:seat:
USEC_INITIALIZED=3589803915

@yuwata yuwata added bug 🐛 Programming errors, that need preferential fixing and removed needs-reporter-feedback ❓ There's an unanswered question, the reporter needs to answer labels Oct 18, 2022
@yuwata
Copy link
Member

yuwata commented Oct 18, 2022

Sorry for the late response.

When you invoke udevadm test /sys/class/input/input6, the udev database file /run/udev/data/+input:input6 seems to be correctly created with seat tag.

Could you try to call udevadm trigger --settle --action add /sys/class/input/input6 /sys/class/input/input6/event5, then try to call TakeDevice (that is, maybe, restart gnome session)?

If the above works, then next, please create the following .rules file:

# cat /etc/udev/rules.d/00-debug-input.rules
SUBSYSTEM=="input", OPTIONS="log_level=debug"

Then, reboot the system, and please provide the debugging logs of systemd-udevd for input6 (and event5).

@yuwata yuwata added the needs-reporter-feedback ❓ There's an unanswered question, the reporter needs to answer label Oct 18, 2022
@jandryuk
Copy link
Author

Thanks for taking a look @yuwata . It's showing up as input9/event8 now (rawhide kernel 6.0-rc6).

This is strange.
Normal boot up -> keyboard doesn't work.
Boot up -> udevadm test /sys/class/input/input9 -> systemctl restart gdm.service -> keyboard works

Running the udevadm trigger commands and restarting gdm doesn't give a working keyboard. udevadm test is what is important.

Also udevadm trigger is showing some errors for input9:

# udevadm trigger --settle --action add /sys/class/input/input9 /sys/class/input/input9/event8
input9: Failed to write 'add' to '/sys/devices/virtual/input/input9/uevent': Cannot allocate memory

# dmesg | tail -n 2
[  426.650702] synth uevent: /devices/virtual/input/input9: failed to send uevent
[  426.650712] input input9: uevent: failed to send synthetic uevent

# udevadm trigger --settle --action add /sys/class/input/input9
input9: Failed to write 'add' to '/sys/devices/virtual/input/input9/uevent': Cannot allocate memory
# udevadm trigger --settle --action add /sys/class/input/input9/event8

Here is the output from 00-debug-input.rules:

# journalctl -b -u systemd-udevd.service | grep -e event8 -e input9 
Oct 18 15:06:01 fedora systemd-udevd[559]: event8: The log level is changed to 'debug' while processing device (SEQNUM=2917, ACTION=add)
Oct 18 15:06:01 fedora systemd-udevd[559]: event8: /usr/lib/udev/rules.d/50-udev-default.rules:33 GROUP 104
Oct 18 15:06:01 fedora systemd-udevd[559]: event8: /usr/lib/udev/rules.d/60-evdev.rules:8 Importing properties from results of builtin command 'hwdb --subsystem=input --lookup-prefix=evdev:'
Oct 18 15:06:01 fedora systemd-udevd[559]: event8: No entry found from hwdb.
Oct 18 15:06:01 fedora systemd-udevd[559]: event8: /usr/lib/udev/rules.d/60-evdev.rules:8 Failed to run builtin 'hwdb --subsystem=input --lookup-prefix=evdev:': No data available
Oct 18 15:06:01 fedora systemd-udevd[559]: event8: /usr/lib/udev/rules.d/60-evdev.rules:18 Importing properties from results of builtin command 'hwdb 'evdev:name:Xen Virtual Keyboard:phys:xenbus/device/vkbd/0:ev:3:dmi:bvnXen:bvr4.16.2-xc:bd10/07/2022:br4.16:svnXen:pnHVMdomU:pvr4.16.2-xc:cvnXen:ct1:cvr:sku:''
Oct 18 15:06:01 fedora systemd-udevd[559]: event8: No entry found from hwdb.
Oct 18 15:06:01 fedora systemd-udevd[559]: event8: /usr/lib/udev/rules.d/60-evdev.rules:18 Failed to run builtin 'hwdb 'evdev:name:Xen Virtual Keyboard:phys:xenbus/device/vkbd/0:ev:3:dmi:bvnXen:bvr4.16.2-xc:bd10/07/2022:br4.16:svnXen:pnHVMdomU:pvr4.16.2-xc:cvnXen:ct1:cvr:sku:'': No data available
Oct 18 15:06:01 fedora systemd-udevd[559]: event8: /usr/lib/udev/rules.d/60-evdev.rules:23 Importing properties from results of builtin command 'hwdb 'evdev:name:Xen Virtual Keyboard:dmi:bvnXen:bvr4.16.2-xc:bd10/07/2022:br4.16:svnXen:pnHVMdomU:pvr4.16.2-xc:cvnXen:ct1:cvr:sku:''
Oct 18 15:06:01 fedora systemd-udevd[559]: event8: No entry found from hwdb.
Oct 18 15:06:01 fedora systemd-udevd[559]: event8: /usr/lib/udev/rules.d/60-evdev.rules:23 Failed to run builtin 'hwdb 'evdev:name:Xen Virtual Keyboard:dmi:bvnXen:bvr4.16.2-xc:bd10/07/2022:br4.16:svnXen:pnHVMdomU:pvr4.16.2-xc:cvnXen:ct1:cvr:sku:'': No data available
Oct 18 15:06:01 fedora systemd-udevd[559]: event8: /usr/lib/udev/rules.d/60-input-id.rules:5 Importing properties from results of builtin command 'input_id'
Oct 18 15:06:01 fedora systemd-udevd[559]: input9: capabilities/ev raw kernel attribute: 3
Oct 18 15:06:01 fedora systemd-udevd[559]: input9: capabilities/abs raw kernel attribute: 0
Oct 18 15:06:01 fedora systemd-udevd[559]: input9: capabilities/rel raw kernel attribute: 0
Oct 18 15:06:01 fedora systemd-udevd[559]: input9: capabilities/key raw kernel attribute: 7fffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffff00000000 0 ffffffffffff ffffffffffffffff ffffffffffffffff fffffffffffffffe
Oct 18 15:06:01 fedora systemd-udevd[559]: input9: properties raw kernel attribute: 0
Oct 18 15:06:01 fedora systemd-udevd[559]: event8: Input device has 44 joystick buttons and 0 axes but also 10 keyboard key sets, assuming this is a keyboard, not a joystick.
Oct 18 15:06:01 fedora systemd-udevd[559]: event8: test_key: checking bit block 0 for any keys; found=yes
Oct 18 15:06:01 fedora systemd-udevd[559]: event8: /usr/lib/udev/rules.d/60-input-id.rules:6 Importing properties from results of builtin command 'hwdb --subsystem=input --lookup-prefix=id-input:modalias:'
Oct 18 15:06:01 fedora systemd-udevd[559]: event8: No entry found from hwdb.
Oct 18 15:06:01 fedora systemd-udevd[559]: event8: /usr/lib/udev/rules.d/60-input-id.rules:6 Failed to run builtin 'hwdb --subsystem=input --lookup-prefix=id-input:modalias:': No data available
Oct 18 15:06:01 fedora systemd-udevd[559]: event8: /usr/lib/udev/rules.d/80-libinput-device-groups.rules:4 Importing properties from results of 'libinput-device-group /sys/devices/virtual/input/input9/event8'
Oct 18 15:06:01 fedora systemd-udevd[559]: event8: Starting 'libinput-device-group /sys/devices/virtual/input/input9/event8'
Oct 18 15:06:01 fedora systemd-udevd[559]: event8: 'libinput-device-group /sys/devices/virtual/input/input9/event8'(out) 'LIBINPUT_DEVICE_GROUP=0/0/0:xenbus/device/vkbd/0'
Oct 18 15:06:01 fedora systemd-udevd[572]: input9: capabilities/ev raw kernel attribute: 3
Oct 18 15:06:01 fedora systemd-udevd[572]: input9: capabilities/abs raw kernel attribute: 0
Oct 18 15:06:01 fedora systemd-udevd[572]: input9: capabilities/rel raw kernel attribute: 0
Oct 18 15:06:01 fedora systemd-udevd[572]: input9: capabilities/key raw kernel attribute: 7fffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffff00000000 0 ffffffffffff ffffffffffffffff ffffffffffffffff fffffffffffffffe
Oct 18 15:06:01 fedora systemd-udevd[572]: input9: properties raw kernel attribute: 0
Oct 18 15:06:01 fedora systemd-udevd[572]: js0: sd-device: Created db file '/run/udev/data/c13:0' for '/devices/virtual/input/input9/js0'
Oct 18 15:06:01 fedora systemd-udevd[559]: event8: Process 'libinput-device-group /sys/devices/virtual/input/input9/event8' succeeded.
Oct 18 15:06:01 fedora systemd-udevd[559]: event8: Setting permissions /dev/input/event8, uid=0, gid=104, mode=0660
Oct 18 15:06:01 fedora systemd-udevd[559]: event8: Handling device node '/dev/input/event8', devnum=c13:72
Oct 18 15:06:01 fedora systemd-udevd[559]: event8: sd-device: Created db file '/run/udev/data/c13:72' for '/devices/virtual/input/input9/event8'
Oct 18 15:06:01 fedora systemd-udevd[559]: event8: Failed to get watch handle, ignoring: No such file or directory
Oct 18 15:06:01 fedora systemd-udevd[559]: event8: Device processed (SEQNUM=2917, ACTION=add)
Oct 18 15:06:01 fedora systemd-udevd[559]: event8: sd-device-monitor: Passed 397 byte to netlink monitor

@yuwata
Copy link
Member

yuwata commented Oct 18, 2022

Several lines for input9 seems to be forgotten, e.g. Device processed is shown only for event8, not for input9...

Ah, so, no event for input9 is triggered. Hmm?

@yuwata
Copy link
Member

yuwata commented Oct 18, 2022

# udevadm trigger --settle --action add /sys/class/input/input9
input9: Failed to write 'add' to '/sys/devices/virtual/input/input9/uevent': Cannot allocate memory

So, let me confirm, systemd-udev-trigger.service logs the similar message(s), right?

@jandryuk
Copy link
Author

Correct - there is no Device processed for input9.

systemd-udev-trigger doesn't have anything interesting:

# journalctl -b -u systemd-udev-trigger | cat
Oct 18 16:26:47 fedora systemd[1]: Starting systemd-udev-trigger.service - Coldplug All udev Devices...
Oct 18 16:26:48 fedora systemd[1]: Finished systemd-udev-trigger.service - Coldplug All udev Devices.
Oct 18 16:26:51 fedora systemd[1]: systemd-udev-trigger.service: Deactivated successfully.
Oct 18 16:26:51 fedora systemd[1]: Stopped systemd-udev-trigger.service - Coldplug All udev Devices.
Oct 18 16:26:56 fedora systemd[1]: Finished systemd-udev-trigger.service - Coldplug All udev Devices.

On a fresh boot, there are no dmesg entries about failing to send the uevent.

@yuwata
Copy link
Member

yuwata commented Oct 18, 2022

# dmesg | tail -n 2
[  426.650702] synth uevent: /devices/virtual/input/input9: failed to send uevent
[  426.650712] input input9: uevent: failed to send synthetic uevent

This sounds a kernel bug or limitation of Xen (or misconfigurarion in your Xen setup?) for me.

The uevent file for input9 (/sys/devices/virtual/input/input9/uevent) is readable? If so, could you provide the contents and access mode (cat and ls -l)?

@yuwata
Copy link
Member

yuwata commented Oct 18, 2022

BTW, a dirty workaround for the issue seems creating the following drop-in file with systemctl edit gdm.service:

# /etc/systemd/system/gdm.service.d/override.conf
[Service]
ExecStartPre=-udevadm test /sys/class/input/input9

@jandryuk
Copy link
Author

It's empty:

[root@fedora ~]# ls -l /sys/devices/virtual/input/input9/uevent 
-rw-r--r--. 1 root root 4096 Oct 18 16:27 /sys/devices/virtual/input/input9/uevent
[root@fedora ~]# cat /sys/devices/virtual/input/input9/uevent 
[root@fedora ~]# cat /sys/devices/virtual/input/input9/event8/uevent 
MAJOR=13
MINOR=72
DEVNAME=input/event8

Here are some of the other Xen virtual devices. udevadm trigger runs for them without error.

[root@fedora ~]# cat /sys/devices/virtual/input/input10/uevent 
PRODUCT=1/5853/fffe/0
NAME="Xen Virtual Pointer"
PHYS="xenbus/device/vkbd/0"
PROP=0
EV=f
KEY=ff0000 0 0 0 0
REL=100
ABS=3
MODALIAS=input:b0001v5853pFFFEe0000-e0,1,2,3,k110,111,112,113,114,115,116,117,r8,a0,1,mlsfw
[root@fedora ~]# cat /sys/devices/virtual/input/input10/event9/uevent 
MAJOR=13
MINOR=73
DEVNAME=input/event9
[root@fedora ~]# 
[root@fedora ~]# 
[root@fedora ~]# cat /sys/devices/virtual/input/input11/uevent 
PRODUCT=1/5853/fffd/0
NAME="Xen Virtual Multi-touch"
PHYS="xenbus/device/vkbd/0"
PROP=2
EV=b
KEY=400 0 0 0 0 0
ABS=261800000000003
MODALIAS=input:b0001v5853pFFFDe0000-e0,1,3,k14A,ra0,1,2F,30,35,36,39,mlsfw
[root@fedora ~]# cat /sys/devices/virtual/input/input11/event10/uevent 
MAJOR=13
MINOR=74
DEVNAME=input/event10

The "Cannot allocate memory" made me remember this: https://lore.kernel.org/xen-devel/87o8dw52jc.fsf@vps.thesusis.net/T/ . It's a bug report with ENOMEM and uevents. May be unrelated, but ENOMEM and uevent seem similar.

@yuwata
Copy link
Member

yuwata commented Oct 18, 2022

The "Cannot allocate memory" made me remember this: https://lore.kernel.org/xen-devel/87o8dw52jc.fsf@vps.thesusis.net/T/ . It's a bug report with ENOMEM and uevents. May be unrelated, but ENOMEM and uevent seem similar.

Seems this is exactly the same issue. The kernel log is about the failure of kobject_uevent_env() called by kobject_synth_uevent().

Could you show the result of udevadm info --attribute-walk /sys/devices/virtual/input/input9 ?

@jandryuk
Copy link
Author

# udevadm info --attribute-walk /sys/devices/virtual/input/input9

Udevadm info starts with the device specified by the devpath and then
walks up the chain of parent devices. It prints for every device
found, all possible attributes in the udev rules key format.
A rule to match, can be composed by the attributes of the device
and the attributes from one single parent device.

  looking at device '/devices/virtual/input/input9':
    KERNEL=="input9"
    SUBSYSTEM=="input"
    DRIVER==""
    ATTR{capabilities/abs}=="0"
    ATTR{capabilities/ev}=="3"
    ATTR{capabilities/ff}=="0"
    ATTR{capabilities/key}=="7fffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffff00000000 0 ffffffffffff ffffffffffffffff ffffffffffffffff fffffffffffffffe"
    ATTR{capabilities/led}=="0"
    ATTR{capabilities/msc}=="0"
    ATTR{capabilities/rel}=="0"
    ATTR{capabilities/snd}=="0"
    ATTR{capabilities/sw}=="0"
    ATTR{id/bustype}=="0001"
    ATTR{id/product}=="ffff"
    ATTR{id/vendor}=="5853"
    ATTR{id/version}=="0000"
    ATTR{inhibited}=="0"
    ATTR{name}=="Xen Virtual Keyboard"
    ATTR{phys}=="xenbus/device/vkbd/0"
    ATTR{power/async}=="disabled"
    ATTR{power/control}=="auto"
    ATTR{power/runtime_active_kids}=="0"
    ATTR{power/runtime_active_time}=="0"
    ATTR{power/runtime_enabled}=="disabled"
    ATTR{power/runtime_status}=="unsupported"
    ATTR{power/runtime_suspended_time}=="0"
    ATTR{power/runtime_usage}=="0"
    ATTR{properties}=="0"
    ATTR{uniq}==""
# cat /sys/class/input/input9/modalias 
input:b0001v5853pFFFFe0000-e0,1,k71,72,73,74,75,76,77,78,79,7A,7B,7C,7D,7E,7F,80,81,82,83,84,85,86,87,88,89,8A,8B,8C,8D,8E,8F,90,91,92,93,94,95,96,97,98,99,9A,9B,9C,9D,9E,9F,A0,A1,A2,A3,A4,A5,A6,A7,A8,A9,AA,AB,AC,AD,AE,AF,B0,B1,B2,B3,B4,B5,B6,B7,B8,B9,BA,BB,BC,BD,BE,BF,C0,C1,C2,C3,C4,C5,C6,C7,C8,C9,CA,CB,CC,CD,CE,CF,D0,D1,D2,D3,D4,D5,D6,D7,D8,D9,DA,DB,DC,DD,DE,DF,E0,E1,E2,E3,E4,E5,E6,E7,E8,E9,EA,EB,EC,ED,EE,EF,160,161,162,163,164,165,166,167,168,169,16A,16B,16C,16D,16E,16F,170,171,172,173,174,175,176,177,178,179,17A,17B,17C,17D,17E,17F,180,181,182,183,184,185,186,187,188,189,18A,18B,18C,18D,18E,18F,190,191,192,193,194,195,196,197,198,199,19A,19B,19C,19D,19E,19F,1A0,1A1,1A2,1A3,1A4,1A5,1A6,1A7,1A8,1A9,1AA,1AB,1AC,1AD,1AE,1AF,1B0,1B1,1B2,1B3,1B4,1B5,1B6,1B7,1B8,1B9,1BA,1BB,1BC,1BD,1BE,1BF,1C0,1C1,1C2,1C3,1C4,1C5,1C6,1C7,1C8,1C9,1CA,1CB,1CC,1CD,1CE,1CF,1D0,1D1,1D2,1D3,1D4,1D5,1D6,1D7,1D8,1D9,1DA,1DB,1DC,1DD,1DE,1DF,1E0,1E1,1E2,1E3,1E4,1E5,1E6,1E7,1E8,1E9,1EA,1EB,1EC,1ED,1EE,1EF,1F0,1F1,1F2,1F3,1F4,1F5,1F6,1F7,1F8,1F9,1FA,1FB,1FC,1FD,1FE,1FF,200,201,202,203,204,205,206,207,208,209,20A,20B,20C,20D,20E,20F,210,211,212,213,214,215,216,217,218,219,21A,21B,21C,21D,21E,21F,220,221,222,223,224,225,226,227,228,229,22A,22B,22C,22D,22E,22F,230,231,232,233,234,235,236,237,238,239,23A,23B,23C,23D,23E,23F,240,241,242,243,244,245,246,247,248,249,24A,24B,24C,24D,24E,24F,250,251,252,253,254,255,256,257,258,259,25A,25B,25C,25D,25E,25F,260,261,262,263,264,265,266,267,268,269,26A,26B,26C,26D,26E,26F,270,271,272,273,274,275,276,277,278,279,27A,27B,27C,27D,27E,27F,280,281,282,283,284,285,286,287,288,289,28A,28B,28C,28D,28E,28F,290,291,292,293,294,295,296,297,298,299,29A,29B,29C,29D,29E,29F,2A0,2A1,2A2,2A3,2A4,2A5,2A6,2A7,2A8,2A9,2AA,2AB,2AC,2AD,2AE,2AF,2B0,2B1,2B2,2B3,2B4,2B5,2B6,2B7,2B8,2B9,2BA,2BB,2BC,2BD,2BE,2BF,2C0,2C1,2C2,2C3,2C4,2C5,2C6,2C7,2C8,2C9,2CA,2CB,2CC,2CD,2CE,2CF,2D0,2D1,2D2,2D3,2D4,2D5,2D6,2D7,2D8,2D9,2DA,2DB,2DC,2DD,2DE,2DF,2E0,2E1,2E2,2E3,2E4,2E5,2E6,2E7,2E8,2E9,2EA,2EB,2EC,2ED,2EE,2EF,2F0,2F1,2F2,2F3,2F4,2F5,2F6,2F7,2F8,2F9,2FA,2FB,2FC,2FD,2FE,ramlsfw
# cat /sys/class/input/input9/modalias | wc -c
2082

Also, the gdm override seems make the Xen keyboard work. I am confused why manually running udevadm test /sys/class/input/input9 helps.

@jandryuk
Copy link
Author

I'll pursue patching the xen_kbdfront driver to cut down on the list of keys. Thanks for your help, @yuwata

intel-lab-lkp pushed a commit to intel-lab-lkp/linux that referenced this issue Oct 20, 2022
xen kbdfront registers itself as being able to deliver *any* key since
it doesn't know what keys the backend may produce.

Unfortunately, the generated modalias gets too large and uevent creation
fails with -ENOMEM.

This can lead to gdm not using the keyboard since there is no seat
associated [1] and the debian installer crashing [2].

Trim the ranges of key capabilities by removing some BTN_* ranges.
While doing this, some neighboring undefined ranges are removed to trim
it further.

This removes:
BTN_DPAD_UP(0x220)..BTN_DPAD_RIGHT(0x223)
Empty space 0x224..0x229

Emtpy space 0x2bd..0x2bf
BTN_TRIGGER_HAPPY(0x2c0)..BTN_TRIGGER_HAPPY40(0x2e7)
Empty space 0x2e8..0x2ff

The modalias shrinks from 2082 to 1754 bytes.

[1] systemd/systemd#22944
[2] https://lore.kernel.org/xen-devel/87o8dw52jc.fsf@vps.thesusis.net/T/

Cc: Phillip Susi <phill@thesusis.net>
Cc: stable@vger.kernel.org
Signed-off-by: Jason Andryuk <jandryuk@gmail.com>
@jandryuk
Copy link
Author

@yuwata yuwata added not-our-bug and removed bug 🐛 Programming errors, that need preferential fixing needs-reporter-feedback ❓ There's an unanswered question, the reporter needs to answer labels Oct 24, 2022
@yuwata
Copy link
Member

yuwata commented Oct 24, 2022

@jandryuk Thank you for your work! Please let us know when your (or another) patch for the issue is merged to the kernel upstream.

intel-lab-lkp pushed a commit to intel-lab-lkp/linux that referenced this issue Dec 9, 2022
xen kbdfront registers itself as being able to deliver *any* key since
it doesn't know what keys the backend may produce.

Unfortunately, the generated modalias gets too large and uevent creation
fails with -ENOMEM.

This can lead to gdm not using the keyboard since there is no seat
associated [1] and the debian installer crashing [2].

Trim the ranges of key capabilities by removing some BTN_* ranges.
While doing this, some neighboring undefined ranges are removed to trim
it further.

An upper limit of KEY_KBD_LCD_MENU5 is still too large.  Use an upper
limit of KEY_BRIGHTNESS_MENU.

This removes:
BTN_DPAD_UP(0x220)..BTN_DPAD_RIGHT(0x223)
Empty space 0x224..0x229

Empty space 0x28a..0x28f
KEY_MACRO1(0x290)..KEY_MACRO30(0x2ad)
KEY_MACRO_RECORD_START          0x2b0
KEY_MACRO_RECORD_STOP           0x2b1
KEY_MACRO_PRESET_CYCLE          0x2b2
KEY_MACRO_PRESET1(0x2b3)..KEY_MACRO_PRESET3(0xb5)
Empty space 0x2b6..0x2b7
KEY_KBD_LCD_MENU1(0x2b8)..KEY_KBD_LCD_MENU5(0x2bc)
Empty space 0x2bd..0x2bf
BTN_TRIGGER_HAPPY(0x2c0)..BTN_TRIGGER_HAPPY40(0x2e7)
Empty space 0x2e8..0x2ff

The modalias shrinks from 2082 to 1550 bytes.

A chunk of keys need to be removed to allow the keyboard to be used.
This may break some functionality, but the hope is these macro keys are
uncommon and don't affect any users.

[1] systemd/systemd#22944
[2] https://lore.kernel.org/xen-devel/87o8dw52jc.fsf@vps.thesusis.net/T/

Cc: Phillip Susi <phill@thesusis.net>
Cc: stable@vger.kernel.org
Signed-off-by: Jason Andryuk <jandryuk@gmail.com>
intel-lab-lkp pushed a commit to intel-lab-lkp/linux that referenced this issue Oct 11, 2023
xen kbdfront registers itself as being able to deliver *any* key since
it doesn't know what keys the backend may produce.

Unfortunately, the generated modalias gets too large and uevent creation
fails with -ENOMEM.

This can lead to gdm not using the keyboard since there is no seat
associated [1] and the debian installer crashing [2].

Trim the ranges of key capabilities by removing some BTN_* ranges.
While doing this, some neighboring undefined ranges are removed to trim
it further.

An upper limit of KEY_KBD_LCD_MENU5 is still too large.  Use an upper
limit of KEY_BRIGHTNESS_MENU.

This removes:
BTN_DPAD_UP(0x220)..BTN_DPAD_RIGHT(0x223)
Empty space 0x224..0x229

Empty space 0x28a..0x28f
KEY_MACRO1(0x290)..KEY_MACRO30(0x2ad)
KEY_MACRO_RECORD_START          0x2b0
KEY_MACRO_RECORD_STOP           0x2b1
KEY_MACRO_PRESET_CYCLE          0x2b2
KEY_MACRO_PRESET1(0x2b3)..KEY_MACRO_PRESET3(0xb5)
Empty space 0x2b6..0x2b7
KEY_KBD_LCD_MENU1(0x2b8)..KEY_KBD_LCD_MENU5(0x2bc)
Empty space 0x2bd..0x2bf
BTN_TRIGGER_HAPPY(0x2c0)..BTN_TRIGGER_HAPPY40(0x2e7)
Empty space 0x2e8..0x2ff

The modalias shrinks from 2082 to 1550 bytes.

A chunk of keys need to be removed to allow the keyboard to be used.
This may break some functionality, but the hope is these macro keys are
uncommon and don't affect any users.

[1] systemd/systemd#22944
[2] https://lore.kernel.org/xen-devel/87o8dw52jc.fsf@vps.thesusis.net/T/

Cc: Phillip Susi <phill@thesusis.net>
Cc: stable@vger.kernel.org
Signed-off-by: Jason Andryuk <jandryuk@gmail.com>
Reviewed-by: Mattijs Korpershoek <mkorpershoek@baylibre.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

No branches or pull requests

3 participants