mkswap01 is racing with udev and sometimes fails

[jstancek]

mkswap01 is sporadically failing on aarch64 systems (kernel 4.18):

# ./testcases/bin/mkswap01.sh
tst_device.c:83: INFO: Found free device '/dev/loop2'
mkswap01 1 TPASS: 'mkswap   /dev/loop2 ' passed.
mkswap01 2 TPASS: 'mkswap   /dev/loop2 262080' passed.
mkswap01 3 TINFO: Can not do swapon on /dev/loop2.
mkswap01 3 TINFO: Device size specified by 'mkswap' greater than real size.
mkswap01 3 TINFO: Swapon failed expectedly.
mkswap01 3 TPASS: 'mkswap -f  /dev/loop2 262208' passed.
mkswap01 4 TPASS: 'mkswap -c  /dev/loop2 ' passed.
mkswap01 5 TINFO: Can not do swapon on /dev/loop2.
mkswap01 5 TINFO: Page size specified by 'mkswap -p' is not equal to system's page size.
mkswap01 5 TINFO: Swapon failed expectedly.
mkswap01 5 TPASS: 'mkswap -p 2048 /dev/loop2 ' passed.
mkswap01 6 TPASS: 'mkswap -L ltp_testswap /dev/loop2 ' passed.
mkswap01 7 TPASS: 'mkswap -v1  /dev/loop2 ' passed.
mkswap01 8 TBROK: "check_for_file /dev/disk/by-uuid/2ca3a3eb-2edb-458f-9549-b0f11ea29252" timed out

Summary:
passed   7
failed   0
skipped  0
warnings 0

udev monitors /dev/loop0 with inotify and reacts to mkswap closing fd of loop device. This triggers some rules and while these run, watch is removed, then re-added back afterwards. So it appears that last "mkswap -U" in test comes in while inotify watch is disabled, and that causes udev to miss change of uuid.

If I add extra sleep to udev rules (on aarch64):

--- /usr/lib/udev/rules.d/60-persistent-storage.rules.orig    2018-12-22 07:21:19.647844560 -0500
+++ /usr/lib/udev/rules.d/60-persistent-storage.rules   2018-12-22 07:16:49.458693213 -0500
@@ -88,6 +88,7 @@
 
 # probe filesystem metadata of disks
 KERNEL!="sr*", IMPORT{builtin}="blkid"
+KERNEL!="sr*", RUN+="/bin/sleep 1"
 
 # by-label/by-uuid links (filesystem metadata)
 ENV{ID_FS_USAGE}=="filesystem|other|crypto", ENV{ID_FS_UUID_ENC}=="?*", SYMLINK+="disk/by-uuid/$env{ID_FS_UUID_ENC}"

Then following 2 mkswap commands end up in same situation with stale link (on aarch64):

# dd if=/dev/zero of=/tmp/img bs=65536 count=4096
# losetup --show -f /tmp/img

# mkswap /dev/loop0 ; sleep 1.5; mkswap /dev/loop0 ; sleep 10; ls -la /dev/disk/by-uuid/
mkswap: /dev/loop0: warning: wiping old swap signature.
Setting up swapspace version 1, size = 256 MiB (268369920 bytes)
no label, UUID=3e2394a4-71b8-4b05-9a39-84ce66fca6d9
mkswap: /dev/loop0: warning: wiping old swap signature.
Setting up swapspace version 1, size = 256 MiB (268369920 bytes)
no label, UUID=6687c735-92a6-45d4-9d9f-9c1e54a4beb3
total 0
drwxr-xr-x. 2 root root 160 Dec 22 07:20 .
drwxr-xr-x. 7 root root 140 Dec 21 17:15 ..
lrwxrwxrwx. 1 root root  10 Dec 21 17:54 1e452ec9-17d0-45db-abf1-d8a5a79ab6cd -> ../../dm-0
lrwxrwxrwx. 1 root root  11 Dec 22 07:20 3e2394a4-71b8-4b05-9a39-84ce66fca6d9 -> ../../loop0
lrwxrwxrwx. 1 root root  10 Dec 21 17:54 5b921c8a-9290-4036-9cd7-e710336bfce4 -> ../../dm-1
lrwxrwxrwx. 1 root root  10 Dec 22 06:28 6c5ae226-bc8c-4693-bc35-0a78e455f177 -> ../../dm-2
lrwxrwxrwx. 1 root root  10 Dec 21 17:54 C6A3-25A3 -> ../../sda1
lrwxrwxrwx. 1 root root  10 Dec 21 17:54 d37592a0-2d05-4be2-b35e-54f6483af071 -> ../../sda2

The timing is not universal across arches. For example on x86_64 I can reproduce it with "sleep 2" in udev rules and "sleep 1" between mkswap commands.

It is also easier to reproduce with LTP test when TMPDIR is backed by ramdisk - to make mkswap complete quickly.

My impression is that watch is dropped by design when udev rules run, so we likely need to add some sleeps to the test, so that udev is able to catch up.
[CC @wangli5665]

[ngie-eign]

Should the inode number be checked between calls to make sure that the loopback device has changed?

[liux2085]

Adding "udevadm trigger" before mkswap_verify should fix this issue.

[wangli5665]

Adding "udevadm trigger" before mkswap_verify should fix this issue.

Sounds reasonable, would you mind to work out a patch to LTP ML?

[wangli5665]

This has been solved via http://lists.linux.it/pipermail/ltp/2019-November/014366.html