losetup -d exits 0 when it doesn't succeed

[HaleTom]

# losetup --version
losetup from util-linux 2.29.2

Steps to reproduce issue:

Create a ext4 filesystem called filesystem:

# cd /tmp
# truncate -s500M filesystem
# mkfs.ext4 filesystem 
mke2fs 1.43.4 (31-Jan-2017)
Discarding device blocks: done                            
Creating filesystem with 512000 1k blocks and 128016 inodes
Filesystem UUID: 56893af2-d159-4c5e-a6a6-09b83da65004
Superblock backups stored on blocks: 
        8193, 24577, 40961, 57345, 73729, 204801, 221185, 401409

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done 

# losetup

Nil output shows that there are currently no loop devices.

Mount filesystem on /mnt/tmp and show loop device:

# mount -oloop filesystem /mnt/tmp
# losetup
NAME       SIZELIMIT OFFSET AUTOCLEAR RO BACK-FILE       DIO
/dev/loop0         0      0         1  0 /tmp/filesystem   0

Have a process use the mountpoint /mnt/tmp as its working directory:

# (cd /mnt/tmp; sleep 9999)&
[1] 2215

Show processes using /mnt/tmp. Note the PID agrees with the one above.

# sudo lsof +f -- /mnt/tmp
COMMAND  PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
bash    2215 root  cwd    DIR    7,0     1024    2 /mnt/tmp
sleep   2216 root  cwd    DIR    7,0     1024    2 /mnt/tmp

Lazy unmount /mnt/tmp, detaching loop device:

# umount -ld /mnt/tmp
# umount -ld /mnt/tmp
umount: /mnt/tmp: not mounted

Try to explicitly remove the loop device:

# losetup -d /dev/loop0
# echo $?
0

But it's still there:

# losetup
NAME       SIZELIMIT OFFSET AUTOCLEAR RO BACK-FILE       DIO
/dev/loop0         0      0         1  0 /tmp/filesystem   0

and I can't find out what processes are using it as a working directory:

# sudo lsof +f -- /mnt/tmp
lsof: not a file system: /mnt/tmp
lsof 4.89

So, let's ask explicitly for the PIDs given above:

# lsof | grep '221[56] .* cwd'
lsof: WARNING: can't stat() fuse.gvfsd-fuse file system /run/user/1000/gvfs
    Output information may be incomplete.
bash       2215          root  cwd       DIR                7,0      1024          2 /
sleep      2216          root  cwd       DIR                7,0      1024          2 /

Hmm, so the working directory has changed from /mnt/tmp to /... then why is /dev/loop0 not unmounting?

Kill the processes which were using /mnt/tmp as working directory:

# kill 2215 2216
[1]+  Terminated              ( cd /mnt/tmp; sleep 9999 )

And now, finally, /dev/loop0 disappears:

# losetup
# 

[HaleTom]

Come to think of it, umount -ld should have printed an error as it was explicitly told to take down the loop device and was also unable to comply. Should I track this as a separate issue?

[karelzak]

The question is if we can do anything in userspace. It would be nice to have strace output from "losetup -d" to see what kernel returns. I guess for losetup it seems that the ioctl has been successful.

[karelzak]

stat("/dev/loop0", {st_mode=S_IFBLK|0660, st_rdev=makedev(7, 0), ...}) = 0
open("/dev/loop0", O_RDONLY|O_CLOEXEC)  = 3
ioctl(3, LOOP_CLR_FD)                   = 0
close(3)                                = 0

... so for losetup, all works as expected.

[HaleTom]

It seems like this is a kernel issue then. Any tips on where to raise it?

[karelzak]

You can ask kernel developers, but I don't think it's a bug.

The command "umount --lazy" removed FS from VFS tree, but the FS is internally still referenced and real clean up of all FS resources is postponed (until no process uses FS). This also affects loop device autoclear flag, the device has to stay active as it's used by the FS.

The LOOP_CLR_FD kernel code does nothing if there is any active device reader, the device is marked as "autoclear" and kernel returns 0. It means syscall is successful, but device will be removed later.

I cannot imagine another solution. I'll add note about this LOOP_CLR_FD behaviour to losetup man page.

[karelzak]

See http://permalink.gmane.org/gmane.linux.kernel.commits.head/348218 for more details.