fuse auto_unmount

[mharvey-jt]

As an alternative to the watchdog process, we could use libfuse3's auto_unmount option.
Initial testing shows it's not working cleanly:

Without the option:

[root@host ~]# /usr/bin/mount -t cvmfs -o libfuse=3,disable_watchdog my.cvmfs.repo /opt/cvmfs/my.cvmfs.repo
CernVM-FS: running with credentials 497:497
CernVM-FS: loading Fuse module... done
CernVM-FS: mounted cvmfs on /opt/cvmfs/my.cvmfs.repo
[root@host ~]# 

giving processes:

[root@host ~]# ps -ef |grep cvmfs2
cvmfs     44479      1  0 10:08 ?        00:00:00 /usr/bin/cvmfs2 __cachemgr__ . 8 9 5085593600 2542796800 0 3 -1 :
cvmfs     44483      1  0 10:08 ?        00:00:00 /usr/bin/cvmfs2 __cachemgr__ . 8 9 5085593600 2542796800 0 3 -1 :
cvmfs     44490      1  0 10:08 ?        00:00:00 /usr/bin/cvmfs2 -o rw,libfuse=3,disable_watchdog,system_mount,fsname=cvmfs2,allow_other,grab_mountpoint,uid=497,gid=497 modules.firm.jump.gcp /opt/cvmfs/my.cvmfs.repo

But with the option:

[root@host~]# /usr/bin/mount -t cvmfs -o libfuse=3,disable_watchdog,auto_unmount my.cvmfs.repo /opt/cvmfs/my.cvmfs.repo
CernVM-FS: running with credentials 497:497
CernVM-FS: loading Fuse module... done
CernVM-FS: mounted cvmfs on /opt/cvmfs/my.cvmfs.repo
<DOES NOT RETURN>

with processes

 ps -ef |grep cvmfs
root      44932  30660  0 10:10 pts/43   00:00:00 /usr/bin/mount -t cvmfs -o libfuse=3,disable_watchdog,auto_unmount my.cvmfs.repo /opt/cvmfs/my.cvmfs.repo
root      44933  44932  0 10:10 pts/43   00:00:00 /sbin/mount.cvmfs my.cvmfs.repo /opt/cvmfs/my.cvmfs.repo -o rw,libfuse=3,disable_watchdog,auto_unmount
cvmfs     44951  44933  0 10:10 pts/43   00:00:00 [cvmfs2] <defunct>
cvmfs     44977      1  0 10:10 ?        00:00:00 /usr/bin/cvmfs2 __cachemgr__ . 8 9 5085593600 2542796800 0 3 -1 :
cvmfs     44981      1  0 10:10 ?        00:00:00 /usr/bin/cvmfs2 __cachemgr__ . 8 9 5085593600 2542796800 0 3 -1 :
root      44983      1  0 10:10 ?        00:00:00 fusermount3 -o ro,nosuid,nodev,allow_other,default_permissions,auto_unmount,fsname=cvmfs2 -- /opt/cvmfs/my.cvmfs.repo
cvmfs     44986      1  0 10:10 ?        00:00:00 /usr/bin/cvmfs2 -o rw,libfuse=3,disable_watchdog,auto_unmount,system_mount,fsname=cvmfs2,allow_other,grab_mountpoint,uid=497,gid=497 my.cvmfs.repo /opt/cvmfs/my.cvmfs.repo

The fusermount3 process appears to be expected -- that's what libfuse keeps around to do the umount() if the cvmfs2 mount process crashes. What appears to be going wrong is that mount.cvmfs isn't exiting, for some reason.
gstacking it gives:

# gstack 44933
#0  0x00007ffff6fb64bb in select () from /lib64/libc.so.6
#1  0x0000000000406a07 in main (argc=<optimized out>, argv=<optimized out>) at /home/mharvey/cvmfs/mount/mount.cvmfs.cc:538

corresponding to

cvmfs/mount/mount.cvmfs.cc

Line 641 in 491857e

retval = select(nfds, &readfds, NULL, NULL, NULL);

sending the mount.cvmfs process a SIGINT gets things moving.

[jblomer]

There are probably more problems related to fuse's auto_umount: libfuse/libfuse#586

It's a pity. I guess we should then fix the watchdog process and give it a mode where it doesn't get involved in producing a stack trace but still takes care of unmounting.

[mharvey-jt]

I think the fix is just to ensure that the stdout/stderr draining loop in mount.cvmfs h as a timeout on the select() and includes a waitpid( pid_cvmfs, .., WNOHANG) in the termination check

That libfuse issue doesn't seem to a problem when the automounter is un use - the fusermount3 watch process runs as root.

[mharvey-jt]

Fixed with https://github.com/cvmfs/cvmfs/pull/3144/files