Successful fstab mount appears to be random at boot on Ubuntu 20.04 LTS #746
Comments
|
I want to add some extra info. Upon a failed mount at boot, I ran the following command to check boot logs - it says it mounted just fine, but obviously hasn't. |
|
2.29.0-17-g831dba3 Why are you using master? You should be using the latest stable release. If it crashes at boot it should crash under normal mount. Have you tried mounting and unmounting in a loop to see if you can reproduce it? It'd going to be very difficult to replicate and debug at boot. |
|
I've downgraded back to the most recent stable release Rebooted and it failed to mount at boot. I then manually unmounted and remounted about a dozen times with no issues. Im stumped. |
|
It could be the kernel. There are/were issues with FUSE on 5.4.X unfortunately. Though odd it would be only at boot. Maybe there is some async thing going on that causes problems. |
|
Unfortunately, there isn't a way to put ordering into fstab. You'd have to put it as a systemd mount. |
|
I thought it could be a boot order issue as well but during the times it would successfully mount at boot, it would appear order didn't matter. Here are the boot logs for a successful mount: |
|
Apr 26 01:54:21 mspiggySERVER systemd[1]: Mounted FUSE Control File System. When it does crash... can you confirm that the fuse control filesystem happens before mergerfs? |
|
I can confirm that is the case as I have checked repeatedly on both successful and unsuccessful mounts. I use the following output to confirm (correct me if I'm wrong). It appears as below on success/non-success and occurs before mounting of any drives. |
|
I was talking about the ordering as showing in your previous post. I imagine it does but just wanted to confirm. |
|
Yes it comes before any drives are mounted. |
|
Just wanted to add some feedback here. I have reinstalled a fresh copy of Ubuntu 20.04 on bare metal with a new SSD and setup mergerFS stable release again but continue to run into the random fail to mount at boot via fstab, can only conclude that its a kernel problem. |
|
OK. I'll try setting up a VM to try and reproduce it. Have you tried older releases? 2.28.3? Your home directory isn't encrypted is it? Have you tried a non-home directory location for your mounts? |
|
I haven’t tried an older release, I can give that a whirl tomorrow. Home directory is not encrypted. I haven’t tried placing the mount outside the home directory, however I did have this identical folder setup on Ubuntu 18.04 without issue. |
|
Any updates? |
General description
Migrated my mergerFS setup from a previous install of Ubuntu 18.04 LTS to a fresh install of Ubuntu 20.04 LTS. I use /etc/fstab to mount at boot but it appears that it is unsuccessful sometimes and successful at other times (appears to be random). I will have to reboot a few times to finally get it to mount.
In one instance of an unsuccessful mount at boot, Ubuntu notified me of a crash and I have attached the crash report here. In all subsequent unsuccessful mount at boot events, I have not been notified of a crash since.
https://www.dropbox.com/s/2d83rw46jb02or3/_usr_bin_mergerfs.0.crash.txt?dl=0
Upon failture to mount at boot, if I attempt to manually mount via this command:
I get the following error:
A quick listing of the mount folder shows "?" for permissions, owner and group:
I have to manually unmount before succesfully remounting via mergerFS:
Expected behavior
Automatically mount at boot 100% of the time.
Actual behavior
It will successfully mount sometimes on boot and unsuccessfully on others (appears to be random).
Precise steps to reproduce the behavior
System information
Please provide as much of the following information as possible:
mergerfs version:
mergerfs -Vmergerfs version: 2.29.0-17-g831dba3
FUSE library version: 2.9.7-mergerfs_2.29.0
fusermount version: 2.9.7-mergerfs_2.29.0
using FUSE kernel interface version 7.31
mergerfs settings:
cat /etc/fstabor the command line arguments/dev/mapper/vgubuntu-root / ext4 errors=remount-ro 0 1
UUID=1DA1-7A4B /boot/efi vfat umask=0077 0 1
/dev/mapper/ubuntu--vg-swap_1 none swap sw 0 0
/dev/mapper/vgubuntu-swap_1 none swap sw 0 0
UUID=a0840e4a-2c92-4ac8-892d-2d7b9044e900 /home/eckoflyte/PARITY01 ext4 defaults 0 0
UUID=c41a45a5-59a0-41b2-85a4-16c8a57ab348 /home/eckoflyte/PARITY02 ext4 defaults 0 0
UUID=0ab0197e-e77a-4ed1-ab8b-293a7e880025 /home/eckoflyte/mergerFS01 ext4 defaults 0 0
UUID=173f2486-ffbd-4622-b79b-74c7dbce2771 /home/eckoflyte/mergerFS02 ext4 defaults 0 0
UUID=008fed4f-a471-4531-9bbf-5a83445bdb80 /home/eckoflyte/mergerFS03 ext4 defaults 0 0
UUID=ccc707b6-11e1-4582-9ed3-ef651c2f0af0 /home/eckoflyte/mergerFS04 ext4 defaults 0 0
UUID=5c8bdfa7-5125-455d-9510-a612308581c3 /home/eckoflyte/mergerFS05 ext4 defaults 0 0
UUID=1d97cee4-fb44-4432-b06a-a991c4f218db /home/eckoflyte/mergerFS06 ext4 defaults 0 0
UUID=00502249-ce2f-4555-b35a-d5a1c7c08719 /home/eckoflyte/mergerFS07 ext4 defaults 0 0
UUID=f88cbed6-43c8-4430-b30a-29abdf095cf2 /home/eckoflyte/mergerFS08 ext4 defaults 0 0
UUID=842a7b25-4519-4ab3-b3a2-a1ffec3e8580 /home/eckoflyte/mergerFS09 ext4 defaults 0 0
UUID=731a2464-b931-41bd-b32f-92c73bc4f361 /home/eckoflyte/mergerFS10 ext4 defaults 0 0
UUID=c9c299df-f968-4a97-a47c-b6e5f64cb786 /home/eckoflyte/mergerFS11 ext4 defaults 0 0
/home/eckoflyte/mergerFS* /home/eckoflyte/drivepool mergerfs use_ino,cache.files=off,dropcacheonclose=true,allow_other,ignorepponrename=true,func.getattr=newest,category.create=mfs,fsname=mergerFS 0 0
Linux version:
uname -aLinux mspiggySERVER 5.4.0-26-generic support RHEL6 #30-Ubuntu SMP Mon Apr 20 16:58:30 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Versions of any additional software being used
na
List of drives, filesystems, & sizes:
df -hFilesystem Size Used Avail Use% Mounted on
udev 32G 0 32G 0% /dev
tmpfs 6.3G 5.9M 6.3G 1% /run
/dev/mapper/vgubuntu-root 915G 499G 370G 58% /
tmpfs 32G 0 32G 0% /dev/shm
tmpfs 5.0M 4.0K 5.0M 1% /run/lock
tmpfs 32G 0 32G 0% /sys/fs/cgroup
mergerFS 89T 82T 6.8T 93% /home/eckoflyte/drivepool
/dev/loop0 9.2M 9.2M 0 100% /snap/canonical-livepatch/95
/dev/loop1 55M 55M 0 100% /snap/core18/1705
/dev/loop3 28M 28M 0 100% /snap/snapd/7264
/dev/loop2 50M 50M 0 100% /snap/snap-store/433
/dev/loop4 63M 63M 0 100% /snap/gtk-common-themes/1506
/dev/loop5 94M 94M 0 100% /snap/core/8935
/dev/sdd1 5.5T 4.9T 568G 90% /home/eckoflyte/mergerFS04
/dev/loop6 94M 94M 0 100% /snap/core/9066
/dev/sdc1 5.5T 5.0T 474G 92% /home/eckoflyte/mergerFS02
/dev/nvme0n1p1 511M 7.8M 504M 2% /boot/efi
/dev/sde1 5.5T 4.8T 640G 89% /home/eckoflyte/mergerFS01
/dev/loop7 243M 243M 0 100% /snap/gnome-3-34-1804/27
/dev/loop8 241M 241M 0 100% /snap/gnome-3-34-1804/24
/dev/sdi1 9.1T 8.5T 143G 99% /home/eckoflyte/PARITY01
/dev/sdm1 9.1T 8.4T 646G 94% /home/eckoflyte/mergerFS03
/dev/sdl1 9.1T 8.4T 647G 93% /home/eckoflyte/mergerFS06
/dev/sdb1 9.1T 8.4T 650G 93% /home/eckoflyte/mergerFS10
/dev/sdg1 9.1T 8.4T 650G 93% /home/eckoflyte/mergerFS05
/dev/sdh1 9.1T 8.4T 649G 93% /home/eckoflyte/mergerFS08
/dev/sda1 9.1T 8.4T 649G 93% /home/eckoflyte/mergerFS11
/dev/sdk1 9.1T 8.4T 651G 93% /home/eckoflyte/mergerFS07
/dev/sdf1 9.1T 8.5T 608G 94% /home/eckoflyte/PARITY02
/dev/sdj1 9.1T 8.4T 646G 94% /home/eckoflyte/mergerFS09
tmpfs 6.3G 20K 6.3G 1% /run/user/1000
strace of application having problem:
strace -f -o /tmp/app.strace.txt <cmd>orstrace -f -p <appPID> -o /tmp/app.strace.txtna
strace of mergerfs while app tried to do it's thing:
strace -f -p <mergerfsPID> -o /tmp/mergerfs.strace.txtna
The text was updated successfully, but these errors were encountered: