Input/output error trying to access repository mounted via borg mount

[n0ctua]

Have you checked borgbackup docs, FAQ, and open Github issues?

Yes

Is this a BUG / ISSUE report or a QUESTION?

Issue

System information. For client/server mode post info for both machines.

Your borg version (borg -V).

Borg: 1.1.11
Python: CPython 3.5.2
msgpack: 0.5.6

Operating system (distribution) and version.

Linux: Ubuntu 16.04 xenial
Kernel: 4.4.0-176-generic

Hardware / network configuration, and filesystems used.

• backup repository on an external hdd, fully encrypted with veracrypt
• mount point on ext4 internal ssd

How much data is handled by borg?

~ 60GB

Full borg commandline that lead to the problem (leave away excludes and passwords)

sudo mkdir /mnt/repo
sudo borg mount /media/veracrypt1/Backup/ /mnt/repo

Describe the problem you're observing.

Trying to access the mounted directory in any way throws an Input/output error.
Borg itself does not report any problems.

Can you reproduce the problem? If so, describe how. If not, describe troubleshooting steps you took before opening the issue.

sudo ls /mnt/repo
ls: cannot access '/mnt/repo': Input/output error

Include any warning/errors/backtraces from the system logs

I could not find any errors or backtraces in the log. borg check doesn't report any problems with my repo. Also the --debug option did not reveal any issues.

[ThomasWaldmann]

• look into kernel log (dmesg) - any fs or hw errors?
• do the mount using the foreground option (-f iirc) - does the output show anything when you retry after that?

[n0ctua]

Thank you very much for your reply.

1. look into kernel log (dmesg) - any fs or hw errors?

This is the log from the moment I connect the external drive up to after I ran the mount command:

[21982.284691] usb 1-9: reset full-speed USB device number 5 using xhci_hcd
[21982.469429] usb 1-9: ep 0x84 - rounding interval to 64 microframes, ep desc says 80 microframes
[22490.241976] usb 1-9: reset full-speed USB device number 5 using xhci_hcd
[22490.426782] usb 1-9: ep 0x84 - rounding interval to 64 microframes, ep desc says 80 microframes
[22605.332265] usb 2-4: new SuperSpeed USB device number 5 using xhci_hcd
[22605.348993] usb 2-4: New USB device found, idVendor=1058, idProduct=1230
[22605.349007] usb 2-4: New USB device strings: Mfr=2, Product=3, SerialNumber=1
[22605.349017] usb 2-4: Product: My Book 1230
[22605.349026] usb 2-4: Manufacturer: Western Digital
[22605.349035] usb 2-4: SerialNumber: xxx
[22605.352058] usb-storage 2-4:1.0: USB Mass Storage device detected
[22605.352301] scsi host5: usb-storage 2-4:1.0
[22606.352978] scsi 5:0:0:0: Direct-Access     WD       My Book 1230     1050 PQ: 0 ANSI: 6
[22606.353503] scsi 5:0:0:1: Enclosure         WD       SES Device       1050 PQ: 0 ANSI: 6
[22606.356847] sd 5:0:0:0: Attached scsi generic sg1 type 0
[22606.357579] ses 5:0:0:1: Attached Enclosure device
[22606.358137] ses 5:0:0:1: Attached scsi generic sg2 type 13
[22606.358443] sd 5:0:0:0: [sdb] Spinning up disk...
[22606.359217] ses 5:0:0:1: Wrong diagnostic page; asked for 1 got 0
[22606.359234] ses 5:0:0:1: Failed to get diagnostic page 0xffffffea
[22606.359247] ses 5:0:0:1: Failed to bind enclosure -19
[22607.359978] ...........ready
[22617.575710] sd 5:0:0:0: [sdb] 976746240 4096-byte logical blocks: (4.00 TB/3.64 TiB)
[22617.576214] sd 5:0:0:0: [sdb] Write Protect is off
[22617.576235] sd 5:0:0:0: [sdb] Mode Sense: 53 00 10 08
[22617.576720] sd 5:0:0:0: [sdb] No Caching mode page found
[22617.576747] sd 5:0:0:0: [sdb] Assuming drive cache: write through
[22617.691862]  sdb: sdb1
[22617.693772] sd 5:0:0:0: [sdb] Attached SCSI disk
[22688.025500] usb 1-9: reset full-speed USB device number 5 using xhci_hcd
[22688.210182] usb 1-9: ep 0x84 - rounding interval to 64 microframes, ep desc says 80 microframes
[22701.116483] usb 1-9: reset full-speed USB device number 5 using xhci_hcd
[22701.301007] usb 1-9: ep 0x84 - rounding interval to 64 microframes, ep desc says 80 microframes
[22712.135947] usb 1-9: reset full-speed USB device number 5 using xhci_hcd
[22712.320151] usb 1-9: ep 0x84 - rounding interval to 64 microframes, ep desc says 80 microframes
[22723.517945]  sdb: sdb1
[22769.726679] usb 1-9: reset full-speed USB device number 5 using xhci_hcd
[22769.913311] usb 1-9: ep 0x84 - rounding interval to 64 microframes, ep desc says 80 microframes
[22772.514901] xhci_hcd 0000:00:14.0: WARN Event TRB for slot 6 ep 2 with no TDs queued?
[22772.533666] xhci_hcd 0000:00:14.0: WARN Event TRB for slot 6 ep 2 with no TDs queued?
[22772.543998] xhci_hcd 0000:00:14.0: WARN Event TRB for slot 6 ep 2 with no TDs queued?
[22775.487227] xhci_hcd 0000:00:14.0: WARN Event TRB for slot 6 ep 2 with no TDs queued?
[22775.505327] xhci_hcd 0000:00:14.0: WARN Event TRB for slot 6 ep 2 with no TDs queued?
[22775.515416] xhci_hcd 0000:00:14.0: WARN Event TRB for slot 6 ep 2 with no TDs queued?
[22950.689885] [drm:intel_pipe_update_end [i915_bpo]] *ERROR* Atomic update failure on pipe A (start=20252 end=20253) time 128 us, min 1430, max 1439, scanline start 1429, end 1441

2. do the mount using the foreground option (-f iirc) - does the output show anything when you retry after that?

This is the terminal output, when I run borg mount using the --foreground parameter and trying to access the mount point.

sudo borg mount --foreground /media/veracrypt1/Backup/ /mnt/repo
Enter passphrase for key /media/veracrypt1/Backup/: 
Exception after kill:
Traceback (most recent call last):
  File "src/llfuse/handlers.pxi", line 69, in llfuse.capi.fuse_getattr (src/llfuse/capi_linux.c:3207)
  File "src/llfuse/handlers.pxi", line 70, in llfuse.capi.fuse_getattr (src/llfuse/capi_linux.c:3158)
  File "/usr/lib/python3/dist-packages/borg/fuse.py", line 523, in getattr
    def getattr(self, inode, ctx=None):
  File "/usr/lib/python3/dist-packages/borg/helpers.py", line 2295, in handler
    raise exc_cls
borg.helpers.SigTerm
Exception after kill:
Traceback (most recent call last):
  File "src/llfuse/handlers.pxi", line 69, in llfuse.capi.fuse_getattr (src/llfuse/capi_linux.c:3207)
  File "src/llfuse/handlers.pxi", line 70, in llfuse.capi.fuse_getattr (src/llfuse/capi_linux.c:3158)
  File "/usr/lib/python3/dist-packages/borg/fuse.py", line 544, in getattr
    entry.st_birthtime_ns = item.get('birthtime', mtime_ns)
AttributeError: 'EntryAttributes' object has no attribute 'st_birthtime_ns'

[infectormp]

look like this caused by #4763 (master) / #4767 (1.1-maint).

[ThomasWaldmann]

Looks like there are 2 issues:

• there are quite a lot of USB troubles in your kernel log - they are not borg-related and you should fix them or use another USB disk that does not make that much troubles.

• there might be an issue with our fuse code, as noted in previous comment.

[ThomasWaldmann]

I see st_birthtime_ns there:

https://github.com/python-llfuse/python-llfuse/blob/master/src/misc.pxi#L503

@n0ctua What llfuse version do you use?

Stuff should be present for llfuse >= 1.3.

Update: Ubuntu Xenial has llfuse 0.41, thus it does not work there.

[ThomasWaldmann]

@LocutusOfBorg in case you want to do a hotfix for xenial, this ^^^ is the patch you need.

[n0ctua]

@ThomasWaldmann Yes, I have llfuse version 0.41.1.

[ThomasWaldmann]

@n0ctua you could try the patch. applying it is a bit dirty as you would have to patch files that come from a package, but would be useful to actually test whether the fix works.

[n0ctua]

@ThomasWaldmann Thank you. I will try that and report back to you.

[n0ctua]

@ThomasWaldmann I tried the patch and mounting works perfectly now, even with llfuse 0.41.1. Thank you for your work! 👍

[ThomasWaldmann]

@n0ctua thanks for testing! I merged it into 1.1-maint branch, so it will be in next release.

[tdahh]

Even if it seems to be fixed. The mountpoint going away might be related to a fuse system that is mounted by a user but borg running as root trying to access the user mounted filesystem.

[ThomasWaldmann]

@tdahh if the borg mount process dies due to an exception, the mount also does not work.

[ThomasWaldmann]

#5072 is the fix in case a package maintainer wants to release a hotfix.

[semente]

Will this fix be released in https://launchpad.net/~costamagnagianfranco/+archive/ubuntu/borgbackup?

[LocutusOfBorg]

yes @semente , it has been released on the ppa.