Backups fail with sshfs >= 3.2.0

[icedaddy]

After running daily backups for well over a year BiT has suddenly failed a few days ago and will no longer backup. Backups are initiated via cron; stored on NAS through sshfs; system is Arch Linux; still plenty of space on the NAS; nothing has changes with the setup. Now no backups are performed and I get the following error:

Traceback (most recent call last):
  File "/usr/share/backintime/common/backintime.py", line 865, in <module>
    start_app()
  File "/usr/share/backintime/common/backintime.py", line 450, in start_app
    args.func(args)
  File "/usr/share/backintime/common/backintime.py", line 634, in backupJob
    backup(args, False)
  File "/usr/share/backintime/common/backintime.py", line 626, in backup
    ret = take_snapshot(cfg, force)
  File "/usr/share/backintime/common/backintime.py", line 70, in take_snapshot
    ret = snapshots.Snapshots( cfg ).take_snapshot( force )
  File "/usr/share/backintime/common/snapshots.py", line 1044, in take_snapshot
    ret_val, ret_error = self._take_snapshot( snapshot_id, now, include_folders )
  File "/usr/share/backintime/common/snapshots.py", line 1334, in _take_snapshot
    saveToContinueFlag.set()
  File "/usr/share/backintime/common/snapshots.py", line 2009, in set
    with open(self.flag, 'a') as f:
OSError: [Errno 22] Invalid argument: '/home/adrian/.local/share/backintime/mnt/1_9480/backintime/coyote/0/1/new_snapshot/save_to_continue'

Any ideas? Thanks

[icedaddy]

Further investigation shows that sshfs was updated from 3.0.0 to 3.2.0 and this is causing the error. I'm not sure what exactly has changes in sshfs but it's not working with BiT anymore!

[icedaddy]

It appears the writeback caching has changed with the latest sshfs. An additional flag is required when mounting the sshfs filesystem: -o unreliable_append.
To fix BiT I have inserted the following line:
sshfs.extend(['-o', 'unreliable_append'])
at around line 118 of common/sshtools.py

I'm not sure how to submit this as an code update - please advise.

[Germar]

Thanks @icedaddy. But I'd rather disable writeback cache than using something called unreliable_append for backups. I have no current Arch at hand, could you please check if it helps to add sshfs.extend(['-o', 'writeback_cache=no']) instead of your line?

[icedaddy]

Hi @Germar
As I'm sure you have read:

... To enable support for unreliable O_APPEND (which may overwrite data if the file changes on the server at a bad time), mount the file system with -o unreliable_append.

Since the files on the server are only ever updated by the backup (in fact, once a file is written by BiT it is never updated since BiT stores history) then I don't see the slightly broken O_APPEND of sshfs as presenting any risk at all in this case.

However, I have tested with the writeback cache disabled as you suggest and it all works perfectly also.

So up to you as what fix you want to include in the BiT code.

Cheers

[Germar]

You're right. That shouldn't be a risk. I was in a hurry while reading it, sorry

Thanks for checking...

[buhtz]

sshfs now is somewhere at version 3.7.*. I can't find the writeback_cache option in the manpage anymore.

I'm also not aware of any other BIT issues related to this.

Seems to me this isn't relevant anymore and/or was an edge case. Maybe we can close this?

[buhtz]

@icedaddy Can you give us feedback please if you can reproduce the problem with the latest BIT version 1.3.3 and without your patch? It seems you have discovered an edge case. Do you have an idea what could be the difference between your setup and the setup of other user not reporting this issue?

I'm still not aware of other Issues related to that "problem". On the other hand I'm not deep enough into sshfs to understand the implications and technical details.

But when there is no further activity here I would close the Issue in the near future.

[buhtz]

Will close this issue in near future if there is no new information on it.