[BUG] Backup aborts on files with Operation not permitted.

[schymans]

Bug summary

My backup keeps aborting with the following error:

ListError: 'Documents/somefile1.pdf' [Errno 1] Operation not permitted:
 b'/home/user/Documents/somefile1.pdf'
ListError: 'Documents/somefile2.pdf' [Errno 1] Operation not permitted:
 b'/home/user/Documents/somefile2.pdf'
Previous backup seems to have failed, regressing destination now.

Version, Python, Operating System

Call rdiff-backup info (version >= 2.1) or rdiff-backup -v9 and replace the following line with the output, repeat for each environment impacted:

2021-10-18 09:25:52.582634 +0200  <CLIENT-129275>  Using rdiff-backup version 2.0.5
2021-10-18 09:25:52.582679 +0200  <CLIENT-129275>  	with cpython /usr/bin/python3 version 3.7.3
2021-10-18 09:25:52.585229 +0200  <CLIENT-129275>  	on Linux-5.10.0-9progress5+u1-amd64-x86_64-with-Progress_Linux-5.99-engywuck-backports, fs encoding utf-8

rdiff-backup call

How did you call rdiff-backup, with which parameters:

sudo rdiff-backup --tempdir ~/rubbish --print-statistics --exclude ~/.gvfs --exclude ~/.cache --exclude '**'/tmp --exclude '**'/temp --exclude '**'/rubbish --exclude '**'/Trash  --exclude ~/Programs --exclude ~/Downloads --exclude ~/Music --exclude ~/polybox --exclude ~/'VirtualBox*' --exclude ~/.local/share/data/'Mendeley*' --terminal-verbosity 7 ~/ /media/usbdrive/rdiff_backup/

What happened and what did you expect?

As it turned out, I could not actually open either of these files on my system, as they were some weird security-enabled Adobe pdfs I had been sent. Still, I expected the backup to proceed with the files that are accessible.

I removed the corrupted files and re-ran the backup, but it failed again on a similar file. It does not even produce a traceback, here is what I get in the terminal:

Incrementing mirror file /media/usbdrive/rdiff_backup/Documents/zotero/storage/GABXSKKU
Touching /media/usbdrive/rdiff_backup/rdiff-backup-data/increments/Documents/zotero/storage/GABXSKKU.2021-03-02T23:19:08+01:00.dir
Copying inc attrs from (b'Documents', b'zotero', b'storage', b'GABXSKKU') to /media/usbdrive/rdiff_backup/rdiff-backup-data/increments/Documents/zotero/storage/GABXSKKU.2021-03-02T23:19:08+01:00.dir
Previous backup seems to have failed, regressing destination now.
Fatal Error: Could not open b'/media/usbdrive/rdiff_backup/Documents/zotero/storage/GABXSKKU/Monteith_Unsworth_2013_Principles.pdf' for reading. Check permissions on file.
Setting time of /media/usbdrive/rdiff_backup/rdiff-backup-data/increments/Documents/zotero/storage/GABXSKKU.2021-03-02T23:19:08+01:00.dir to 1552294098
Getting signature of Documents/zotero/storage/GABXSKKU/Monteith_Unsworth_2013_Principles.pdf with blocksize 3472
Getting signature of Documents/zotero/storage/GABXSKKU/Monteith_Unsworth_2013_Principles.pdf with blocksize 3472

And then it just proceeds to the next backup in my script. Again, I cannot open the pdf on my system, but I am pretty sure that I was able to open it earlier. I can't check, as I cannot restore an older version of the file since my previous backup has failed.

More information

If you can, please repeat the action leading to the error, adding the option -v9 and attach the output to this bug report.

I cannot right now, as it takes hours and hours before the error occurs. Will run it over night again.

[ericzolf]

I'm wondering about the actual issue at hand:

1. rdiff-backup should skip unreadable files and not care further. This said, it might choke on unexpected errors.
2. As I understand Adobe DRM, it's just an encrypted file with fancy decryption algorithm and nothing much more. It shouldn't keep rdiff-backup from reading the file. What do the following commands output:

la -laZ xxx.pdf
file xxx.pdf
cat xxx.pdf | wc

3. I'm also wondering about rdiff-backup complaining about the PDF-files before noticing that the previous backup failed. Is this really the order?

[schymans]

Thanks, @ericzolf!

1. Thanks for the confirmation, I wondered.
2. I don't understand the la command and nor does my system, so couldn't run it. But here is the output of the others:

$ file Monteith_Unsworth_2013_Principles.pdf 
Monteith_Unsworth_2013_Principles.pdf: writable, regular file, no read permission

$ cat Monteith_Unsworth_2013_Principles.pdf | wc
cat: Monteith_Unsworth_2013_Principles.pdf: Operation not permitted
      0       0       0

3. Yes, the terminal output is very strange. I will try again with the -v9 option and report back.

[schymans]

Holy cow, apparently Sophos AV can cause this: https://forums.centos.org/viewtopic.php?t=70024
I have MacAfee instead and after de-activating it, I am able to open the file. This is a ridiculous 'feature' of an antivirus program, just blocking a file without any explanation. I guess this qualifies as "unexptected error"!

[schymans]

Actually, if there is a way to catch such errors, this could save quite a few people the headache I got.

[ikus060]

We would require the exact Exception raise in such a scenario. Ideally if you could run this backup using `-v 9` would be helpful.

…

On Mon, Oct 18, 2021 at 2:15 PM Stan Schymanski ***@***.***> wrote: Actually, if there is a way to catch such errors, this could save quite a few people the headache I got. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#628 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAHA5I5Z7R5FXAPCSJHPRS3UHRP5HANCNFSM5GF5M5MA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

-- IKUS Software inc. https://www.ikus-soft.com/ 514-971-6442 130 rue Doris St-Colomban, QC J5K 1T9

[ericzolf]

Errno 1 ist errno.EPERM which we already ignore at

rdiff-backup/src/rdiff_backup/robust.py

Line 30 in cb01639

_robust_errno_list = [errno.EPERM, errno.ENOENT, errno.EACCES, errno.EBUSY,

So this isn't the issue. There must be something else, and here I agree with Patrik, we'll understand it only with a full log in debug mode.

[ikus060]

@schymans Could you provide more details for this issue ? Otherwise we should close this ticket.

[schymans]

Sorry, I'm travelling right now and cannot re-run the backup with the verbose option. Will try next week.

[ericzolf]

Next week was a long time ago...

[schymans]

Ooops, I'm really sorry about this. After I had de-activated my antivirus the backup ran smoothly and since then the error did not re-occur, even with the antivirus active. So unfortunately, I cannot reproduce the error myself any more. I will therefore close this issue and if the problem re-occurs, I will run it with -v 9 and open a new issue.