borg create results to exit code 1

[hubitor]

Have you checked borgbackup docs, FAQ, and open Github issues?

Yes

Is this a BUG / ISSUE report or a QUESTION?

ISSUE

System information. For client/server mode post info for both machines.

Your borg version (borg -V).

borg 1.1.6

Operating system (distribution) and version.

Arch Linux

Hardware / network configuration, and filesystems used.

How much data is handled by borg?

350 GB

Full borg commandline that lead to the problem (leave away excludes and passwords)

borg create -spv --compression lz4 --list --exclude-caches --exclude-from '/mybackups/borg_exclude.txt' /mybackups/Borg/home/::mybackup-{now:%Y%m%d-%H%M%S} /home/user/

Describe the problem you're observing.

The backup seems to be created normally:

Archive name: mybackup-20180625-172629
Archive fingerprint: ...
Time (start): Mon, 2018-06-25 17:26:35
Time (end):   Mon, 2018-06-25 17:29:03
Duration: 2 minutes 27.57 seconds
Number of files: 488237
Utilization of max. archive size: 0%

Original size      Compressed size    Deduplicated size
This archive:              298.33 GB            251.67 GB             20.17 MB
All archives:                6.21 TB              5.22 TB            325.74 GB

Unique chunks         Total chunks
Chunk index:                  573789             11663204

but if run after the output:
echo $?
the output is 1.

Besides that when I check the repo with:
borg check /mybackups/Borg/home/
there is no output.

So I'm not sure if the backup is OK or not.

Can you reproduce the problem? If so, describe how. If not, describe troubleshooting steps you took before opening the issue.

Yes. Running the commands mentioned previously.

Include any warning/errors/backtraces from the system logs

[ThomasWaldmann]

Try again without -p (--progress). It doesn't mix nicely with --list (nor with any line-by-line output).

[hubitor]

Tried again without -p option. Then exit code is again 1:

echo $?                                                                
1

[ThomasWaldmann]

Can you check again if there is a warning visible now?

[hubitor]

I checked again and I see no warning.

[ThomasWaldmann]

OK. The problem now is that this likely depends somehow on your environment (files to backup, permissions, ...) so likely I will not be able to reproduce it.

[hubitor]

1. Is there no way to get more debug information?
2. What else can I try? Or the last option is just to create a new repo?

[ThomasWaldmann]

borg defaults to emit log messages at >= warning level, so they should be seen.
if you use --info (-v) or --debug, this adjusts the log level to display these levels also.
usage of some command line switches also influences log output.

a new repo likely won't change anything.

[jdchristensen]

Maybe also add --show-rc, just to confirm the exit code. Show us the full output.

Also, if -v or --debug don't reveal the problem, you could try backing up smaller portions of your source files to try to locate what is causing the warning.

[hubitor]

I tried again this way:
borg create -spv --show-rc --compression lz4 --exclude-caches --exclude-from ...

and then an error showed up :
file: open: [Errno 13] Permission denied: 'file'

it was a file in my home directory but with root permissions.
I've changed the file permissions to the normal user and now I'm getting rc 0:
terminating with success status, rc 0

but when I run:
$ borg check REPO

there is no output like before with rc 1. Is this normal? Shouldn't be some kind of output informing the user if everything is alright or not?

[ThomasWaldmann]

now that you know what to search for, can you use the original command (and the original file permissions) again and check for that warning in its output?

you can use --show-rc for borg check also.

borg commands follow unix style (either there is no problem and they are silent or there is a problem and they will tell you about it).

[hubitor]

I used the original command with the original permissions. I see no warning but these 2 lines in the output:

...
file: open: [Errno 13] Permission denied: 'file'
E file
...

I couldn't see the error with --list option.

When I run borg with --show-rc I get rc 0:

$ borg check --show-rc REPO
terminating with success status, rc 0

so I assume everything is OK now.

[ThomasWaldmann]

preparation:

(borg-env) tw@tux:~/w/borg$ touch src/rootonly
(borg-env) tw@tux:~/w/borg$ sudo chown root.root src/rootonly
[sudo] password for tw: 
(borg-env) tw@tux:~/w/borg$ sudo chmod 600 src/rootonly
(borg-env) tw@tux:~/w/borg$ borg init -e none repo

using borg 1.1-maint code for testing:

(borg-env) tw@tux:~/w/borg$ borg create repo::arch src
src/rootonly: open: [Errno 13] Permission denied: 'src/rootonly'
(borg-env) tw@tux:~/w/borg$ echo $?
1

warning msg and rc 1 as expected.

(borg-env) tw@tux:~/w/borg$ borg create --list repo::arch2 src
src/rootonly: open: [Errno 13] Permission denied: 'src/rootonly'
E src/rootonly
U src/borg/lrucache.py
U src/borg/archiver.py
...
d src
(borg-env) tw@tux:~/w/borg$ echo $?
1

warning msg and rc 1 as expected. additionally, the list output was tagged with "E" (an error happened for this file) as expected.

(borg-env) tw@tux:~/w/borg$ borg create --progress --show-rc repo::arch3 src
src/rootonly: open: [Errno 13] Permission denied: 'src/rootonly'
terminating with warning status, rc 1

even with progress output: warning clearly visible.

(borg-env) tw@tux:~/w/borg$ borg create -spv --list --show-rc repo::arch6 src
src/rootonly: open: [Errno 13] Permission denied: 'src/rootonly'                                                                                                                                        
E src/rootonly
U src/borg/lrucache.py                                                                                                                                                                                  
...
d src
------------------------------------------------------------------------------                                                                                                                          
Archive name: arch6
Archive fingerprint: 33278ac5e9854a92b59a53e903292e8d338217354be19b1a60430102c46d5a99
Time (start): Sat, 2018-06-30 18:02:06
Time (end):   Sat, 2018-06-30 18:02:06
Duration: 0.11 seconds
Number of files: 248
Utilization of max. archive size: 0%
------------------------------------------------------------------------------
                       Original size      Compressed size    Deduplicated size
This archive:               16.36 MB              6.53 MB                427 B
All archives:               32.72 MB             13.05 MB              6.53 MB

                       Unique chunks         Total chunks
Chunk index:                     251                  502
------------------------------------------------------------------------------
terminating with warning status, rc 1
(borg-env) tw@tux:~/w/borg$ 

Mixing progress and list output: warning, list output, rc like expected.

Sorry, I don't see an issue here.

[hubitor]

The original command with the original permissions produced the 2 lines I mentioned before. There was no word "error" or "warning". I was using this command with the --list option in order to get as much output as possible. But these 2 lines appeared not at the end but in between some thousands other lines. I have copied the output into a text editor and I was looking for one of these words (warning, error) and I wasn't able to see what it was going on.

With the command you used:
(borg-env) tw@tux:~/w/borg$ borg create --list repo::arch2 src

warning msg and rc 1 as expected. additionally, the list output was tagged with "E" (an error happened for this file) as expected.

there is also no such message
terminating with warning status, rc 1
like in the other cases.

As I used this command:
borg create -spv --show-rc --compression lz4 --exclude-caches --exclude-from ...
namely without the --list option I was able to see that is something is wrong since there were only a few lines in the output.

So what I'm trying to say is that although there were 1-2 lines in the output saying something went wrong during the backup creation, the error report was kind of obscured.

[ThomasWaldmann]

Well, if you request lots of output (--list without filter), you get just that.

Such full output style is maybe interesting for first run while setting up the backup, but not that much for daily runs later. Then you can either leave away --list or use it together with --filter AME to only see changes and errors/warnings.

The log level (INFO, WARNING, ERROR) is not used for the default log options as it wastes screen space and is not the usual thing for screen output. You can reconfigure the logging to include the log level (and timestamps) if you like.