restic backup into existing repository fails with error "encoding/hex: invalid byte: U+002E '.'"

[tbraun-de]

Output of restic version

I've just re-compiled restic with a new git clone, but this didn't solve the problem:

user@dagger:~$ ./restic/restic version
restic 0.1.0 (v0.1.0-428-gd84dec4)
compiled at 2016-02-24 16:55:49 with go1.3.3

Another version on a Mac OS X machine that shows the same behaviour:

some-host:restic user$ ./restic version
restic 0.1.0 (v0.1.0-250-g4d7e802)
compiled at 2016-01-19 09:20:45 with go1.5.3

Expected behavior

Restic should - after successful authentication - start the backup process and insert an additional snapshot into the existing repository

Actual behavior

user@dagger:~$ ./restic/restic --repo /bulk/restic_repo backup /bulk/backup/mitarbeiterpaket
enter password for repository: 
encoding/hex: invalid byte: U+002E '.'
user@dagger:~$ 

Steps to reproduce the behavior

See above

Preconditions

An existing restic repository with some existing snapshots. By the way, the repository can be opened and read:

user@dagger:~$ ./restic/restic  --repo /bulk/restic_repo/ snapshots
enter password for repository: 
ID        Date                 Source      Directory
----------------------------------------------------------------------
c5a684a2  2016-01-14 21:37:21  dagger      /bulk/backup/imap
                                           /storage/data/backup
                                           /storage/music
7d083eb5  2016-01-15 09:48:25  dagger      /bulk/backup/imap
                                           /storage/data/backup
                                           /storage/music
7d3cbc27  2016-01-19 20:56:44  some-host  /Users
e80b8065  2016-01-20 06:52:03  dagger      /bulk/backup/imap
                                           /home/user
                                           /storage/data/backup
                                           /storage/music
                                           /var/www
ff0cb95f  2016-01-28 11:57:28  dagger      /bulk/backup/imap
                                           /home/user
                                           /storage/data/backup
                                           /storage/music
                                           /var/www
cbe7b851  2016-02-01 21:18:33  some-host  /Users
b97badab  2016-02-01 21:39:56  dagger      /bulk/backup/imap
                                           /home/user
                                           /storage/data/backup
                                           /storage/music
                                           /var/www
59b2e5cb  2016-02-12 14:29:46  dagger      /bulk/backup/imap
                                           /home/user
                                           /storage/data/backup
                                           /storage/music
                                           /var/www

I don't know how relevant this is, but the repository is quite large:

user@dagger:/bulk/restic_repo$ du -hs *
4.0K    config
224G    data
131M    index
8.0K    keys
8.0K    locks
40K snapshots
182M    tmp

[fd0]

Hi, thanks for reporting, this seems to be a bug. Could you please compile restic with debug support by running:

$ go run build.go -tags debug

Then please run the backup command again:

$ DEBUG_LOG=/tmp/restic-debug.log ./restic --repo /bulk/restic_repo backup /bulk/backup/mitarbeiterpaket

Afterwards please have a look at the last entries of the debug log file, maybe you can see what's going on. I suspect that there is an additional file in the index dir in the repo that shouldn't be there and contains a dot. Could you please have a look?

[tbraun-de]

Hi fd0, thanks for the quick reply! Your guess was spot-on. Some days ago, I mounted the volume that contains the restic repository and browsed the folders inside from my OS X machine. I don't really know why, I guess just because it's possible.
Now, Mac OS X has the very bad habit to leave its ".AppleDouble" folders in each and every subdirectory on a server that you visit. This is was what confused restic:

user@dagger:/bulk/restic_repo$ find . -name .AppleDouble
./.AppleDouble
./data/.AppleDouble
./snapshots/.AppleDouble
./index/.AppleDouble
./locks/.AppleDouble
./keys/.AppleDouble
./tmp/.AppleDouble

Deleting all those folders with rm -rf /bulk/restic_repo/.AppleDouble /bulk/restic_repo/*/.AppleDouble solved my issue.

By the way: Thank you very much for creating restic - it is the backup tool I've been waiting for years without really knowing what I missed with all the other tools I tried. Restic really does 100% what I need and nothing more. Great work 👍

Best regards,
Tobias

[rawtaz]

Hehe, nice guesswork there fd0 =D

[fd0]

Thanks for trying it and providing feedback! And especially thanks for taking the extra effort and register a GitHub account just to create an issue 😄

I've added issue #475 as a reminder to ignore invalid index files. Especially for local repositories, files with invalid names should be silently ignored.