New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support unattended encrypted backups without key disclosure #533

Open
robbat2 opened this Issue Jun 12, 2016 · 28 comments

Comments

Projects
None yet
@robbat2
Copy link

robbat2 commented Jun 12, 2016

Expected behavior

restic should allow unattended encrypted backups, without disclosing any key material that can lead to decrypting backups. This does tie into #187 as asymmetric keys can be used to general session keys for example.

Actual behavior

restic requires a password or unsecure key material to do a new backup.

@fd0

This comment has been minimized.

Copy link
Member

fd0 commented Jun 13, 2016

Hi, thanks for suggesting this feature. Can you please elaborate a bit on how this issue is different than #187?

@fd0 fd0 added the feature label Jun 13, 2016

@robbat2

This comment has been minimized.

Copy link
Author

robbat2 commented Jun 13, 2016

Asymmetric keys are only one of the ways this can be accomplished. Another is immediately pushing the backup to write-only destination (eg a S3 bucket where you can put new content but not read or modify anything already present). Asymmetric key usage (eg generate a session key for the data, use GPG or RSA to protect that key) do ensure that the attacker can't decrypt the backups if they do get their hands on them.

@anarcat

This comment has been minimized.

Copy link
Contributor

anarcat commented Sep 18, 2017

couldn't we implement this by simply storing the password in a file and use --password-file? that file could then be encrypted with $WHATEVER mechanism you prefer. for example, to encrypt the password file with GnuPG, you could do:

restic backup --password-file <(gpg --decrypt keyfile.gpg)

then of course, it's only as unattended as gpg will allow it, but at least there's some level of automation there... i guess that storing the password in standard keyrings (e.g. gnome-keyring and so on) might be another solution...

but yes, #187 would probably be better in the long term.

@jcollie

This comment has been minimized.

Copy link

jcollie commented Oct 20, 2017

The ability to use HashiCorp's Vault https://www.vaultproject.io/ to store the password would be awesome!

@osamuaoki

This comment has been minimized.

Copy link

osamuaoki commented Oct 25, 2017

For convenience, I like to see Gnome keyring supported :-)

FYI: I think the C code "char *msmtp_password_callback ..." in src/msmtp.c in the source of msmtp gives good example for how to implement such. It also has code for MAC and WINDOWS equivalents.
See code http://msmtp.sourceforge.net/

@robbat2

This comment has been minimized.

Copy link
Author

robbat2 commented Oct 26, 2017

@osamuaoki there are multiple implementations already for accessing keyring: https://github.com/tmc/keyring

@osamuaoki

This comment has been minimized.

Copy link

osamuaoki commented Oct 27, 2017

@spanezz

This comment has been minimized.

Copy link

spanezz commented Dec 24, 2017

I wish there was a --password-script as a --password-file equivalent that read the password from the standard output of a script. Or maybe just --password-file '!script', where the file is executed if it is prefixed with a !.

That way I could have a script that does something like gpg --decrypt restic.gpg that would then work with gpg-agent so that a sequence of restic invocations (say, backup, forget, prune) would only ask for a password once. That's an idea. Having a script would be rather flexible, and communication via a pipe is more secure than via the environment or via a file in the filesystem.

@rawtaz

This comment has been minimized.

Copy link
Contributor

rawtaz commented Dec 24, 2017

Specifying stdin with arguments/options that take a file is usually done by specifying stdin as -. In this case it would be --password-file -. It's quite common in the UNIX/Linux world.

Edit: But perhaps that's not what you want (reading the password from stdin)? You really want restic to execute a script to get the password?

@fd0

This comment has been minimized.

Copy link
Member

fd0 commented Dec 24, 2017

@spanezz there's no need to implement that in restic, as it is provided by your shell:

$ restic --password-file <( gpg --decrypt path-to-key.gpg ) backup [...]

Calling restic in this way, the shell starts gpg and provides restic with file descriptor that it can read like a file.

@spanezz

This comment has been minimized.

Copy link

spanezz commented Dec 24, 2017

@fd0 nice idea!

It looks like there is too much magic going on, though, at least on restic init, which works if --password-file points to a file, but not if it points to /dev/stdin or /dev/fd/nn:

$ restic init --repo test --password-file <(echo password)
Fatal: passwords do not match
$ echo password > /tmp/foo
$ restic init --repo test --password-file /tmp/foo
created restic backend 4ff0ddf23e at test

Please note that knowledge of your password is required to access
the repository. Losing your password means that your data is
irrecoverably lost.
@fd0

This comment has been minimized.

Copy link
Member

fd0 commented Dec 24, 2017

Yeah, init is a special case, that indeed does not work. We could fix the code to support it though.

@mfsu

This comment has been minimized.

Copy link

mfsu commented Feb 10, 2018

How about this keyring provider?

https://github.com/zalando/go-keyring

@matthijskooijman

This comment has been minimized.

Copy link

matthijskooijman commented Aug 26, 2018

Isn't this discussion getting off-topic? AFAIU storing the password in a keyring and allowing it to be extracted automatically for unattended backups does not solve this request at all. During the backup, restic will have access to the password / key used for encryption. When an attacker compromises the backed-up-machine, he can also gain the key and access to the backup storage, allowing access to and decryption of the backups (including older versions and perhaps also backups of other host, though I haven't checked if the multiple key support fixes this).

@mfsu

This comment has been minimized.

Copy link

mfsu commented Aug 26, 2018

@tcely

This comment has been minimized.

Copy link

tcely commented Aug 29, 2018

My summary of this issue:

  • fix init password file case
  • password script feature
  • vault integration
  • keyring integration

I haven't seen a reasonable alternative to asymmetric keys for automated backups mentioned. As I understand it, both vault and keyring integrations are just specific cases of the more general script feature. All of these do not have the property of not being able to decrypt backups using the same key material used to encrypt new backups.

It's also worth noting that authorization to access repository data is a distinct concern.

As I see it, there are potentially four levels of encryption when creating backups:

  1. The encrypted chunk of backed up data.
  2. The repository meta-data
  3. The storage of the repository
  4. The transport used to access the repository

A solution that provides forward secrecy and uses asymmetric keys to protect the meta-data seems ideal to me. The encryption for storage and transport is likely outside the scope of restic. The data chunks are best served by symmetric encryption in my opinion. How we store those symmetric keys is where we have some flexibility to play with.

@graphixillusion

This comment has been minimized.

Copy link

graphixillusion commented Nov 11, 2018

restic --password-file <( gpg --decrypt path-to-key.gpg ) ...

If i use this command under windows, i get: "unable to find specified file". Any way to do this under windows? Thank you!

@mathstuf

This comment has been minimized.

Copy link

mathstuf commented Nov 12, 2018

@graphixillusion The <( command ) syntax is a bash-ism. Are you using bash or cmd? I don't know of a way to do it with cmd.

@graphixillusion

This comment has been minimized.

Copy link

graphixillusion commented Nov 13, 2018

i'm using cmd. i tried with winbash too but it doesn't work either. For now the only way to do it is temp decrypt to a file, use the decrypted file with restic and then delete it.

@mathstuf

This comment has been minimized.

Copy link

mathstuf commented Nov 13, 2018

You could probably use gpg --decrypt | restic --password-file /dev/stdin for winbash. cmd might has something similar too.

@graphixillusion

This comment has been minimized.

Copy link

graphixillusion commented Nov 14, 2018

@mathstuf
with cmd i can use the findstr "^" command. For example this command works good in cmd:

gpg --decrypt "blablabla" | findstr "^" ---> it will output the password

The problem seems that restic doesn't accept this comand followed by the flag -p/--password-file

restic -p (or --password-file) findstr "^" doesn't work.

I tried with winbash too but it doesn't work either:

Resolving password failed: Fatal: /dev/stdin does not exist

@mathstuf

This comment has been minimized.

Copy link

mathstuf commented Nov 14, 2018

Even on Linux, it isn't restic that's doing the "command output as argument". That's bash. If cmd doesn't have an equivalent, that's a deficiency of Windows. Here's a ServerFault question which might help. Though I think this SuperUser question may be more useful.

@graphixillusion

This comment has been minimized.

Copy link

graphixillusion commented Nov 14, 2018

I think, for now, i found the right compromise: make a ramdisk; temp save the decrypted password file into it; make whatever you need to do with restic; dismount and delete the ramdisk. Better than nothing. ;) In the meantime, if anyone knows how to do this with the right syntax with cmd under windows, please let me know. thank you.

@pdf

This comment has been minimized.

Copy link

pdf commented Nov 14, 2018

Even on Linux, it isn't restic that's doing the "command output as argument". That's bash. If cmd doesn't have an equivalent, that's a deficiency of Windows.

To be fair, not even all shells on Linux support this syntax - this is definitely a bashism. As such, I'd say relying on this is a deficiency of restic.

@mholt

This comment has been minimized.

Copy link
Contributor

mholt commented Nov 14, 2018

a deficiency of restic.

Or a deficiency in other shells, for not being able to write files like that.

Just throwing that out there. 😇

Anyway, is this issue different from #187? On the surface, it doesn't seem very feasible, since doesn't some data have to be decrypted in order to encrypt new data and add it to the backup? So even if it was "asymmetric" you would still need a decryption key in order to make new backups, no?

@smlx

This comment has been minimized.

Copy link

smlx commented Nov 15, 2018

Something like this might work for shells without process substitution:

tmppipe=$(mktemp -u)
mkfifo -m 600 $tmppipe
gpg --decrypt ... | $tmppipe &
restic --password-file $tmppipe
@fd0

This comment has been minimized.

Copy link
Member

fd0 commented Nov 15, 2018

First, please ask usage questions in the forum. We're using the GitHub issue tracker for bugs and features only.

Then: Why so complicated: echo foo | restic -r /tmp/repo snapshots works great...

@graphixillusion

This comment has been minimized.

Copy link

graphixillusion commented Nov 15, 2018

@fd0

Amazing, thank you! I did it using your suggestion with this command:

gpg --decrypt blablabla | restic -r /path/to/repo snapshots

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment