Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support unattended encrypted backups without key disclosure #533

Closed
robbat2 opened this issue Jun 12, 2016 · 35 comments
Closed

Support unattended encrypted backups without key disclosure #533

robbat2 opened this issue Jun 12, 2016 · 35 comments

Comments

@robbat2
Copy link

@robbat2 robbat2 commented Jun 12, 2016

Expected behavior

restic should allow unattended encrypted backups, without disclosing any key material that can lead to decrypting backups. This does tie into #187 as asymmetric keys can be used to general session keys for example.

Actual behavior

restic requires a password or unsecure key material to do a new backup.

@fd0
Copy link
Member

@fd0 fd0 commented Jun 13, 2016

Hi, thanks for suggesting this feature. Can you please elaborate a bit on how this issue is different than #187?

@fd0 fd0 added the feature label Jun 13, 2016
@robbat2
Copy link
Author

@robbat2 robbat2 commented Jun 13, 2016

Asymmetric keys are only one of the ways this can be accomplished. Another is immediately pushing the backup to write-only destination (eg a S3 bucket where you can put new content but not read or modify anything already present). Asymmetric key usage (eg generate a session key for the data, use GPG or RSA to protect that key) do ensure that the attacker can't decrypt the backups if they do get their hands on them.

@anarcat
Copy link
Contributor

@anarcat anarcat commented Sep 18, 2017

couldn't we implement this by simply storing the password in a file and use --password-file? that file could then be encrypted with $WHATEVER mechanism you prefer. for example, to encrypt the password file with GnuPG, you could do:

restic backup --password-file <(gpg --decrypt keyfile.gpg)

then of course, it's only as unattended as gpg will allow it, but at least there's some level of automation there... i guess that storing the password in standard keyrings (e.g. gnome-keyring and so on) might be another solution...

but yes, #187 would probably be better in the long term.

@jcollie
Copy link

@jcollie jcollie commented Oct 20, 2017

The ability to use HashiCorp's Vault https://www.vaultproject.io/ to store the password would be awesome!

@osamuaoki
Copy link

@osamuaoki osamuaoki commented Oct 25, 2017

For convenience, I like to see Gnome keyring supported :-)

FYI: I think the C code "char *msmtp_password_callback ..." in src/msmtp.c in the source of msmtp gives good example for how to implement such. It also has code for MAC and WINDOWS equivalents.
See code http://msmtp.sourceforge.net/

@robbat2
Copy link
Author

@robbat2 robbat2 commented Oct 26, 2017

@osamuaoki there are multiple implementations already for accessing keyring: https://github.com/tmc/keyring

@osamuaoki
Copy link

@osamuaoki osamuaoki commented Oct 27, 2017

@spanezz
Copy link

@spanezz spanezz commented Dec 24, 2017

I wish there was a --password-script as a --password-file equivalent that read the password from the standard output of a script. Or maybe just --password-file '!script', where the file is executed if it is prefixed with a !.

That way I could have a script that does something like gpg --decrypt restic.gpg that would then work with gpg-agent so that a sequence of restic invocations (say, backup, forget, prune) would only ask for a password once. That's an idea. Having a script would be rather flexible, and communication via a pipe is more secure than via the environment or via a file in the filesystem.

@rawtaz
Copy link
Contributor

@rawtaz rawtaz commented Dec 24, 2017

Specifying stdin with arguments/options that take a file is usually done by specifying stdin as -. In this case it would be --password-file -. It's quite common in the UNIX/Linux world.

Edit: But perhaps that's not what you want (reading the password from stdin)? You really want restic to execute a script to get the password?

@fd0
Copy link
Member

@fd0 fd0 commented Dec 24, 2017

@spanezz there's no need to implement that in restic, as it is provided by your shell:

$ restic --password-file <( gpg --decrypt path-to-key.gpg ) backup [...]

Calling restic in this way, the shell starts gpg and provides restic with file descriptor that it can read like a file.

@spanezz
Copy link

@spanezz spanezz commented Dec 24, 2017

@fd0 nice idea!

It looks like there is too much magic going on, though, at least on restic init, which works if --password-file points to a file, but not if it points to /dev/stdin or /dev/fd/nn:

$ restic init --repo test --password-file <(echo password)
Fatal: passwords do not match
$ echo password > /tmp/foo
$ restic init --repo test --password-file /tmp/foo
created restic backend 4ff0ddf23e at test

Please note that knowledge of your password is required to access
the repository. Losing your password means that your data is
irrecoverably lost.
@fd0
Copy link
Member

@fd0 fd0 commented Dec 24, 2017

Yeah, init is a special case, that indeed does not work. We could fix the code to support it though.

@mfsu
Copy link

@mfsu mfsu commented Feb 10, 2018

How about this keyring provider?

https://github.com/zalando/go-keyring

@matthijskooijman
Copy link

@matthijskooijman matthijskooijman commented Aug 26, 2018

Isn't this discussion getting off-topic? AFAIU storing the password in a keyring and allowing it to be extracted automatically for unattended backups does not solve this request at all. During the backup, restic will have access to the password / key used for encryption. When an attacker compromises the backed-up-machine, he can also gain the key and access to the backup storage, allowing access to and decryption of the backups (including older versions and perhaps also backups of other host, though I haven't checked if the multiple key support fixes this).

@mfsu
Copy link

@mfsu mfsu commented Aug 26, 2018

@tcely
Copy link

@tcely tcely commented Aug 29, 2018

My summary of this issue:

  • fix init password file case
  • password script feature
  • vault integration
  • keyring integration

I haven't seen a reasonable alternative to asymmetric keys for automated backups mentioned. As I understand it, both vault and keyring integrations are just specific cases of the more general script feature. All of these do not have the property of not being able to decrypt backups using the same key material used to encrypt new backups.

It's also worth noting that authorization to access repository data is a distinct concern.

As I see it, there are potentially four levels of encryption when creating backups:

  1. The encrypted chunk of backed up data.
  2. The repository meta-data
  3. The storage of the repository
  4. The transport used to access the repository

A solution that provides forward secrecy and uses asymmetric keys to protect the meta-data seems ideal to me. The encryption for storage and transport is likely outside the scope of restic. The data chunks are best served by symmetric encryption in my opinion. How we store those symmetric keys is where we have some flexibility to play with.

@graphixillusion
Copy link

@graphixillusion graphixillusion commented Nov 11, 2018

restic --password-file <( gpg --decrypt path-to-key.gpg ) ...

If i use this command under windows, i get: "unable to find specified file". Any way to do this under windows? Thank you!

@mathstuf
Copy link

@mathstuf mathstuf commented Nov 12, 2018

@graphixillusion The <( command ) syntax is a bash-ism. Are you using bash or cmd? I don't know of a way to do it with cmd.

@graphixillusion
Copy link

@graphixillusion graphixillusion commented Nov 13, 2018

i'm using cmd. i tried with winbash too but it doesn't work either. For now the only way to do it is temp decrypt to a file, use the decrypted file with restic and then delete it.

@mathstuf
Copy link

@mathstuf mathstuf commented Nov 13, 2018

You could probably use gpg --decrypt | restic --password-file /dev/stdin for winbash. cmd might has something similar too.

@graphixillusion
Copy link

@graphixillusion graphixillusion commented Nov 14, 2018

@mathstuf
with cmd i can use the findstr "^" command. For example this command works good in cmd:

gpg --decrypt "blablabla" | findstr "^" ---> it will output the password

The problem seems that restic doesn't accept this comand followed by the flag -p/--password-file

restic -p (or --password-file) findstr "^" doesn't work.

I tried with winbash too but it doesn't work either:

Resolving password failed: Fatal: /dev/stdin does not exist

@mathstuf
Copy link

@mathstuf mathstuf commented Nov 14, 2018

Even on Linux, it isn't restic that's doing the "command output as argument". That's bash. If cmd doesn't have an equivalent, that's a deficiency of Windows. Here's a ServerFault question which might help. Though I think this SuperUser question may be more useful.

@graphixillusion
Copy link

@graphixillusion graphixillusion commented Nov 14, 2018

I think, for now, i found the right compromise: make a ramdisk; temp save the decrypted password file into it; make whatever you need to do with restic; dismount and delete the ramdisk. Better than nothing. ;) In the meantime, if anyone knows how to do this with the right syntax with cmd under windows, please let me know. thank you.

@pdf
Copy link

@pdf pdf commented Nov 14, 2018

Even on Linux, it isn't restic that's doing the "command output as argument". That's bash. If cmd doesn't have an equivalent, that's a deficiency of Windows.

To be fair, not even all shells on Linux support this syntax - this is definitely a bashism. As such, I'd say relying on this is a deficiency of restic.

@mholt
Copy link
Contributor

@mholt mholt commented Nov 14, 2018

a deficiency of restic.

Or a deficiency in other shells, for not being able to write files like that.

Just throwing that out there. 😇

Anyway, is this issue different from #187? On the surface, it doesn't seem very feasible, since doesn't some data have to be decrypted in order to encrypt new data and add it to the backup? So even if it was "asymmetric" you would still need a decryption key in order to make new backups, no?

@smlx
Copy link

@smlx smlx commented Nov 15, 2018

Something like this might work for shells without process substitution:

tmppipe=$(mktemp -u)
mkfifo -m 600 $tmppipe
gpg --decrypt ... | $tmppipe &
restic --password-file $tmppipe
@fd0
Copy link
Member

@fd0 fd0 commented Nov 15, 2018

First, please ask usage questions in the forum. We're using the GitHub issue tracker for bugs and features only.

Then: Why so complicated: echo foo | restic -r /tmp/repo snapshots works great...

@graphixillusion
Copy link

@graphixillusion graphixillusion commented Nov 15, 2018

@fd0

Amazing, thank you! I did it using your suggestion with this command:

gpg --decrypt blablabla | restic -r /path/to/repo snapshots

@edysli
Copy link

@edysli edysli commented Apr 30, 2019

Since version 0.9.4, there is the --password-command option (#2094).

@rawtaz
Copy link
Contributor

@rawtaz rawtaz commented Nov 21, 2019

@robbat2 AS @edysli mentioned above, restic now has the --password-command argument. Does that solve your problem? If so we can close this issue.

@ja0nz
Copy link

@ja0nz ja0nz commented Dec 18, 2019

Recalling @matthijskooijman I would say the problem is not really solved as there is no true asymmetric key handling. Unattented gpg --decrypt could be compromised at runtime.

Saying that a reader of this thread may take a look at Duplicacy RSA encryption which supports public key chunk encryption and private key restoring.

@mathstuf
Copy link

@mathstuf mathstuf commented Dec 19, 2019

Note that without a key, dedupe doesn't work so well because you need to decrypt to compare contents. I think the way obnam did it is that it allowed different host-specific keys to decrypt the real key. That would allow for at least revoking compromised/deprecated backup hosts without having to change the repository password.

@7usr7local
Copy link

@7usr7local 7usr7local commented Feb 14, 2020

I don't see any notable increase in security in using any kind of keyring, key store or whatsoever compared to storing the "secret" (aka password) in plaintext. If your backup script has access to the keyring or whatever, any attacker can just use that part of your backup script to decrypt the secret.
The only scenario this makes any sense is to "open" the keyring manually and keep it open until shutdown of the server. That way loosing your harddisk would not compromise your backup's security. This can of course be achieved as well by using encryption for the filesystem the password-file is stored on. All approaches of this kind imply that backup won't work after an unattended reboot.
In the end, @robbat2 s request

without disclosing any key material

is not fulfilled with any of the shell tricks, key ring accesses or meanwhile available command line options.

Just my two pence though.

@mfsu
Copy link

@mfsu mfsu commented Feb 15, 2020

@MichaelEischer
Copy link
Member

@MichaelEischer MichaelEischer commented Oct 4, 2020

Closing this as the remaining issue seems to require asymmetric encryption which belongs into #187.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Linked pull requests

Successfully merging a pull request may close this issue.

None yet