Manual decryption of backup files

[timka]

Since a backup is just a copy of PG data directory I'd like to find a way to use a copy of that, after decryption and decompression, as is. However I don't seem to find anything related in docs.

I tried something like openssl enc -d -${MY_CIPHER_TYPE} -kfile ${MY_PASS_FILE} -in ${SOME_ZST_FILE_FROM_BACKUP} -out output -pbkdf2 which resulted in bad decrypt. Tried supplying the password both as is in config and base64 decoded.

What approach should I take?

Thanks!

[pgstef]

Hi,

It really depends on what you want to do exactly. Data files are totally useless without the recovery being performed, replaying wal activity from the archives to at least make the data consistent. So the easiest way to get a data directory decrypted and ready for the recovery is to use the restore command (even if you restore to a different temporary location with restore --pg1-path=/some_temp_location --archive-mode=off.

Different level of encryption are applied and if you only need to recover 1 specific file. You could look at the source code to sort this out, or you can simply use the repo-get command which is made for that and will perform the decryption for you (usually useful to decrypt WAL archives or backup manifests).

[timka]

@pgstef Thanks for the quick reply. Ideally I'd like to have a way to decrypt stuff even in the circumstances where I don't have pgbackrest at hand (at all or the exact required version).

[pgstef]

The repo-get command is there exactly for that purpose. If you want to do that by hand yourself, then you can learn from the source-code how to do it. Here's the code of the repo-get command: https://github.com/pgbackrest/pgbackrest/blob/main/src/command/repo/get.c#L59

Regards,

[dwsteele]

You can do this with the following command:

openssl enc -d -aes-256-cbc -md sha1 -k <key> -in archive.info

Note that if you want to decrypt and use your backups manually you'll need to use only full backups (or use repo-hardlink and repo-type=posix for diff/incr) and you must not enable repo-bundle or repo-block.

There may be other options that I'm not thinking of that would prevent you from manually restoring a backup. You'll need to test your method thoroughly to be sure it works.

[timka]

@dwsteele That openssl line didn't work for me. I must be missing something.
@pgstef Before opening this issue I tired figuring this out from the source. At that point I gained the understanding that there's a generalized approach involving IO filters for encryption and compression. But sorting out the lower level would require some more time to grow my synapses and I decided to ask for help in the meantime. Seems like the next step is understanding how a key is derived from the passphrase.

[dwsteele]

That openssl line didn't work for me. I must be missing something.

You didn't give us the settings you are using for backup, or the command you are trying to run, or the error -- so not a lot we can do to help.

[timka]

That openssl line didn't work for me. I must be missing something.

You didn't give us the settings you are using for backup, or the command you are trying to run, or the error -- so not a lot we can do to help.

Sorry for not being explicit enough. I've only set cipher-type=aes-256-cbc and cipher-pass is some random base64-encoded string. I know sha1 is the default digest algo and haven't set that explicitly. So I thought my error has something to do with how passphrase/key is passed. The openssl-enc (1.1.1f) man page says:

      -k password
           The password to derive the key from. This is for compatibility with previous versions of OpenSSL. Superseded by the -pass argument.
     -kfile filename
           Read the password to derive the key from the first line of filename.  This is for compatibility with previous versions of OpenSSL. Superseded by the -pass argument.

…

       -K key
           The actual key to use: this must be represented as a string comprised only of hex digits. If only the key is specified, the IV must additionally specified using the -iv option. When
           both a key and a password are specified, the key given with the -K option will be used and the IV generated from the password will be taken. It does not make much sense to specify both
           key and password.

       -iv IV
           The actual IV to use: this must be represented as a string comprised only of hex digits. When only the key is specified using the -K option, the IV must explicitly be defined. When a
           password is being specified using one of the other options, the IV is generated from this password.

So I assumed key is derived in pgbackrest since the encryption filter besides the mode (enc/dec) accepts only 2 args. That's how I came up with the line in the issue description where I used the -kfile option. Which I tried with two different passphrase files: one with exact same base64-encoded string from my pgbackrest.conf and a binary one with that string decoded.

[dwsteele]

That's how I came up with the line in the issue description where I used the -kfile option. Which I tried with two different passphrase files: one with exact same base64-encoded string from my pgbackrest.conf and a binary one with that string decoded.

The key to pass to -k or -kfile is the same value you provide to cipher-pass.

I know sha1 is the default digest algo and haven't set that explicitly

You must set it explicitly.

[timka]

I know sha1 is the default digest algo and haven't set that explicitly

You must set it explicitly.

I mean it's the default in pgbackrest.conf and yes, I pass -md has1 to openssl enc as you wrote earlier. So namely this line:
openssl enc -d -aes-256-cbc -md sha1 -kfile openssl-pass -in backup_label.zst results in 800B7B92C47F0000:error:1C800064:Provider routines:ossl_cipher_unpadblock:bad decrypt:../providers/implementations/ciphers/ciphercommon_block.c:124:. It does produce some output which is broken and cannot be decompressed by zstd.

I think I'll abandon this approach. Maybe after all it's easier and cleaner to ensure the pgbackrest binary is around anywhere I may need it.

Thanks for your help!

[dwsteele]

800B7B92C47F0000:error:1C800064:Provider routines:ossl_cipher_unpadblock:bad decrypt:../providers/implementations/ciphers/ciphercommon_block.c:124

Looks to me like something is missing from your build of openssl.

I think I'll abandon this approach. Maybe after all it's easier and cleaner to ensure the pgbackrest binary is around anywhere I may need it.

That will be quite a bit easier -- and safer.