Enable users to set restic repo passwords

[skriss]

Currently, Ark uses a static encryption password for restic repositories it creates. Ark assumes that the data is protected at rest by IAM policies on the object storage bucket where the data is stored.

Some users would like to be able to explicitly set an encryption password for these Ark-managed restic repositories. This issue serves as a placeholder for discussion on the whys/hows/whens.

[dharmab]

Really looking forward to this. IAM policies be changed, and backups can be downloaded to operator devices where they are vulnerable to theft or malware. This would add another layer of security.

[sh0rez]

Ark is already creating a secret containing the restic key (ark-restic-credentials, value is static-passw0rd). I believe it would be a good first step to allow users to supply their own secret instead so the key would at least differ across individual ark install. And if the user does not supply one then maybe randomize the key and put it in the secret .. This is still better than using the same password for all users everywhere!

[ncdc]

@rosskukulinski might want to consider for 1.0?

[sh0rez]

@ncdc if the next version of ark would silently start encrypting with randomized tokens stored in a secret, allow users to provide their own secrets and continue to run with the old static secret if already existent this change would not be breaking so there is no need for a major release, or do I miss something?

[rosskukulinski]

@ncdc I would be supportive of having this part of the workstream, but I do wonder if it requires a breaking change. 1.x might be more reasonable from a timeline perspective?

[ac-hibbert]

Hi there, we've taken a second looks at the code and worked out that we can create our own restic repo passwords using the secret "velero-restic-credentials" and data "repository-password" and when it initializes the repo it will use this. In addition to this it can also be changed if you move sideways the restic folder in s3 and delete restic repositories found in resticrepositories.velero.io. You can still get at the old restic repository using the old password and native restic command. It would be good if this was included in docs as according to the code it will only use the static password if this secret isn't created, this could be useful for people.

As we want to be able to able to run restic to restore single files sideways (see #1210) we would need to be able to find out the password in order to do this

[ThoTischner]

We also create the secret on our ci/cd via helm before the velero helm chart will be applied.
But this should be just a workaround, one local helm chart with just one resource (the secret) does not make much sense.

[jeanlucmongrain]

Why not add a secret key to ResticRepository CRD?

[varac]

Please prioritize this, users should have the ability to use proper and safe restic encryption.

[Ahmadre]

@carlisia @jenting could someone plrease respond? This is repeatedly requested. Thanks guys.

[maximilize]

For a full solution, aside from restic encryption, also the manifests should be encrypted on the client side. This would require the secret key stored a more central place than in ResticRepository. This feature would be really nice to have, else people who are concerned about security would always need self host S3 servers to store their backups.

[dsu-igeek]

We could add an optional EncryptionCredential to the Backup Storage Location. I'd prefer not to do anything that is Restic only as we're going to change the architecture there soon.

[Ahmadre]

We could add an optional EncryptionCredential to the Backup Storage Location. I'd prefer not to do anything that is Restic only as we're going to change the architecture there soon.

Definetly not! I also wouldn't recommend that. I reached out a clean solution like @ThoTischner mentioned. Just set your restic-credentials before deploying your velero-chart. works perfectly 👌

[eleanor-millman]

Closing, work around stated in comments and we can address it more generally if there is enough demand.

[tlvenn]

Hi @eleanor-millman, how do you gauge enough demand ? I would say that this feature is highly requested and for anyone using the restic backend, you definitely want to provide them with a secure solution outside of the box.

The workaround is totally impracticable in a Infra as code environment which is the norm these days and it seems it would be fairly easy to address this issue unless there is a technical hurdle I am not seeing.

So please can you reconsider and prioritize this issue that has been waiting around for 3years now ?

Many thanks.

[varac]

I second @tlvenn, if there would be an easy workaround within a pure infrastructure as code deployment I'd be fine. But the limitation that you first need to set the restic encr. password before you apply the velero helm chart i.e. is not applicable easily in this scenario. So please consider supporting custom encryption passwords out of the box, which shouldn't be too much work.

[eleanor-millman]

@tlvenn @varac Apologies for the delay - I didn't have notifications correctly set on Github.

Agreed that the workaround isn't as good as having the feature implemented. Unfortunately, the Velero team is stretched way too thin to implement most of the feature requests that come in. Our issue backlog had grown to 411 issues as of last week and we felt that that wasn't great for the community, both for requesters like you who have your requests sitting in limbo and for Velero developers who have this mountain of issues to stare at. We are currently triaging them into issues we hope to tackle in 1.7/1.8 (i.e. in 2021), issues we hope to tackle in 2.0 (sometime in 2022 and perhaps 2023) and then the rest. We are closing the rest for now, since we feel like issues that won't be tackled for 2 years are not helped by sitting open, in limbo. While not a perfect decision mechanism, one of the things that lead us to prioritize some issues over others was if a workaround existed.

That being said, this triage is regarding work that the maintainers and active contributors will be doing. If you or anyone else who commented on this issue are able to make a PR, we are delighted to chat with you about possible design of the feature and are trying to prioritize PR review (something that we were falling behind on with too much on our plates). So please let us know if this is something that interests you and if you want a suggestion on how to implement this.

[JustinGuese]

+1 from me, would be an essential feature for us EU people

[gaffneyd4]

Hi there, we've taken a second looks at the code and worked out that we can create our own restic repo passwords using the secret "velero-restic-credentials" and data "repository-password" and when it initializes the repo it will use this. In addition to this it can also be changed if you move sideways the restic folder in s3 and delete restic repositories found in resticrepositories.velero.io. You can still get at the old restic repository using the old password and native restic command. It would be good if this was included in docs as according to the code it will only use the static password if this secret isn't created, this could be useful for people.

As we want to be able to able to run restic to restore single files sideways (see #1210) we would need to be able to find out the password in order to do this

It looks like the current version of the code (v1.10 at time of writing) uses the secret name velero-repo-credentials now, with the same key repository-password.

To configure velero with a custom password for restic repositories, I run the following command before I run velero install... on a new cluster:

kubectl create namespace velero
kubectl create secret generic velero-repo-credentials \
    --namespace velero \
    --from-file=repository-password=./secret.txt

If you have installed velero already and have any restic repositories created, you will have to:

• Delete your backups that use restic (velero backup delete)
• Delete the repositories (kubectl delete BackupRespository)
• Delete the restic data in your storage backend (in S3 this meant I deleted the s3://my-bucket/restic folder)
• Replace the current velero-repo-credentials secret with your new one
• Roll the velero and node-agent pods so that the scratch directory cache is updated with the new credentials.
• Recreate your backups

Now your backups should be accessible using the plain restic tool and your new password: restic -r s3:s3.amazonaws.com/my-bucket/restic/my-backup snapshots.