Improve the velero backup-location command

[carlisia]

• --cacert [set|create] feat: support set BackupStorageLocation(BSL) CA certificate #3167
• --default [set|create|get] feat: support configure BSL CR to indicate which one is the default #3092
• --credentials [set|create] Add credential field to the bsl #3190

[nrb]

@carlisia Can you detail what improvements are required, as someone else is taking up this work?

[carlisia]

Yes, so, the specs for all of the new commands are linked in the epic (#2419). I didn't want to copy/paste that into every single one.

[jenting]

• --credentials [set|create] WIP

For the credentials part, I think it related to how Velero enhances the multiple credential secrets #2403. So, it'd be great to implement it along with the multiple credential secrets enhancement.

[carlisia]

@jenting absolutely. I have started looking into this as part of reviewing the multiple secrets design. I am running some tests so I can understand what is possible. For example, it seems that it is possible to have multiple entries for the same provider, example (in the Velero deployment):

            - name: AWS_SHARED_CREDENTIALS_FILE
              value: /credentials/cloud4
            - name: AWS_SHARED_CREDENTIALS_FILE
              value: /credentials/cloud5

*It seems to me that as long as either cloud4 OR cloud5 contains a valid credentials, the plugin will work. I want to run more tests to confirm this conclusively. If this is true, then my question is: does the BSL need to specify which credential? Seems that only configuring the BSL for the provider would be enough, meaning, no change needed to the BSL. Unless, we want a BSL to ONLY work with credential X (valid), but not with credentialY (also valid).

I don't fully understand what needs to be done yet but yes, they are coupled.

[carlisia]

@jenting let me know if you have any idea how to handle this or if you want to work on implementing this change.

[jenting]

🤔 I read the CLI change design doc https://github.com/vmware-tanzu/velero/blob/master/design/cli-install-changes.md and I found that there is another command to configure the plugin credential velero plugin set --credentials-file. I don't know what's the difference against velero backup-location set --credentials because currently, I don't know will the plugins other than AWS/GCP/Azure/CSI have to configure credentials? If not, then I don't know why we have to keep the velero plugin set --credentials-file.

Anyway, from our use case, we have a scenario that at on-premise K8s cluster, we'd like to have 2 BSLs: one back up to local Minio server; the other one back up to AWS S3. Then, each BSL would have a different credential but the problem is how to enhance the Velero server could differentiate passing different credential files with the same key AWS_SHARED_CREDENTIALS_FILE?

[carlisia]

I understand your questions. I'm running some experiments to help figure out both what we want to do and what we can do.

The idea with the velero plugin set --credentials-file is this cmd will add 1.* credentials for a given plugin. Right now the plugins already use the credential. We are aiming to not have to modify the plugins to do this change. The velero backup-location set --credentials would specify which credential the BSL would need.