Use credentials file passed through config (vmware-tanzu#69)

* Use credentials file passed through config

Add a new supported key to the config which allows the path to a
credentials file to be passed through to the ObjectStore plugin.
If that key is set in the config, use that file as the credentials
file for the AWS client.

This can be tested by adding a new key to the `cloud-credentials` secret
and adding the key value pair `credentialsFile=/credentials/<key_name>`.

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>

* Add function to create session options

We have some repeated logic for creating session option. This introduces
a new function to handle the creation of those session options.

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>

* Add documentation for per-BSL credentials

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>

* Address code review and add changelog

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>

README.md

@@ -34,7 +34,8 @@ To set up Velero on AWS, you:
 * [Create an S3 bucket][1]
 * [Set permissions for Velero][2]
 * [Install and start Velero][3]
-* [Migrating PVs across clusters][5]
+
+You can also use this plugin to [migrate PVs across clusters][5] or create an additional [Backup Storage Location][12].
 
 If you do not have the `aws` CLI locally installed, follow the [user guide][6] to set it up.
 
@@ -289,6 +290,52 @@ Additionally, you can specify `--use-restic` to enable restic support, and `--wa
 
 For more complex installation needs, use either the Helm chart, or add `--dry-run -o yaml` options for generating the YAML representation for the installation.
 
+## Create an additional Backup Storage Location
+
+If you are using Velero v1.6.0 or later, you can create additional AWS [Backup Storage Locations][13] that use their own credentials.
+These can also be created alongside Backup Storage Locations that use other providers.
+
+### Limitations
+It is not possible to use different credentials for additional Backup Storage Locations if you are pod based authentication such as [kube2iam][14].
+
+### Prerequisites
+
+* Velero 1.6.0 or later
+* AWS plugin must be installed, either at install time, or by running `velero plugin install velero/velero-plugin-for-aws:v1.2.0`
+
+### Configure S3 bucket and credentials
+
+To configure a new Backup Storage Location with its own credentials, it is necessary to follow the steps above to [create the bucket to use][15] and to [generate the credentials file][16] to interact with that bucket.
+Once you have created the credentials file, create a [Kubernetes Secret][17] in the Velero namespace that contains these credentials:
+
+```bash
+kubectl create secret generic -n velero bsl-credentials --from-file=aws=</path/to/credentialsfile>
+```
+
+This will create a secret named `bsl-credentials` with a single key (`aws`) which contains the contents of your credentials file.
+The name and key of this secret will be given to Velero when creating the Backup Storage Location, so it knows which secret data to use.
+
+### Create Backup Storage Location
+
+Once the bucket and credentials have been configured, these can be used to create the new Backup Storage Location:
+
+```bash
+velero backup-location create <bsl-name> \
+  --provider aws \
+  --bucket $BUCKET \
+  --config region=$REGION \
+  --credential=bsl-credentials=aws
+```
+
+The Backup Storage Location is ready to use when it has the phase `Available`.
+You can check this with the following command:
+
+```bash
+velero backup-location get
+```
+
+To use this new Backup Storage Location when performing a backup, use the flag `--storage-location <bsl-name>` when running `velero backup create`.
+
 ## Migrating PVs across clusters
 
 ### Setting AWS_CLUSTER_NAME (Optional)
@@ -330,6 +377,12 @@ Copy one of the returned IDs `<ID>` and use it with the `aws` CLI tool to search
 [9]: https://velero.io/docs/customize-installation/
 [10]: http://docs.aws.amazon.com/IAM/latest/UserGuide/introduction.html
 [11]: https://velero.io/docs/faq/
+[12]: #Create-an-additional-Backup-Storage-Location
+[13]: https://velero.io/docs/latest/api-types/backupstoragelocation/
+[14]: #option-2-set-permissions-using-kube2iam
+[15]: #create-s3-bucket
+[16]: #option-1-set-permissions-with-an-iam-user
+[17]: https://kubernetes.io/docs/concepts/configuration/secret/
 [101]: https://github.com/vmware-tanzu/velero-plugin-for-aws/workflows/Main%20CI/badge.svg
 [102]: https://github.com/vmware-tanzu/velero-plugin-for-aws/actions?query=workflow%3A"Main+CI"
 [103]: https://github.com/vmware-tanzu/velero/issues/new/choose

changelogs/unreleased/69-zubron

@@ -0,0 +1 @@
+Add support for the new `credentialsFile` config key which enables per-BSL credentials. If set, the plugin will use this path as the credentials file for authentication rather than the credentials file path in the environment.

velero-plugin-for-aws/object_store.go

@@ -1,5 +1,5 @@
 /*
-Copyright 2017, 2019 the Velero contributors.
+Copyright the Velero contributors.
 
 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -20,6 +20,7 @@ import (
 	"crypto/tls"
 	"io"
 	"net/http"
+	"os"
 	"sort"
 	"strconv"
 	"strings"
@@ -46,6 +47,7 @@ const (
 	s3ForcePathStyleKey      = "s3ForcePathStyle"
 	bucketKey                = "bucket"
 	signatureVersionKey      = "signatureVersion"
+	credentialsFileKey       = "credentialsFile"
 	credentialProfileKey     = "profile"
 	serverSideEncryptionKey  = "serverSideEncryption"
 	insecureSkipTLSVerifyKey = "insecureSkipTLSVerify"
@@ -90,6 +92,7 @@ func (o *ObjectStore) Init(config map[string]string) error {
 		kmsKeyIDKey,
 		s3ForcePathStyleKey,
 		signatureVersionKey,
+		credentialsFileKey,
 		credentialProfileKey,
 		serverSideEncryptionKey,
 		insecureSkipTLSVerifyKey,
@@ -105,6 +108,7 @@ func (o *ObjectStore) Init(config map[string]string) error {
 		s3ForcePathStyleVal      = config[s3ForcePathStyleKey]
 		signatureVersion         = config[signatureVersionKey]
 		credentialProfile        = config[credentialProfileKey]
+		credentialsFile          = config[credentialsFileKey]
 		serverSideEncryption     = config[serverSideEncryptionKey]
 		insecureSkipTLSVerifyVal = config[insecureSkipTLSVerifyKey]
 
@@ -166,10 +170,11 @@ func (o *ObjectStore) Init(config map[string]string) error {
 		}
 	}
 
-	sessionOptions := session.Options{Config: *serverConfig, Profile: credentialProfile}
-	if len(caCert) > 0 {
-		sessionOptions.CustomCABundle = strings.NewReader(caCert)
+	sessionOptions, err := newSessionOptions(*serverConfig, credentialProfile, caCert, credentialsFile)
+	if err != nil {
+		return err
 	}
+
 	serverSession, err := getSession(sessionOptions)
 	if err != nil {
 		return err
@@ -192,11 +197,13 @@ func (o *ObjectStore) Init(config map[string]string) error {
 		if err != nil {
 			return err
 		}
-		sessionOptions := session.Options{Config: *publicConfig, Profile: credentialProfile}
-		if len(caCert) > 0 {
-			sessionOptions.CustomCABundle = strings.NewReader(caCert)
+
+		publicSessionOptions, err := newSessionOptions(*publicConfig, credentialProfile, caCert, credentialsFile)
+		if err != nil {
+			return err
 		}
-		publicSession, err := getSession(sessionOptions)
+
+		publicSession, err := getSession(publicSessionOptions)
 		if err != nil {
 			return err
 		}
@@ -208,6 +215,29 @@ func (o *ObjectStore) Init(config map[string]string) error {
 	return nil
 }
 
+// newSessionOptions creates a session.Options with the given config and profile. If
+// caCert and credentialsFile are provided, these will be used for the CustomCABundle
+// and the credentials for the session.
+func newSessionOptions(config aws.Config, profile string, caCert string, credentialsFile string) (session.Options, error) {
+	sessionOptions := session.Options{Config: config, Profile: profile}
+
+	if caCert != "" {
+		sessionOptions.CustomCABundle = strings.NewReader(caCert)
+	}
+
+	if credentialsFile != "" {
+		if _, err := os.Stat(credentialsFile); err != nil {
+			if os.IsNotExist(err) {
+				return session.Options{}, errors.Wrapf(err, "provided credentialsFile does not exist")
+			}
+			return session.Options{}, errors.Wrapf(err, "could not get credentialsFile info")
+		}
+		sessionOptions.SharedConfigFiles = []string{credentialsFile}
+	}
+
+	return sessionOptions, nil
+}
+
 func newAWSConfig(url, region string, forcePathStyle bool) (*aws.Config, error) {
 	awsConfig := aws.NewConfig().
 		WithRegion(region).