Skip to content

Commit

Permalink
Add bootstrap command
Browse files Browse the repository at this point in the history 
This command creates the GCS bucket and KMS keys automatically with the
properly scoped minimal permissions instead of relying on gcloud/gsutil.
This makes for an easier getting started experience, especially for
users who may be unfamiliar with the platform.
  • Loading branch information
@sethvargo
sethvargo committed Apr 23, 2019
1 parent c7b44d7 commit 533bf49
Show file tree
Hide file tree
Showing 6 changed files with 406 additions and 81 deletions.
126 changes: 47 additions & 79 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,11 @@ stored in [Cloud Storage][cloud-storage].
## Setup

1. Install the [Cloud SDK][cloud-sdk] for your operating system. Alternatively,
you can run these commands from [Cloud Shell][cloud-shell], which has the SDK
and other popular tools pre-installed.
you can run these commands from [Cloud Shell][cloud-shell], which has the SDK
and other popular tools pre-installed.

If you are running from your local machine, you also need Default Application Credentials:
If you are running from your local machine, you also need Default
Application Credentials:

```text
gcloud auth application-default login
Expand Down Expand Up @@ -63,7 +64,7 @@ and other popular tools pre-installed.
```

1. Export your project ID as an environment variable. The rest of this setup
guide assumes this environment variable is set:
guide assumes this environment variable is set:

```text
export PROJECT_ID=my-gcp-project-id
Expand All @@ -73,106 +74,72 @@ guide assumes this environment variable is set:
_number_. You can find the project ID by running `gcloud projects list` or
in the web UI.

1. Enable required services on the project:
1. Export your desired Cloud Storage bucket name. The rest of this setup guide
assumes this environment variable is set:

```text
gcloud services enable --project ${PROJECT_ID} \
compute.googleapis.com \
cloudkms.googleapis.com \
storage-api.googleapis.com \
storage-component.googleapis.com
```

1. Create a [Cloud KMS][cloud-kms] keyring and crypto key for encrypting
secrets:

```text
gcloud kms keyrings create my-keyring \
--project ${PROJECT_ID} \
--location global
```

```text
gcloud kms keys create my-key \
--project ${PROJECT_ID} \
--location global \
--keyring my-keyring \
--purpose encryption
```

1. Create a [Cloud Storage][cloud-storage] bucket for storing secrets:

```text
export BUCKET_ID=my-secrets
```

Replace `my-secrets` with the name of your bucket. Bucket names must be
globally unique across all of Google Cloud. You can also create a bucket
using the Google Cloud Console from the web.
Replace `my-secrets` with the name of your bucket. **This bucket should not
exist yet!**

```text
gsutil mb -p ${PROJECT_ID} gs://${BUCKET_ID}
```
1. Bootstrap a Berglas environment. This will create a new Cloud Storage bucket
for storing secrets and a Cloud KMS key for encrypting data.

**It is strongly recommended that you create a new bucket instead of using
an existing one. Berglas should be the only entity managing IAM permissions
on the bucket.**

1. Set the default ACL permissions on the bucket to private:

```text
gsutil defacl set private gs://${BUCKET_ID}
berglas bootstrap --project $PROJECT_ID --bucket $BUCKET_ID
```

```text
gsutil acl set private gs://${BUCKET_ID}
```
This command uses the default values. You can customize the storage bucket
and KMS key configuration using the optional flags. Run `berglas bootstrap
-h` for more details.

The default permissions grant anyone with Owner/Editor access on the project
access to the bucket and its objects. These commands restrict access to the
bucket to the bucket creator (you). Everyone else must be granted explicit
access via IAM to the bucket or an object inside the bucket.
If you want full control over the creation of the Cloud Storage and Cloud
KMS keys, please see the [custom setup documentation][custom-setup].

1. _(Optional)_ Enable [Cloud Audit logging][cloud-audit] on the bucket:

Please note this will enable audit logging on all Cloud KMS keys and all
Cloud Storage buckets in the project, which may incur costs.
Cloud Storage buckets in the project, which may incur additional costs.

Download the exiting project IAM policy:
1. Download the exiting project IAM policy:

```text
gcloud projects get-iam-policy ${PROJECT_ID} > policy.yaml
```
```text
gcloud projects get-iam-policy ${PROJECT_ID} > policy.yaml
```

Add Cloud Audit logging for Cloud KMS and Cloud Storage:
1. Add Cloud Audit logging for Cloud KMS and Cloud Storage:

```text
cat <<EOF >> policy.yaml
auditConfigs:
- auditLogConfigs:
- logType: DATA_READ
- logType: ADMIN_READ
- logType: DATA_WRITE
service: cloudkms.googleapis.com
- auditLogConfigs:
- logType: ADMIN_READ
- logType: DATA_READ
- logType: DATA_WRITE
service: storage.googleapis.com
EOF
```
```text
cat <<EOF >> policy.yaml
auditConfigs:
- auditLogConfigs:
- logType: DATA_READ
- logType: ADMIN_READ
- logType: DATA_WRITE
service: cloudkms.googleapis.com
- auditLogConfigs:
- logType: ADMIN_READ
- logType: DATA_READ
- logType: DATA_WRITE
service: storage.googleapis.com
EOF
```

Submit the new policy:
1. Submit the new policy:

```text
gcloud projects set-iam-policy ${PROJECT_ID} policy.yaml
```
```text
gcloud projects set-iam-policy ${PROJECT_ID} policy.yaml
```

Remove the updated policy from local disk:
1. Remove the updated policy from local disk:

```text
rm policy.yaml
```
```text
rm policy.yaml
```


## CLI Usage
Expand Down Expand Up @@ -494,6 +461,7 @@ This library is licensed under Apache 2.0. Full license text is available in
[k8s-mutating]: https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/
[go-crypto]: https://golang.org/pkg/crypto/
[envelope-encryption]: https://cloud.google.com/kms/docs/envelope-encryption
[custom-setup]: https://github.com/GoogleCloudPlatform/berglas/blob/master/doc/custom-setup.md
[reference-syntax]: https://github.com/GoogleCloudPlatform/berglas/blob/master/doc/reference-syntax.md
[threat-model]: https://github.com/GoogleCloudPlatform/berglas/blob/master/doc/threat-model.md
[releases]: https://github.com/GoogleCloudPlatform/berglas/releases
Expand Down
83 changes: 83 additions & 0 deletions doc/custom-setup.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
# Berglas Custom Setup

This document describes the steps required to create a Berglas Cloud Storage
bucket and Cloud KMS keys manually. **This is an advanced user topic and is not
required to use Berglas. Users should use the `berglas boostrap` command where
possible!**

1. Install the [Cloud SDK][cloud-sdk]. More detailed instructions are available
in the main README.

1. Export your project ID as an environment variable. The rest of this setup
guide assumes this environment variable is set:

```text
export PROJECT_ID=my-gcp-project-id
```

Please note, this is the project _ID_, not the project _name_ or project
_number_. You can find the project ID by running `gcloud projects list` or
in the web UI.

1. Enable required services on the project:

```text
gcloud services enable --project ${PROJECT_ID} \
cloudkms.googleapis.com \
storage-api.googleapis.com \
storage-component.googleapis.com
```

1. Create a [Cloud KMS][cloud-kms] keyring and crypto key for encrypting
secrets:

```text
gcloud kms keyrings create my-keyring \
--project ${PROJECT_ID} \
--location global
```

```text
gcloud kms keys create my-key \
--project ${PROJECT_ID} \
--location global \
--keyring my-keyring \
--purpose encryption
```

You can choose alternate locations and names, but the `purpose` must remain
as "encryption".

1. Create a [Cloud Storage][cloud-storage] bucket for storing secrets:

```text
export BUCKET_ID=my-secrets
```

Replace `my-secrets` with the name of your bucket. Bucket names must be
globally unique across all of Google Cloud. You can also create a bucket
using the Google Cloud Console from the web.

```text
gsutil mb -p ${PROJECT_ID} gs://${BUCKET_ID}
```

**It is strongly recommended that you create a new bucket instead of using
an existing one. Berglas should be the only entity managing IAM permissions
on the bucket.**

1. Set the default ACL permissions on the bucket to private:

```text
gsutil defacl set private gs://${BUCKET_ID}
```

```text
gsutil acl set private gs://${BUCKET_ID}
```

The default permissions grant anyone with Owner/Editor access on the project
access to the bucket and its objects. These commands restrict access to the
bucket to project owners and access to bucket objects to only their owner.
Everyone else must be granted explicit access via IAM to an object inside
the bucket.
4 changes: 2 additions & 2 deletions go.mod
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ go 1.12

require (
cloud.google.com/go v0.37.4
github.com/golang/protobuf v1.3.1 // indirect
github.com/golang/protobuf v1.3.1
github.com/hashicorp/golang-lru v0.5.1 // indirect
github.com/inconshreveable/mousetrap v1.0.0 // indirect
github.com/kr/pretty v0.1.0 // indirect
Expand All @@ -20,6 +20,6 @@ require (
google.golang.org/api v0.3.2
google.golang.org/appengine v1.5.0 // indirect
google.golang.org/genproto v0.0.0-20190418145605-e7d98fc518a7
google.golang.org/grpc v1.20.0 // indirect
google.golang.org/grpc v1.20.0
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 // indirect
)
83 changes: 83 additions & 0 deletions main.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,13 @@ var (
execLocal bool

members []string

projectID string
bucket string
bucketLocation string
kmsLocation string
kmsKeyRing string
kmsCryptoKey string
)

var rootCmd = &cobra.Command{
Expand Down Expand Up @@ -81,6 +88,32 @@ characters.
Run: accessRun,
}

var bootstrapCmd = &cobra.Command{
Use: "bootstrap",
Short: "Bootstrap a berglas environment",
Long: strings.Trim(`
Bootstrap a Berglas environment by creating a Cloud Storage bucket and a Cloud
KMS key with properly scoped permissions to the caller.
This command will create a new Cloud Storage bucket with "private" ACLs and
grant permission only to the caller in the specified project. It will enable
versioning on the bucket, configured to retain the last 10 verions. If the
bucket already exists, an error is returned.
This command will also create a Cloud KMS key ring and crypto key in the
specified project. If the key ring or crypto key already exist, no errors are
returned.
`, "\n"),
Example: strings.Trim(`
# Bootstrap a berglas environment
berglas bootstrap my-secrets/api-key \
--project my-project \
--bucket my-bucket
`, "\n"),
Args: cobra.ExactArgs(0),
Run: bootstrapRun,
}

var createCmd = &cobra.Command{
Use: "create [secret] [data]",
Short: "Create or overwrite a secret",
Expand Down Expand Up @@ -256,6 +289,22 @@ Show berglas version.
func main() {
rootCmd.AddCommand(accessCmd)

rootCmd.AddCommand(bootstrapCmd)
bootstrapCmd.Flags().StringVarP(&projectID, "project", "p", "",
"Google Cloud Project ID")
createCmd.MarkFlagRequired("project")
bootstrapCmd.Flags().StringVarP(&bucket, "bucket", "b", "",
"Name of the Cloud Storage bucket to create")
createCmd.MarkFlagRequired("bucket")
bootstrapCmd.Flags().StringVarP(&bucketLocation, "bucket-location", "l", "US",
"Location in which to create Cloud Storage bucket")
bootstrapCmd.Flags().StringVarP(&kmsLocation, "kms-location", "m", "global",
"Location in which to create the Cloud KMS key ring")
bootstrapCmd.Flags().StringVarP(&kmsKeyRing, "kms-keyring", "r", "berglas",
"Name of the KMS key ring to create")
bootstrapCmd.Flags().StringVarP(&kmsCryptoKey, "kms-key", "k", "berglas-key",
"Name of the KMS key to create")

rootCmd.AddCommand(createCmd)
createCmd.Flags().StringVarP(&key, "key", "k", "",
"KMS key to use for encryption")
Expand Down Expand Up @@ -300,6 +349,40 @@ func accessRun(_ *cobra.Command, args []string) {
fmt.Fprintf(stdout, "%s", plaintext)
}

func bootstrapRun(_ *cobra.Command, args []string) {
ctx := cliCtx()
if err := berglas.Bootstrap(ctx, &berglas.BootstrapRequest{
ProjectID: projectID,
Bucket: bucket,
BucketLocation: bucketLocation,
KMSLocation: kmsLocation,
KMSKeyRing: kmsKeyRing,
KMSCryptoKey: kmsCryptoKey,
}); err != nil {
handleError(err, 1)
}

kmsKeyID := fmt.Sprintf("projects/%s/locations/%s/keyRings/%s/cryptoKeys/%s",
projectID, kmsLocation, kmsKeyRing, kmsCryptoKey)

fmt.Fprintf(stdout, "Successfully created berglas environment:\n")
fmt.Fprintf(stdout, "\n")
fmt.Fprintf(stdout, " Bucket: %s\n", bucket)
fmt.Fprintf(stdout, " KMS key: %s\n", kmsKeyID)
fmt.Fprintf(stdout, "\n")
fmt.Fprintf(stdout, "To create a secret:\n")
fmt.Fprintf(stdout, "\n")
fmt.Fprintf(stdout, " berglas create %s/my-secret abcd1234 \\\n", bucket)
fmt.Fprintf(stdout, " --key %s\n", kmsKeyID)
fmt.Fprintf(stdout, "\n")
fmt.Fprintf(stdout, "To grant access to that secret:\n")
fmt.Fprintf(stdout, "\n")
fmt.Fprintf(stdout, " berglas grant %s/my-secret \\\n", bucket)
fmt.Fprintf(stdout, " --member user:jane.doe@mycompany.com\n")
fmt.Fprintf(stdout, "\n")
fmt.Fprintf(stdout, "For more help and examples, please run \"berglas -h\".\n")
}

func createRun(_ *cobra.Command, args []string) {
bucket, object, err := parseRef(args[0])
if err != nil {
Expand Down
Loading

0 comments on commit 533bf49

Please sign in to comment.