Read GCP key content from environment?

[reidab]

The GCP KMS implementation uses Google's Application Default Credentials scheme, which requires the path to a key file to be passed set as GOOGLE_APPLICATION_CREDENTIALS.

We've run into a few places in our build process where we'd like to use sops with the GCP KMS, but where it's not feasible for us to write credentials to a file beforehand. Would it make sense for sops to provide an environment variable to accept Google credentials as content instead of as a path?

The Google API client offers an initialization method using credentials content: https://pkg.go.dev/google.golang.org/api/option?tab=doc#WithCredentialsJSON

[autrilla]

This functionality should probably be added to the GCP SDK instead. Regardless, how are you able to get the sops file into the build environment, but not a GCP credentials file?

…

On Tue, 16 Jun 2020 at 07:48, Reid Beels ***@***.***> wrote: The GCP KMS implementation uses Google's Application Default Credentials scheme, which requires the path to a key file to be passed set as GOOGLE_APPLICATION_CREDENTIALS. We've run into a few places in our build process where we'd like to use sops with the GCP KMS, but where it's not feasible for us to write credentials to a file beforehand. Would it make sense for sops to provide an environment variable to accept Google credentials as content instead of as a path? The Google API client offers an initialization method using credentials content: https://pkg.go.dev/google.golang.org/api/option?tab=doc#WithCredentialsJSON — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#681>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AARH4V7K4T3FBOEFV5BKCPDRW4BUFANCNFSM4N7I2AOA> .

[reidab]

It's definitely something that should be added to the GCP SDK, but every discussion I've ever seen about it has ended with "we implement Application Default Credentials, that's just how it works". 😞

The specific use case that I was trying to make work is:

• We're using Terraform to manage infrastructure and the hosted Terraform Cloud environment to apply changes.

• We have existing secrets managed in sops files that I'd like to read as part of the Terraform run. These sops files would be checked in to git alongside the terraform config.

• https://github.com/carlpett/terraform-provider-sops uses sops as a library to decrypt sops files and expose them to Terraform.

• In the context of the Terraform Cloud runner, I can set environment vars and terraform input vars, but I can't execute arbitrary setup code (e.g. to write the contents of an environment variable to a file).

• I'd like to patch terraform-provider-sops so that it can accept credentials directly in a terraform input var and pass them on to sops. I don't think I can do that without a change to sops to add an option that will initialize a Google client with explicit params instead of the defaults here:

  https://github.com/mozilla/sops/blob/4bc27f6eb72b1b4090753e9f3dba1d094544e1c3/gcpkms/keysource.go#L134

[farant]

I might be wrong but my understanding is that GOOGLE_APPLICATION_CREDENTIALS can either be a path to the json credentials or the actual json credential contents?

Could be worth testing if that would help your use case.

[reidab]

The Application Default Credentials spec says it's a file path, and that's how the Go library implements it:

https://github.com/golang/oauth2/blob/5d25da1a8d43b66f2898c444f899c7bcfd6a407e/google/default.go#L78-L85

The Google Auth clients for some other languages implement a way to pass credentials directly as a different environment variable, but there's no equivalent in Go.

The official recommendation from the Go OAuth2 team is to use CredentialsFromJSON to construct a client if you want to pass JSON directly, which would have to be implemented as part of sops.

[Morriz]

Since a gcloud update we now see that GOOGLE_APPLICATION_CREDENTIALS is used for kubectl access, which stopped working as it points to a service account that only has decrypt perms.

Blocker

[Morriz]

If we need to combine perms in one account we can (even tho its a pita), but is that documented?

[shyukri]

Any progress on this ?