Encrypt GCS Remote State

[negz]

Terraform Version

$ terraform version
Terraform v0.11.0

References

• The recently updated GCS backend: GCloud backend #15592

Feature Request

Currently all files in GCS are transparently encrypted at rest using Google's keys, and IAM/ACL can be used to restrict access to GCS buckets. Allowing state files in GCS to be encrypted with a custom key would provide an additional layer of defense around sensitive Terraform state.

https://cloud.google.com/storage/docs/encryption#customer-supplied
It's possible in GCS to supply custom encryption keys. Objects created with a customer supplied encryption key can only be read when the key used for creation is passed in with the read request.

https://godoc.org/cloud.google.com/go/storage#ObjectHandle.Key
https://github.com/hashicorp/terraform/blob/c054bd0/backend/remote-state/gcs/client.go#L154
The cloud.google.com/go/storage client Terraform uses supports customer supplied encryption keys, and the design of the GCS backend makes it relatively straightforward to pass in a key.

I considered Google KMS for this purpose, but I don't believe it would work because KMS does not expose the actual key data that we would need to pass to GCS. We could forego customer supplied encryption keys altogether and simply encrypt our state files by calling out to the Google KMS API, but as best I can tell KMS can only encrypt data that is 64kB or smaller when base64 encoded.

Rather than encrypt the actual state data using a KMS key, we may be able to have the GCS backend use KMS to decrypt a GCS compatible key at runtime, then use said GCS compatible key to encrypt and decrypt state in GCS. This would require the user to create a GCS compatible key and encrypt it using KMS out of band.

A GCS backend configured to use encryption would look something like:

backend "gcs" {
  project = "my-project"
  bucket  = "terraform-state"
  prefix  = "terraform/state/"

  kms_keyring_id        = "mykeyring"
  kms_key_id            = "mykey"
  kms_encrypted_gcs_key = "kmsencryptedkey"
}

Does this seem like a worthwhile feature and a reasonable approach?

[apparentlymart]

Hi @negz,

Thanks for the detailed description of the options here! I must admit to not being incredibly familiar with GCS or Google KMS but having read through the docs you linked I think I got enough to understand the tradeoffs of the different approaches you discussed here.

I'm a little nervous about the complexity of using KMS to decrypt a key that is then separately used to decrypt the state. While it is nice that this effectively avoids the need for the "real" key to be available locally when Terraform is run, in all other cases we expect credentials and secrets for the backend to be provided directly, and expect the user to run Terraform in a secure environment (e.g. in an automation system) in order to avoid key sprawl.

Therefore I'd lean towards starting with a simpler option of providing the "customer-supplied encryption key" directly via an environment variable, similarly to the auth credentials. This is easier for a user to reason about, requires less setup, and any effort the user takes to protect credentials and other secrets would apply to this too.

The approach of unwrapping a secret using KMS could then, in principle, be implemented by a separate utility that accesses the KMS API before running Terraform. I'm not opposed to this being integrated into Terraform in the long run, but I feel we should start with something more straightforward and see what patterns emerge around it before adding the extra complexity.

Does that seem reasonable?

I'm also curious as to whether @danawillow has any thoughts here, since she'll have more familiarity with these APIs and features and the tradeoffs between these options.

[negz]

@apparentlymart I'd be happy with your suggested simpler approach. Also happy to work on this if it's deemed to be useful functionality.

[danawillow]

(just getting to this now, was OOO the last two weeks)
I pretty much agree with both of you on this. Just having an encryption option would be fantastic, and I think it does make sense to start off by letting the user specify the key directly. In time if it seems like a desirable feature, we could add the KMS option alongside.

[negz]

Implemented in #16936

[alwaysastudent]

@danawillow @negz - The KMS option would be ideal since the GCS now supports the Customer Managed Keys. https://cloud.google.com/storage/docs/encryption/customer-managed-keys

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.