gcs

Google Cloud Storage Module

• Simple bucket example
• Cloud KMS
• Retention policy, soft delete policy and logging
• Lifecycle rule
• GCS notifications
• Object upload
• IAM
• Tags
• Variables
• Outputs

Simple bucket example

module "bucket" {
  source     = "./fabric/modules/gcs"
  project_id = var.project_id
  prefix     = var.prefix
  name       = "my-bucket"
  versioning = true
  labels = {
    cost-center = "devops"
  }
}
# tftest modules=1 resources=1 inventory=simple.yaml e2e

Cloud KMS

module "project" {
  source         = "./fabric/modules/project"
  name           = var.project_id
  project_create = false
}

module "kms" {
  source     = "./fabric/modules/kms"
  project_id = var.project_id
  keyring = {
    location = "europe" # location of the KMS must match location of the bucket
    name     = "test"
  }
  keys = {
    bucket_key = {
      iam_bindings = {
        bucket_key_iam = {
          members = ["serviceAccount:${module.project.service_accounts.robots.storage}"]
          role    = "roles/cloudkms.cryptoKeyEncrypterDecrypter"
        }
      }
    }
  }
}

module "bucket" {
  source         = "./fabric/modules/gcs"
  project_id     = var.project_id
  prefix         = var.prefix
  name           = "my-bucket"
  encryption_key = module.kms.keys.bucket_key.id
  location       = "EU"
}

# tftest modules=3 skip e2e

Retention policy, soft delete policy and logging

module "bucket" {
  source     = "./fabric/modules/gcs"
  project_id = var.project_id
  prefix     = var.prefix
  name       = "my-bucket"
  retention_policy = {
    retention_period = 100
    is_locked        = true
  }
  soft_delete_retention = 7776000
  logging_config = {
    log_bucket        = "log-bucket"
    log_object_prefix = null
  }
}
# tftest modules=1 resources=1 inventory=retention-logging.yaml

Lifecycle rule

module "bucket" {
  source     = "./fabric/modules/gcs"
  project_id = var.project_id
  prefix     = var.prefix
  name       = "my-bucket"
  lifecycle_rules = {
    lr-0 = {
      action = {
        type          = "SetStorageClass"
        storage_class = "STANDARD"
      }
      condition = {
        age = 30
      }
    }
  }
}
# tftest modules=1 resources=1 inventory=lifecycle.yaml e2e

GCS notifications

module "project" {
  source         = "./fabric/modules/project"
  name           = var.project_id
  project_create = false
}

module "bucket-gcs-notification" {
  source     = "./fabric/modules/gcs"
  project_id = var.project_id
  prefix     = var.prefix
  name       = "my-bucket"
  notification_config = {
    enabled           = true
    payload_format    = "JSON_API_V1"
    sa_email          = module.project.service_accounts.robots.storage
    topic_name        = "gcs-notification-topic"
    event_types       = ["OBJECT_FINALIZE"]
    custom_attributes = {}
  }
}
# tftest skip e2e

Object upload

module "bucket" {
  source     = "./fabric/modules/gcs"
  project_id = var.project_id
  prefix     = var.prefix
  name       = "my-bucket"
  objects_to_upload = {
    sample-data = {
      name         = "example-file.csv"
      source       = "assets/example-file.csv"
      content_type = "text/csv"
    }
  }
}
# tftest modules=1 resources=2 inventory=object-upload.yaml e2e

IAM

IAM is managed via several variables that implement different features and levels of control:

• iam and iam_by_principals configure authoritative bindings that manage individual roles exclusively, and are internally merged
• iam_bindings configure authoritative bindings with optional support for conditions, and are not internally merged with the previous two variables
• iam_bindings_additive configure additive bindings via individual role/member pairs with optional support conditions

The authoritative and additive approaches can be used together, provided different roles are managed by each. Some care must also be taken with the iam_by_principals variable to ensure that variable keys are static values, so that Terraform is able to compute the dependency graph.

Refer to the project module for examples of the IAM interface.

module "bucket" {
  source     = "./fabric/modules/gcs"
  project_id = var.project_id
  prefix     = var.prefix
  name       = "my-bucket"
  iam = {
    "roles/storage.admin" = ["group:${var.group_email}"]
  }
}
# tftest modules=1 resources=2 inventory=iam-authoritative.yaml e2e

module "bucket" {
  source     = "./fabric/modules/gcs"
  project_id = var.project_id
  prefix     = var.prefix
  name       = "my-bucket"
  iam_bindings = {
    storage-admin-with-delegated_roles = {
      role    = "roles/storage.admin"
      members = ["group:${var.group_email}"]
      condition = {
        title = "delegated-role-grants"
        expression = format(
          "api.getAttribute('iam.googleapis.com/modifiedGrantsByRole', []).hasOnly([%s])",
          join(",", formatlist("'%s'",
            [
              "roles/storage.objectAdmin",
              "roles/storage.objectViewer",
            ]
          ))
        )
      }
    }
  }
}
# tftest modules=1 resources=2 inventory=iam-bindings.yaml e2e

module "bucket" {
  source     = "./fabric/modules/gcs"
  project_id = var.project_id
  prefix     = var.prefix
  name       = "my-bucket"
  iam_bindings_additive = {
    storage-admin-with-delegated_roles = {
      role   = "roles/storage.admin"
      member = "group:${var.group_email}"
      condition = {
        title = "delegated-role-grants"
        expression = format(
          "api.getAttribute('iam.googleapis.com/modifiedGrantsByRole', []).hasOnly([%s])",
          join(",", formatlist("'%s'",
            [
              "roles/storage.objectAdmin",
              "roles/storage.objectViewer",
            ]
          ))
        )
      }
    }
  }
}
# tftest modules=1 resources=2 inventory=iam-bindings-additive.yaml e2e

Tags

Refer to the Creating and managing tags documentation for details on usage.

module "org" {
  source          = "./fabric/modules/organization"
  organization_id = var.organization_id
  tags = {
    environment = {
      description = "Environment specification."
      values = {
        dev     = {}
        prod    = {}
        sandbox = {}
      }
    }
  }
}

module "bucket" {
  source     = "./fabric/modules/gcs"
  project_id = var.project_id
  prefix     = var.prefix
  name       = "my-bucket"
  tag_bindings = {
    env-sandbox = module.org.tag_values["environment/sandbox"].id
  }
}
# tftest modules=2 resources=6

Variables

name	description	type	required	default
name	Bucket name suffix.	string	✓
project_id	Bucket project id.	string	✓
autoclass	Enable autoclass to automatically transition objects to appropriate storage classes based on their access pattern. If set to true, storage_class must be set to STANDARD. Defaults to false.	bool		false
cors	CORS configuration for the bucket. Defaults to null.	object({…})		null
custom_placement_config	The bucket's custom location configuration, which specifies the individual regions that comprise a dual-region bucket. If the bucket is designated as REGIONAL or MULTI_REGIONAL, the parameters are empty.	list(string)		null
default_event_based_hold	Enable event based hold to new objects added to specific bucket, defaults to false.	bool		null
encryption_key	KMS key that will be used for encryption.	string		null
force_destroy	Optional map to set force destroy keyed by name, defaults to false.	bool		false
iam	IAM bindings in {ROLE => [MEMBERS]} format.	map(list(string))		{}
iam_bindings	Authoritative IAM bindings in {KEY => {role = ROLE, members = [], condition = {}}}. Keys are arbitrary.	map(object({…}))		{}
iam_bindings_additive	Individual additive IAM bindings. Keys are arbitrary.	map(object({…}))		{}
labels	Labels to be attached to all buckets.	map(string)		{}
lifecycle_rules	Bucket lifecycle rule.	map(object({…}))		{}
location	Bucket location.	string		"EU"
logging_config	Bucket logging configuration.	object({…})		null
notification_config	GCS Notification configuration.	object({…})		null
objects_to_upload	Objects to be uploaded to bucket.	map(object({…}))		{}
prefix	Optional prefix used to generate the bucket name.	string		null
public_access_prevention	Prevents public access to a bucket. Acceptable values are inherited or enforced. If inherited, the bucket uses public access prevention, only if the bucket is subject to the public access prevention organization policy constraint.	string		null
requester_pays	Enables Requester Pays on a storage bucket.	bool		null
retention_policy	Bucket retention policy.	object({…})		null
soft_delete_retention	The duration in seconds that soft-deleted objects in the bucket will be retained and cannot be permanently deleted. Set to 0 to override the default and disable.	number		null
storage_class	Bucket storage class.	string		"MULTI_REGIONAL"
tag_bindings	Tag bindings for this folder, in key => tag value id format.	map(string)		null
uniform_bucket_level_access	Allow using object ACLs (false) or not (true, this is the recommended behavior) , defaults to true (which is the recommended practice, but not the behavior of storage API).	bool		true
versioning	Enable versioning, defaults to false.	bool		false
website	Bucket website.	object({…})		null

Outputs

name	description	sensitive
bucket	Bucket resource.
id	Fully qualified bucket id.
name	Bucket name.
notification	GCS Notification self link.
objects	Objects in GCS bucket.
topic	Topic ID used by GCS.
url	Bucket URL.