Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for subnet-level service network user grants to project module, improve docs #1907

Merged
merged 4 commits into from
Dec 7, 2023
Merged

Add support for subnet-level service network user grants to project module, improve docs #1907

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
f2f67bb
improve project factory example
ludoo Dec 7, 2023
3b7f32c
light refactor of project modules shared vpc internals and docs
ludoo Dec 7, 2023
0e6f24f
add support for subnet-level grants on host project
ludoo Dec 7, 2023
9851a53
add support for subnet-level IAM to project factory
ludoo Dec 7, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
19 changes: 14 additions & 5 deletions blueprints/factories/project-factory/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ module "project-factory" {
# location where the yaml files are read from
factory_data_path = "data"
}
# tftest modules=7 resources=26 files=prj-app-1,prj-app-2,prj-app-3 inventory=example.yaml
# tftest modules=7 resources=31 files=prj-app-1,prj-app-2,prj-app-3 inventory=example.yaml
```

```yaml
Expand Down Expand Up @@ -92,10 +92,19 @@ service_accounts:
app-2-be: {}
services:
- compute.googleapis.com
- container.googleapis.com
- run.googleapis.com
- storage.googleapis.com
shared_vpc_service_config:
host_project: foo-host
service_identity_iam:
"roles/compute.networkUser":
- cloudservices
- container-engine
"roles/vpcaccess.user":
- cloudrun
"roles/container.hostServiceAgentUser":
- container-engine

# tftest-file id=prj-app-2 path=data/prj-app-2.yaml
```
Expand All @@ -113,10 +122,10 @@ services:

| name | description | type | required | default |
|---|---|:---:|:---:|:---:|
| [factory_data_path](variables.tf#L88) | Path to folder with YAML project description data files. | <code>string</code> | ✓ | |
| [data_defaults](variables.tf#L17) | Optional default values used when corresponding project data from files are missing. | <code title="object&#40;&#123;&#10; billing_account &#61; optional&#40;string&#41;&#10; contacts &#61; optional&#40;map&#40;list&#40;string&#41;&#41;, &#123;&#125;&#41;&#10; labels &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10; metric_scopes &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; parent &#61; optional&#40;string&#41;&#10; prefix &#61; optional&#40;string&#41;&#10; service_encryption_key_ids &#61; optional&#40;map&#40;list&#40;string&#41;&#41;, &#123;&#125;&#41;&#10; service_perimeter_bridges &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; service_perimeter_standard &#61; optional&#40;string&#41;&#10; services &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; shared_vpc_service_config &#61; optional&#40;object&#40;&#123;&#10; host_project &#61; string&#10; service_identity_iam &#61; optional&#40;map&#40;list&#40;string&#41;&#41;, &#123;&#125;&#41;&#10; service_iam_grants &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; &#125;&#41;, &#123; host_project &#61; null &#125;&#41;&#10; tag_bindings &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10; service_accounts &#61; optional&#40;map&#40;object&#40;&#123;&#10; display_name &#61; optional&#40;string, &#34;Terraform-managed.&#34;&#41;&#10; iam_project_roles &#61; optional&#40;list&#40;string&#41;&#41;&#10; &#125;&#41;&#41;, &#123;&#125;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>&#123;&#125;</code> |
| [data_merges](variables.tf#L46) | Optional values that will be merged with corresponding data from files. Combines with `data_defaults`, file data, and `data_overrides`. | <code title="object&#40;&#123;&#10; contacts &#61; optional&#40;map&#40;list&#40;string&#41;&#41;, &#123;&#125;&#41;&#10; labels &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10; metric_scopes &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; service_encryption_key_ids &#61; optional&#40;map&#40;list&#40;string&#41;&#41;, &#123;&#125;&#41;&#10; service_perimeter_bridges &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; services &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; tag_bindings &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10; service_accounts &#61; optional&#40;map&#40;object&#40;&#123;&#10; display_name &#61; optional&#40;string, &#34;Terraform-managed.&#34;&#41;&#10; iam_project_roles &#61; optional&#40;list&#40;string&#41;&#41;&#10; &#125;&#41;&#41;, &#123;&#125;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>&#123;&#125;</code> |
| [data_overrides](variables.tf#L66) | Optional values that override corresponding data from files. Takes precedence over file data and `data_defaults`. | <code title="object&#40;&#123;&#10; billing_account &#61; optional&#40;string&#41;&#10; contacts &#61; optional&#40;map&#40;list&#40;string&#41;&#41;&#41;&#10; parent &#61; optional&#40;string&#41;&#10; prefix &#61; optional&#40;string&#41;&#10; service_encryption_key_ids &#61; optional&#40;map&#40;list&#40;string&#41;&#41;&#41;&#10; service_perimeter_bridges &#61; optional&#40;list&#40;string&#41;&#41;&#10; service_perimeter_standard &#61; optional&#40;string&#41;&#10; tag_bindings &#61; optional&#40;map&#40;string&#41;&#41;&#10; services &#61; optional&#40;list&#40;string&#41;&#41;&#10; service_accounts &#61; optional&#40;map&#40;object&#40;&#123;&#10; display_name &#61; optional&#40;string, &#34;Terraform-managed.&#34;&#41;&#10; iam_project_roles &#61; optional&#40;list&#40;string&#41;&#41;&#10; &#125;&#41;&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>&#123;&#125;</code> |
| [factory_data_path](variables.tf#L89) | Path to folder with YAML project description data files. | <code>string</code> | ✓ | |
| [data_defaults](variables.tf#L17) | Optional default values used when corresponding project data from files are missing. | <code title="object&#40;&#123;&#10; billing_account &#61; optional&#40;string&#41;&#10; contacts &#61; optional&#40;map&#40;list&#40;string&#41;&#41;, &#123;&#125;&#41;&#10; labels &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10; metric_scopes &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; parent &#61; optional&#40;string&#41;&#10; prefix &#61; optional&#40;string&#41;&#10; service_encryption_key_ids &#61; optional&#40;map&#40;list&#40;string&#41;&#41;, &#123;&#125;&#41;&#10; service_perimeter_bridges &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; service_perimeter_standard &#61; optional&#40;string&#41;&#10; services &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; shared_vpc_service_config &#61; optional&#40;object&#40;&#123;&#10; host_project &#61; string&#10; service_identity_iam &#61; optional&#40;map&#40;list&#40;string&#41;&#41;, &#123;&#125;&#41;&#10; service_identity_subnet_iam &#61; optional&#40;map&#40;list&#40;string&#41;&#41;, &#123;&#125;&#41;&#10; service_iam_grants &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; &#125;&#41;, &#123; host_project &#61; null &#125;&#41;&#10; tag_bindings &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10; service_accounts &#61; optional&#40;map&#40;object&#40;&#123;&#10; display_name &#61; optional&#40;string, &#34;Terraform-managed.&#34;&#41;&#10; iam_project_roles &#61; optional&#40;list&#40;string&#41;&#41;&#10; &#125;&#41;&#41;, &#123;&#125;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>&#123;&#125;</code> |
| [data_merges](variables.tf#L47) | Optional values that will be merged with corresponding data from files. Combines with `data_defaults`, file data, and `data_overrides`. | <code title="object&#40;&#123;&#10; contacts &#61; optional&#40;map&#40;list&#40;string&#41;&#41;, &#123;&#125;&#41;&#10; labels &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10; metric_scopes &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; service_encryption_key_ids &#61; optional&#40;map&#40;list&#40;string&#41;&#41;, &#123;&#125;&#41;&#10; service_perimeter_bridges &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; services &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; tag_bindings &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10; service_accounts &#61; optional&#40;map&#40;object&#40;&#123;&#10; display_name &#61; optional&#40;string, &#34;Terraform-managed.&#34;&#41;&#10; iam_project_roles &#61; optional&#40;list&#40;string&#41;&#41;&#10; &#125;&#41;&#41;, &#123;&#125;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>&#123;&#125;</code> |
| [data_overrides](variables.tf#L67) | Optional values that override corresponding data from files. Takes precedence over file data and `data_defaults`. | <code title="object&#40;&#123;&#10; billing_account &#61; optional&#40;string&#41;&#10; contacts &#61; optional&#40;map&#40;list&#40;string&#41;&#41;&#41;&#10; parent &#61; optional&#40;string&#41;&#10; prefix &#61; optional&#40;string&#41;&#10; service_encryption_key_ids &#61; optional&#40;map&#40;list&#40;string&#41;&#41;&#41;&#10; service_perimeter_bridges &#61; optional&#40;list&#40;string&#41;&#41;&#10; service_perimeter_standard &#61; optional&#40;string&#41;&#10; tag_bindings &#61; optional&#40;map&#40;string&#41;&#41;&#10; services &#61; optional&#40;list&#40;string&#41;&#41;&#10; service_accounts &#61; optional&#40;map&#40;object&#40;&#123;&#10; display_name &#61; optional&#40;string, &#34;Terraform-managed.&#34;&#41;&#10; iam_project_roles &#61; optional&#40;list&#40;string&#41;&#41;&#10; &#125;&#41;&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>&#123;&#125;</code> |

## Outputs

Expand Down
6 changes: 5 additions & 1 deletion blueprints/factories/project-factory/factory.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,11 @@ locals {
shared_vpc_service_config = (
try(v.shared_vpc_service_config, null) != null
? merge(
{ service_identity_iam = {}, service_iam_grants = [] },
{
service_identity_iam = {}
service_identity_subnet_iam = {}
service_iam_grants = []
},
v.shared_vpc_service_config
)
: var.data_defaults.shared_vpc_service_config
Expand Down
7 changes: 4 additions & 3 deletions blueprints/factories/project-factory/variables.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,10 @@ variable "data_defaults" {
service_perimeter_standard = optional(string)
services = optional(list(string), [])
shared_vpc_service_config = optional(object({
host_project = string
service_identity_iam = optional(map(list(string)), {})
service_iam_grants = optional(list(string), [])
host_project = string
service_identity_iam = optional(map(list(string)), {})
service_identity_subnet_iam = optional(map(list(string)), {})
service_iam_grants = optional(list(string), [])
}), { host_project = null })
tag_bindings = optional(map(string), {})
# non-project resources
Expand Down
65 changes: 55 additions & 10 deletions modules/project/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -228,9 +228,21 @@ This table lists all affected services and roles that you need to grant to servi

## Shared VPC

The module allows managing Shared VPC status for both hosts and service projects, and includes a simple way of assigning Shared VPC roles to service identities.
The module allows managing Shared VPC status for both hosts and service projects, and control of IAM bindings for API service identities.

You can enable Shared VPC Host at the project level and manage project service association independently.
Project service association for VPC host projects can be

- autoritatively managed in the host project by enabling Shared VPC and specifying the set of service projects, or
- additively managed in service projects by by enabling Shared VPC in the host project and then "attaching" each service project independently

IAM bindings in the host project for API service identities can be managed from service projects in two different ways:

- via the `service_identity_iam` attribute, by specifying the set of roles and service agents
- via the `service_iam_grants` attribute that leverages a [fixed list of roles for each service](./sharedvpc-agent-iam.yaml), by specifying a list of services
wiktorn marked this conversation as resolved.
Show resolved Hide resolved

While the first method is more explicit and readable, the second method is simpler and less error prone as all appropriate roles are predefined for all required service agents (eg compute and cloud services). You can mix and match as the two sets of bindings are then internally combined.

This example shows a simple configuration with a host project, and a service project independently attached with granular IAM bindings for service identities.

```hcl
module "host-project" {
Expand Down Expand Up @@ -272,7 +284,7 @@ module "service-project" {
# tftest modules=2 resources=10 inventory=shared-vpc.yaml e2e
```

The module allows also granting necessary permissions in host project to service identities by specifying which services will be used in service project in `grant_iam_for_services`.
This example shows a similar configuration, with the simpler way of defining IAM bindings for service identities. The list of services passed to `service_iam_grants` uses the same module's outputs to establish a dependency, as service identities are only typically available after service (API) activation.

```hcl
module "host-project" {
Expand All @@ -296,13 +308,47 @@ module "service-project" {
"container.googleapis.com",
]
shared_vpc_service_config = {
host_project = module.host-project.project_id
host_project = module.host-project.project_id
# reuse the list of services from the module's outputs
service_iam_grants = module.service-project.services
}
}
# tftest modules=2 resources=9 inventory=shared-vpc-auto-grants.yaml e2e
```

In specific cases it might make sense to selectively grant the `compute.networkUser` role for service identities at the subnet level, and while that is best done via org policies it's also supported by this module.
wiktorn marked this conversation as resolved.
Show resolved Hide resolved

```hcl
module "host-project" {
source = "./fabric/modules/project"
billing_account = var.billing_account_id
name = "host"
parent = var.folder_id
prefix = var.prefix
shared_vpc_host_config = {
enabled = true
}
}

module "service-project" {
source = "./fabric/modules/project"
billing_account = var.billing_account_id
name = "service"
parent = var.folder_id
prefix = var.prefix
services = [
"compute.googleapis.com",
]
shared_vpc_service_config = {
host_project = module.host-project.project_id
service_identity_subnet_iam = {
"europe-west1/gce" = ["compute"]
}
}
}
# tftest modules=2 resources=6 inventory=shared-vpc-subnet-grants.yaml
```

## Organization Policies

To manage organization policies, the `orgpolicy.googleapis.com` service should be enabled in the quota project.
Expand Down Expand Up @@ -617,7 +663,7 @@ output "compute_robot" {

### Managing project related configuration without creating it

The module offers managing all related resources without ever touching the project itself by using `project_create = false`
The module offers managing all related resources without ever touching the project itself by using `project_create = false`

```hcl
module "create-project" {
Expand Down Expand Up @@ -827,7 +873,6 @@ module "bucket" {
# tftest modules=7 resources=53 inventory=data.yaml e2e
```


<!-- TFDOC OPTS files:1 -->
<!-- BEGIN TFDOC -->
## Files
Expand All @@ -840,7 +885,7 @@ module "bucket" {
| [organization-policies.tf](./organization-policies.tf) | Project-level organization policies. | <code>google_org_policy_policy</code> |
| [outputs.tf](./outputs.tf) | Module outputs. | |
| [service-accounts.tf](./service-accounts.tf) | Service identities and supporting resources. | <code>google_kms_crypto_key_iam_member</code> · <code>google_project_default_service_accounts</code> · <code>google_project_iam_member</code> · <code>google_project_service_identity</code> |
| [shared-vpc.tf](./shared-vpc.tf) | Shared VPC project-level configuration. | <code>google_compute_shared_vpc_host_project</code> · <code>google_compute_shared_vpc_service_project</code> · <code>google_project_iam_member</code> |
| [shared-vpc.tf](./shared-vpc.tf) | Shared VPC project-level configuration. | <code>google_compute_shared_vpc_host_project</code> · <code>google_compute_shared_vpc_service_project</code> · <code>google_compute_subnetwork_iam_member</code> · <code>google_project_iam_member</code> |
| [tags.tf](./tags.tf) | None | <code>google_tags_tag_binding</code> |
| [variables.tf](./variables.tf) | Module variables. | |
| [versions.tf](./versions.tf) | Version pins. | |
Expand Down Expand Up @@ -879,9 +924,9 @@ module "bucket" {
| [service_perimeter_standard](variables.tf#L276) | Name of VPC-SC Standard perimeter to add project into. See comment in the variables file for format. | <code>string</code> | | <code>null</code> |
| [services](variables.tf#L282) | Service APIs to enable. | <code>list&#40;string&#41;</code> | | <code>&#91;&#93;</code> |
| [shared_vpc_host_config](variables.tf#L288) | Configures this project as a Shared VPC host project (mutually exclusive with shared_vpc_service_project). | <code title="object&#40;&#123;&#10; enabled &#61; bool&#10; service_projects &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
| [shared_vpc_service_config](variables.tf#L297) | Configures this project as a Shared VPC service project (mutually exclusive with shared_vpc_host_config). | <code title="object&#40;&#123;&#10; host_project &#61; string&#10; service_identity_iam &#61; optional&#40;map&#40;list&#40;string&#41;&#41;, &#123;&#125;&#41;&#10; service_iam_grants &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code title="&#123;&#10; host_project &#61; null&#10;&#125;">&#123;&#8230;&#125;</code> |
| [skip_delete](variables.tf#L319) | Allows the underlying resources to be destroyed without destroying the project itself. | <code>bool</code> | | <code>false</code> |
| [tag_bindings](variables.tf#L325) | Tag bindings for this project, in key => tag value id format. | <code>map&#40;string&#41;</code> | | <code>null</code> |
| [shared_vpc_service_config](variables.tf#L297) | Configures this project as a Shared VPC service project (mutually exclusive with shared_vpc_host_config). | <code title="object&#40;&#123;&#10; host_project &#61; string&#10; service_identity_iam &#61; optional&#40;map&#40;list&#40;string&#41;&#41;, &#123;&#125;&#41;&#10; service_identity_subnet_iam &#61; optional&#40;map&#40;list&#40;string&#41;&#41;, &#123;&#125;&#41;&#10; service_iam_grants &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code title="&#123;&#10; host_project &#61; null&#10;&#125;">&#123;&#8230;&#125;</code> |
| [skip_delete](variables.tf#L320) | Allows the underlying resources to be destroyed without destroying the project itself. | <code>bool</code> | | <code>false</code> |
| [tag_bindings](variables.tf#L326) | Tag bindings for this project, in key => tag value id format. | <code>map&#40;string&#41;</code> | | <code>null</code> |

## Outputs

Expand Down