Add example profile 3 canary workload services projects around bigquery development

[obriensystems]

Update: investigate owner role on project or superfolder and folderIAMAdmin
Update: lay out prod/non-prod folder structure

follow document ai canary at https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit/blob/canary/solutions/document-processing/gcloud/deployment.sh#L88

https://cloud.google.com/bigquery/docs/access-control#bq-permissions
https://cloud.google.com/bigquery/docs/access-control#bigquery

training
https://www.cloudskillsboost.google/focuses/1145?parent=catalog

Roles

gcloud projects add-iam-policy-binding $CC_PROJECT_ID  --member=sgroup:$EMAIL --role=roles/bigquery.admin --quiet > /dev/null 1>&1

Services

Big Query is usually enabled by default on a project - but the lz workflow has it unset - fixing

root_@cloudshell:~/lz-tls/_lz2/pbmm-on-gcp-onboarding (lz-tls)$ gcloud config set project tzpe-tlz-tlz-de
Updated property [core/project].
root_@cloudshell:~/lz-tls/_lz2/pbmm-on-gcp-onboarding (tzpe-tlz-tlz-de)$ gcloud services list | grep NAME
NAME: accesscontextmanager.googleapis.com
NAME: appengine.googleapis.com
NAME: artifactregistry.googleapis.com
NAME: cloudbilling.googleapis.com
NAME: cloudbuild.googleapis.com
NAME: cloudidentity.googleapis.com
NAME: cloudkms.googleapis.com
NAME: cloudresourcemanager.googleapis.com
NAME: compute.googleapis.com
NAME: containerregistry.googleapis.com
NAME: iam.googleapis.com
NAME: iamcredentials.googleapis.com
NAME: logging.googleapis.com
NAME: oslogin.googleapis.com
NAME: pubsub.googleapis.com
NAME: secretmanager.googleapis.com
NAME: serviceusage.googleapis.com
NAME: sourcerepo.googleapis.com
NAME: storage-api.googleapis.com

add

bigquery.googleapis.com
bigquerymigration.googleapis.com
bigquerystorage.googleapis.com

gcloud services enable bigquerymigration.googleapis.com
gcloud services enable bigquery.googleapis.com
gcloud services enable bigquerystorage.googleapis.com

[obriensystems]

before - no access as expected

after

in prod

Step #3 - "tf plan": Terraform will perform the following actions:
Step #3 - "tf plan": 
Step #3 - "tf plan":   # module.prod-client-prj.google_project_service.project_services["bigquery.googleapis.com"] will be created
Step #3 - "tf plan":   + resource "google_project_service" "project_services" {
Step #3 - "tf plan":       + disable_dependent_services = true
Step #3 - "tf plan":       + disable_on_destroy         = true
Step #3 - "tf plan":       + id                         = (known after apply)
Step #3 - "tf plan":       + project                    = "tzpe-tlz-prd-client8"
Step #3 - "tf plan":       + service                    = "bigquery.googleapis.com"
Step #3 - "tf plan":     }
Step #3 - "tf plan": 
Step #3 - "tf plan":   # module.prod-client-prj.google_project_service.project_services["bigquerymigration.googleapis.com"] will be created
Step #3 - "tf plan":   + resource "google_project_service" "project_services" {
Step #3 - "tf plan":       + disable_dependent_services = true
Step #3 - "tf plan":       + disable_on_destroy         = true
Step #3 - "tf plan":       + id                         = (known after apply)
Step #3 - "tf plan":       + project                    = "tzpe-tlz-prd-client8"
Step #3 - "tf plan":       + service                    = "bigquerymigration.googleapis.com"
Step #3 - "tf plan":     }
Step #3 - "tf plan": 
Step #3 - "tf plan":   # module.prod-client-prj.google_project_service.project_services["bigquerystorage.googleapis.com"] will be created
Step #3 - "tf plan":   + resource "google_project_service" "project_services" {
Step #3 - "tf plan":       + disable_dependent_services = true
Step #3 - "tf plan":       + disable_on_destroy         = true
Step #3 - "tf plan":       + id                         = (known after apply)
Step #3 - "tf plan":       + project                    = "tzpe-tlz-prd-client8"
Step #3 - "tf plan":       + service                    = "bigquerystorage.googleapis.com"


and in common
Step #3 - "tf plan":   # module.iam-groups-role_opsadmin.module.organization_iam["group:opsadmin@terraform.landing.systems"].google_organization_iam_member.organization["roles/bigquery.admin"] will be created
Step #3 - "tf plan":   + resource "google_organization_iam_member" "organization" {
Step #3 - "tf plan":       + etag   = (known after apply)
Step #3 - "tf plan":       + id     = (known after apply)
Step #3 - "tf plan":       + member = "group:opsadmin@terraform.landing.systems"
Step #3 - "tf plan":       + org_id = "131880894992"
Step #3 - "tf plan":       + role   = "roles/bigquery.admin"
Step #3 - "tf plan":     }
Step #3 - "tf plan": 
Step #3 - "tf plan":   # module.iam-groups-role_read.module.organization_iam["group:read@terraform.landing.systems"].google_organization_iam_member.organization["roles/bigquery.admin"] will be created
Step #3 - "tf plan":   + resource "google_organization_iam_member" "organization" {
Step #3 - "tf plan":       + etag   = (known after apply)
Step #3 - "tf plan":       + id     = (known after apply)
Step #3 - "tf plan":       + member = "group:read@terraform.landing.systems"
Step #3 - "tf plan":       + org_id = "131880894992"
Step #3 - "tf plan":       + role   = "roles/bigquery.admin"
Step #3 - "tf plan":     }

[obriensystems]

check limited access

getting iam admin via group role

although only the terraform service account can modify group membership

removing opsadmin group role

diff --git a/environments/common/iam-groups.auto.tfvars b/environments/common/iam-groups.auto.tfvars
index 019403f..dd6fdab 100644
--- a/environments/common/iam-groups.auto.tfvars
+++ b/environments/common/iam-groups.auto.tfvars
@@ -32,7 +32,7 @@ iam-group_opsadmin = {
   domain       = "terraform.landing.systems"
   #owners       = ["ro..ding.systems"]#, "tfsa..tlz-tlz-de.iam.gserviceaccount.com"] # var.service_accounts
   #managers     = ["roo..m.landing.systems"]
-  members      = ["root..stems", "devel...landing.systems"]
+  members      = ["roo..landing.systems"]

test results in #279

membership renewal in this case can be done without tearing down the group first - additive not authorative

Step #3 - "tf plan": Terraform will perform the following actions:
Step #3 - "tf plan": 
Step #3 - "tf plan":   # module.group_opsadmin.google_cloud_identity_group_membership.members["developer@terraform.landing.systems"] will be destroyed
Step #3 - "tf plan":   - resource "google_cloud_identity_group_membership" "members" {
Step #3 - "tf plan":       - create_time = "2023-08-02T17:07:46.973190Z" -> null
Step #3 - "tf plan":       - group       = "groups/00gjdgxs0p378zo" -> null
Step #3 - "tf plan":       - id          = "groups/00gjdgxs0p378zo/memberships/118409452485276160936" -> null
Step #3 - "tf plan":       - name        = "groups/00gjdgxs0p378zo/memberships/118409452485276160936" -> null
Step #3 - "tf plan":       - type        = "USER" -> null
Step #3 - "tf plan":       - update_time = "2023-08-02T17:07:46.973190Z" -> null
Step #3 - "tf plan": 
Step #3 - "tf plan":       - member_key {
Step #3 - "tf plan":           - id = "developer@terraform.landing.systems" -> null
Step #3 - "tf plan":         }
Step #3 - "tf plan": 
Step #3 - "tf plan":       - preferred_member_key {
Step #3 - "tf plan":           - id = "developer@terraform.landing.systems" -> null
Step #3 - "tf plan":         }
Step #3 - "tf plan": 
Step #3 - "tf plan":       - roles {
Step #3 - "tf plan":           - name = "MEMBER" -> null
Step #3 - "tf plan":         }
Step #3 - "tf plan":     }

[obriensystems]

verified lower iam permissions on dev user

big query ok on the org - reduce to the project
projects are limited

[fmichaelobrien]

20240406: Closing issue during retrofit/rebase of this TEF V1 based/modified repo to TEF V4 standards
This issue may participate in the LZ refactor after rebase
Query on all issues related to the older V1 version via the tag
2024-pre-tef-v4