Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Canary onboarding: SaaS use case: Add KCC/Gcloud deployment options limited roles around BigQuery/AutoML/DocumentAI #220

Open
fmichaelobrien opened this issue Dec 13, 2022 · 7 comments
Assignees

Comments

@fmichaelobrien
Copy link
Contributor

fmichaelobrien commented Dec 13, 2022

Code in the following branch/location
https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit/tree/canary/solutions/document-processing
Readme/Architecture
https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit/tree/canary/solutions/document-processing#readme

Details and automation around onboarding a BigQuery/AutoML/DocumentAI use case for serverless/SaaS

  • manual gcloud for discovery
  • target KCC yaml for on-demand deployment via OCI

Services

IAM roles/permissions

Reference for scripting
https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit/blob/dev/solutions/landing-zone/deployment.sh

Service Account permissions

  • AutoML Editor
  • BigQuery Data Editor
  • BigQuery Job User
  • BigQuery User
  • Document AI Administrator
  • Document AI Editor
  • Storage Admin
  • Vertex AI Custom Code Service Agent

bootstrap

export ORG_ID=$(gcloud projects get-ancestors $BOOT_PROJECT_ID --format='get(id)' | tail -1)
export EMAIL=$(gcloud config list --format json | jq .core.account | sed 's/"//g')

service enablement

  • bigquery already enabled
michael@cloudshell:~ (minimal-dev-od)$ gcloud services list --enabled | grep NAME
NAME: bigquery.googleapis.com
NAME: bigquerymigration.googleapis.com
NAME: bigquerystorage.googleapis.com
NAME: cloudapis.googleapis.com
NAME: clouddebugger.googleapis.com
NAME: cloudtrace.googleapis.com
NAME: datastore.googleapis.com
NAME: logging.googleapis.com
NAME: monitoring.googleapis.com
NAME: servicemanagement.googleapis.com
NAME: serviceusage.googleapis.com
NAME: sql-component.googleapis.com
NAME: storage-api.googleapis.com
NAME: storage-component.googleapis.com
NAME: storage.googleapis.com

gcloud services enable cloudbilling.googleapis.com

predefined roles

gcloud organizations add-iam-policy-binding "${ORG_ID}" --member "user:${EMAIL}" --role roles/aiplatform.admin

Document AI Service Account
https://cloud.google.com/document-ai/docs/setup?continue=https%3A%2F%2Fdevelopers.google.com%2Flearn%2Ftopics%2Fdocumentai&utm_source=developers.google.com&utm_medium=referral

Systems
transport.g.z

@fmichaelobrien fmichaelobrien self-assigned this Dec 13, 2022
fmichaelobrien added a commit that referenced this issue Dec 14, 2022
fmichaelobrien added a commit that referenced this issue Dec 15, 2022
obriensystems added a commit to cloud-quickstart/pubsec-declarative-toolkit that referenced this issue Dec 16, 2022
fmichaelobrien added a commit that referenced this issue Dec 16, 2022
#220 - add iam user role set for project
@fmichaelobrien fmichaelobrien changed the title Canary onboarding: SaaS use case: limited roles around BigQuery/AutoML/DocumentAI Canary onboarding: SaaS use case: Add KCC/Gcloud deployment options limited roles around BigQuery/AutoML/DocumentAI Dec 16, 2022
@obriensystems
Copy link
Collaborator

test results: create and immediate delete -d true

root_@cloudshell:~/_current/pubsec-declarative-toolkit/solutions/document-processing/gcloud (pdt-tgz)$ ./deployment.sh -b pdt-tgz -u pdt3 -c true -l false -d true -e res....zone 
Date: Fri 16 Dec 2022 06:41:02 PM UTC
Timestamp: 1671216062
running with: -b pdt-tgz -u pdt3 -c true -l false -d true -e restricted@transport.gcp.zone -p
Updated property [core/project].
Switched back to boot project pdt-tgz
Start: 1671216062
unique string: pdt3
REGION: northamerica-northeast1
NETWORK: pdt-pdt3-vpc
SUBNET: pdt-pdt3-sn
CLUSTER: pdt-pdt3
Creating project: kcc-lz-6300
CC_PROJECT_ID:
BOOT_PROJECT_ID: pdt-tgz
BILLING_ID: 01...85D
ORG_ID: 442178577666
Creating KCC project: kcc-lz-6300
Create in progress for [https://cloudresourcemanager.googleapis.com/v1/projects/kcc-lz-6300].
Waiting for [operations/cp.9195833045414069903] to finish...done.    
Enabling service [cloudapis.googleapis.com] on project [kcc-lz-6300]...
Operation "operations/acat.p2-1038023658165-1c361f27-31ec-43a1-a765-839f6baee541" finished successfully.
Updated property [core/project] to [kcc-lz-6300].
Updated property [core/project].
billingAccountName: billingAccounts/011...5D
billingEnabled: true
name: projects/kcc-lz-6300/billingInfo
projectId: kcc-lz-6300
Adding roles to project for user: restr...ne
Updated IAM policy for project [kcc-lz-6300].
Updated IAM policy for project [kcc-lz-6300].
Updated IAM policy for project [kcc-lz-6300].
Updated IAM policy for project [kcc-lz-6300].
API's before
NAME: bigquery.googleapis.com
NAME: bigquerymigration.googleapis.com
NAME: bigquerystorage.googleapis.com
NAME: cloudapis.googleapis.com
NAME: clouddebugger.googleapis.com
NAME: cloudtrace.googleapis.com
NAME: datastore.googleapis.com
NAME: logging.googleapis.com
NAME: monitoring.googleapis.com
NAME: servicemanagement.googleapis.com
NAME: serviceusage.googleapis.com
NAME: sql-component.googleapis.com
NAME: storage-api.googleapis.com
NAME: storage-component.googleapis.com
NAME: storage.googleapis.com
Enabling APIs
Operation "operations/acat.p2-1038023658165-8770c1fb-0d3e-4311-8df1-e8051662001e" finished successfully.
Operation "operations/acf.p2-1038023658165-fc1be01a-75cb-4ef3-9846-1dc140fbd9ea" finished successfully.
Operation "operations/acat.p2-1038023658165-c97312fd-28a4-4140-93f9-bad947073aba" finished successfully.
Operation "operations/acf.p2-1038023658165-8966b2f5-ebb7-4c90-85cf-719ff23a1b40" finished successfully.
Operation "operations/acf.p2-1038023658165-99889578-3ff4-444e-ab6c-6fdfcf88cfb9" finished successfully.
Operation "operations/acf.p2-1038023658165-3024943c-e1d6-4409-8a1e-3e8600fd4825" finished successfully.
Operation "operations/acat.p2-1038023658165-e06c74a3-ad6a-495b-821e-b3002919af15" finished successfully.
API's after
NAME: aiplatform.googleapis.com
NAME: autoscaling.googleapis.com
NAME: bigquery.googleapis.com
NAME: bigquerymigration.googleapis.com
NAME: bigquerystorage.googleapis.com
NAME: cloudapis.googleapis.com
NAME: cloudbilling.googleapis.com
NAME: clouddebugger.googleapis.com
NAME: cloudresourcemanager.googleapis.com
NAME: cloudtrace.googleapis.com
NAME: compute.googleapis.com
NAME: container.googleapis.com
NAME: containerfilesystem.googleapis.com
NAME: containerregistry.googleapis.com
NAME: dataflow.googleapis.com
NAME: datastore.googleapis.com
NAME: deploymentmanager.googleapis.com
NAME: documentai.googleapis.com
NAME: healthcare.googleapis.com
NAME: iam.googleapis.com
NAME: iamcredentials.googleapis.com
NAME: logging.googleapis.com
NAME: monitoring.googleapis.com
NAME: notebooks.googleapis.com
NAME: oslogin.googleapis.com
NAME: pubsub.googleapis.com
NAME: servicemanagement.googleapis.com
NAME: serviceusage.googleapis.com
NAME: sql-component.googleapis.com
NAME: storage-api.googleapis.com
NAME: storage-component.googleapis.com
NAME: storage.googleapis.com
Deleting kcc-lz-6300
disable billing on - and delete kcc-lz-6300
billingAccountName: ''
billingEnabled: false
name: projects/kcc-lz-6300/billingInfo
projectId: kcc-lz-6300
Deleted [https://cloudresourcemanager.googleapis.com/v1/projects/kcc-lz-6300].

You can undo this operation for a limited period by running the command below.
    $ gcloud projects undelete kcc-lz-6300

See https://cloud.google.com/resource-manager/docs/creating-managing-projects for information on shutting down projects.
Updated property [core/project].
Switched back to boot project pdt-tgz
Use the following command to switch to your new project
gcloud config set project kcc-lz-6300
**** Done ****

create only
root_@cloudshell:~/_current/pubsec-declarative-toolkit/solutions/document-processing/gcloud (pdt-tgz)$ ./deployment.sh -b pdt-tgz -u pdt3 -c true -l false -d false -e rest...one
Date: Fri 16 Dec 2022 06:46:45 PM UTC
Timestamp: 1671216405
running with: -b pdt-tgz -u pdt3 -c true -l false -d false -e restricted@transport.gcp.zone -p
Updated property [core/project].
Switched back to boot project pdt-tgz
Start: 1671216406
unique string: pdt3
REGION: northamerica-northeast1
NETWORK: pdt-pdt3-vpc
SUBNET: pdt-pdt3-sn
CLUSTER: pdt-pdt3
Creating project: kcc-lz-3479
CC_PROJECT_ID:
BOOT_PROJECT_ID: pdt-tgz
BILLING_ID: 011...5D
ORG_ID: 442178577666
Creating KCC project: kcc-lz-3479
Create in progress for [https://cloudresourcemanager.googleapis.com/v1/projects/kcc-lz-3479].
Waiting for [operations/cp.6912354144468666393] to finish...done.    
Enabling service [cloudapis.googleapis.com] on project [kcc-lz-3479]...
Operation "operations/acat.p2-374013806670-2e97bc09-84b0-49e9-94b3-45c545cce84e" finished successfully.
Updated property [core/project] to [kcc-lz-3479].
Updated property [core/project].
billingAccountName: billingAccounts/011...
billingEnabled: true
name: projects/kcc-lz-3479/billingInfo
projectId: kcc-lz-3479
Adding roles to project for user: restr...one
Updated IAM policy for project [kcc-lz-3479].
Updated IAM policy for project [kcc-lz-3479].
Updated IAM policy for project [kcc-lz-3479].
Updated IAM policy for project [kcc-lz-3479].
API's before
NAME: bigquery.googleapis.com
NAME: bigquerymigration.googleapis.com
NAME: bigquerystorage.googleapis.com
NAME: cloudapis.googleapis.com
NAME: clouddebugger.googleapis.com
NAME: cloudtrace.googleapis.com
NAME: datastore.googleapis.com
NAME: logging.googleapis.com
NAME: monitoring.googleapis.com
NAME: servicemanagement.googleapis.com
NAME: serviceusage.googleapis.com
NAME: sql-component.googleapis.com
NAME: storage-api.googleapis.com
NAME: storage-component.googleapis.com
NAME: storage.googleapis.com
Enabling APIs
Operation "operations/acat.p2-374013806670-2954f6a2-f36e-4183-b5bc-9be9b1a95172" finished successfully.
Operation "operations/acf.p2-374013806670-2ebdba0c-e94a-4fcc-b320-32fc027105a2" finished successfully.
Operation "operations/acat.p2-374013806670-67affaf9-d93f-434f-a0c8-d2acddc9f68d" finished successfully.
Operation "operations/acf.p2-374013806670-b1882ff1-4bdb-4d55-9ae2-b8735d074a11" finished successfully.
Operation "operations/acf.p2-374013806670-8913ebaa-3bde-4be9-9e41-f60c64ddc2d2" finished successfully.
Operation "operations/acf.p2-374013806670-7cde82ff-3975-4d44-a5eb-1edf273322ca" finished successfully.
Operation "operations/acat.p2-374013806670-d5e6834a-a664-49de-943c-55bbe075c461" finished successfully.
API's after
NAME: aiplatform.googleapis.com
NAME: autoscaling.googleapis.com
NAME: bigquery.googleapis.com
NAME: bigquerymigration.googleapis.com
NAME: bigquerystorage.googleapis.com
NAME: cloudapis.googleapis.com
NAME: cloudbilling.googleapis.com
NAME: clouddebugger.googleapis.com
NAME: cloudresourcemanager.googleapis.com
NAME: cloudtrace.googleapis.com
NAME: compute.googleapis.com
NAME: container.googleapis.com
NAME: containerfilesystem.googleapis.com
NAME: containerregistry.googleapis.com
NAME: dataflow.googleapis.com
NAME: datastore.googleapis.com
NAME: deploymentmanager.googleapis.com
NAME: documentai.googleapis.com
NAME: healthcare.googleapis.com
NAME: iam.googleapis.com
NAME: iamcredentials.googleapis.com
NAME: logging.googleapis.com
NAME: monitoring.googleapis.com
NAME: notebooks.googleapis.com
NAME: oslogin.googleapis.com
NAME: pubsub.googleapis.com
NAME: servicemanagement.googleapis.com
NAME: serviceusage.googleapis.com
NAME: sql-component.googleapis.com
NAME: storage-api.googleapis.com
NAME: storage-component.googleapis.com
NAME: storage.googleapis.com
Updated property [core/project].
Switched back to boot project pdt-tgz
Use the following command to switch to your new project
gcloud config set project kcc-lz-3479
**** Done ****

@obriensystems
Copy link
Collaborator

dev user with expected least priv roles
Screen Shot 2022-12-16 at 13 57 34

Screen Shot 2022-12-16 at 14 04 23

obriensystems added a commit to cloud-quickstart/pubsec-declarative-toolkit that referenced this issue Dec 16, 2022
fmichaelobrien added a commit that referenced this issue Dec 16, 2022
fmichaelobrien added a commit to cloud-quickstart/pubsec-declarative-toolkit that referenced this issue Dec 19, 2022
fmichaelobrien added a commit that referenced this issue Dec 19, 2022
#220 - update readme with architecture
obriensystems added a commit to cloud-quickstart/pubsec-declarative-toolkit that referenced this issue Dec 20, 2022
fmichaelobrien added a commit that referenced this issue Dec 20, 2022
obriensystems added a commit to cloud-quickstart/pubsec-declarative-toolkit that referenced this issue Dec 20, 2022
fmichaelobrien added a commit that referenced this issue Dec 20, 2022
fmichaelobrien added a commit that referenced this issue Dec 21, 2022
obriensystems added a commit to cloud-quickstart/pubsec-declarative-toolkit that referenced this issue Dec 22, 2022
fmichaelobrien added a commit that referenced this issue Dec 22, 2022
obriensystems added a commit to obriensystems/pubsec-declarative-toolkit that referenced this issue Dec 24, 2022
fmichaelobrien added a commit that referenced this issue Dec 24, 2022
fmichaelobrien added a commit to cloud-quickstart/pubsec-declarative-toolkit that referenced this issue Jan 7, 2023
fmichaelobrien added a commit to cloud-quickstart/pubsec-declarative-toolkit that referenced this issue Jan 7, 2023
obriensystems added a commit to cloud-quickstart/pubsec-declarative-toolkit that referenced this issue Jan 7, 2023
fmichaelobrien added a commit that referenced this issue Jan 9, 2023
#220 - canary trigger cloud build via random commit
@obriensystems
Copy link
Collaborator

Run

michael@cloudshell:~/docproc-old/pubsec-declarative-toolkit/solutions/document-processing/gcloud (docproc-old)$ ./deployment.sh -u pdt -c true -l false -d false
Default USER_EMAIL if set is: mi....ev
USER_EMAIL reset to mi....v
Date: Mon 09 Jan 2023 01:51:33 AM UTC
Timestamp: 1673229093
running with: -b docproc-old -u pdt -c true -l false -d false -e mi....ev -p
Updated property [core/project].
Switched back to boot project docproc-old
Start: 1673229094
unique string: pdt
REGION: us-central1
NETWORK: pdt-pdt-vpc
SUBNET: pdt-pdt-sn
CLUSTER: pdt-pdt
Creating project: docai-gen-6623
CC_PROJECT_ID: docai-gen-6623
passed in KCC_PROJECT_ID:
BOOT_PROJECT_ID: docproc-old
BILLING_ID: 0...
ORG_ID: 5...
Creating KCC project: docai-gen-6623
Create in progress for [https://cloudresourcemanager.googleapis.com/v1/projects/docai-gen-6623].
Waiting for [operations/cp.8658187408847568157] to finish...done.    
Enabling service [cloudapis.googleapis.com] on project [docai-gen-6623]...
Operation "operations/acat.p2-81974734379-1a06f2cf-88ef-4dbc-baf6-580d9ed64d9f" finished successfully.
Updated property [core/project] to [docai-gen-6623].
Updated property [core/project].
billingAccountName: billingAccounts/0...5D
billingEnabled: true
name: projects/docai-gen-6623/billingInfo
projectId: docai-gen-6623
Adding roles to project for user: mi...ev
Updated IAM policy for project [docai-gen-6623].
Updated IAM policy for project [docai-gen-6623].
Enabling APIs
Operation "operations/acat.p2-81974734379-f101eb7e-a0e4-47c2-aafc-aa5dced08792" finished successfully.
Operation "operations/acf.p2-81974734379-6d1de5ca-89ae-4f7e-ba7b-7dd379f6de57" finished successfully.
Operation "operations/acat.p2-81974734379-d280de57-b0b6-449e-9771-adf06835f5b7" finished successfully.
Operation "operations/acf.p2-81974734379-271ececa-b018-49cc-887f-3d59ebf1776d" finished successfully.
Operation "operations/acf.p2-81974734379-ffaf1b82-8747-4c8a-af32-e4e20269c11a" finished successfully.
Operation "operations/acat.p2-81974734379-a10a72d2-cf71-46bf-827a-c5f9840ab157" finished successfully.
Operation "operations/acat.p2-81974734379-4646a690-f8cd-4959-bed3-4951fc9c2e43" finished successfully.
Operation "operations/acf.p2-81974734379-3e62e082-ca39-428d-8066-9d01c2829fcd" finished successfully.
Operation "operations/acf.p2-81974734379-0c498681-e4cb-4551-9dbe-e468627d3dd1" finished successfully.
Operation "operations/acf.p2-81974734379-3de0d800-c292-465a-9d6e-1491fbb8af6a" finished successfully.
Operation "operations/acat.p2-81974734379-84e13435-e2e4-494e-8cd5-9ecc0e86b413" finished successfully.
Operation "operations/acf.p2-81974734379-a5278b81-eaba-4d57-9437-fc4054eace4d" finished successfully.
Operation "operations/acf.p2-81974734379-d1087173-f32d-461e-abf2-c36332d057a6" finished successfully.
Operation "operations/acat.p2-81974734379-b9f15602-13fd-4875-8113-448f63dad594" finished successfully.
Operation "operations/acat.p2-81974734379-20e0539f-23d8-479d-a6f7-ad1fbb4d1479" finished successfully.
Operation "operations/acf.p2-81974734379-a994cbc3-33f1-46de-8756-3fc0cc34b668" finished successfully.
Operation "operations/acat.p2-81974734379-0b38321d-a7d4-4ed9-810f-aef9f2b22806" finished successfully.
Operation "operations/acat.p2-81974734379-e11ecb39-9ae9-464e-999f-ad910f81af93" finished successfully.
Create VPC: pdt-pdt-vpc
Created [https://www.googleapis.com/compute/v1/projects/docai-gen-6623/global/networks/pdt-pdt-vpc].
NAME: pdt-pdt-vpc
SUBNET_MODE: CUSTOM
BGP_ROUTING_MODE: REGIONAL
IPV4_RANGE:
GATEWAY_IPV4:

Instances on this network will not be reachable until firewall rules
are created. As an example, you can allow all internal traffic between
instances as well as SSH, RDP, and ICMP by running:

$ gcloud compute firewall-rules create <FIREWALL_NAME> --network pdt-pdt-vpc --allow tcp,udp,icmp --source-ranges <IP_RANGE>
$ gcloud compute firewall-rules create <FIREWALL_NAME> --network pdt-pdt-vpc --allow tcp:22,tcp:3389,icmp

Create subnet pdt-pdt-sn off VPC: pdt-pdt-vpc
Created [https://www.googleapis.com/compute/v1/projects/docai-gen-6623/regions/us-central1/subnetworks/pdt-pdt-sn].
NAME: pdt-pdt-sn
REGION: us-central1
NETWORK: pdt-pdt-vpc
RANGE: 192.168.0.0/16
STACK_TYPE: IPV4_ONLY
IPV6_ACCESS_TYPE:
INTERNAL_IPV6_PREFIX:
EXTERNAL_IPV6_PREFIX:
create backup dir in pr..ud dir
/home/michael/docproc-old/pubsec-declarative-toolkit/solutions/document-processing/gcloud
Create service account: service-account-main@docai-gen-6623.iam.gserviceaccount.com
Created service account [service-account-main].
Email: service-account-main@docai-gen-6623.iam.gserviceaccount.com
Updated IAM policy for project [docai-gen-6623].
Updated IAM policy for project [docai-gen-6623].
Updated IAM policy for project [docai-gen-6623].
Updated IAM policy for project [docai-gen-6623].
Updated IAM policy for project [docai-gen-6623].
Updated IAM policy for project [docai-gen-6623].
Updated IAM policy for project [docai-gen-6623].
Updated IAM policy for project [docai-gen-6623].
Creating CSR
Created [docproc3].
WARNING: You may be billed for this repository. See https://cloud.google.com/source-repositories/docs/pricing for details.
cloning for CSR repo https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit
Cloning into 'pubsec-declarative-toolkit'...
remote: Enumerating objects: 6081, done.
remote: Counting objects: 100% (108/108), done.
remote: Compressing objects: 100% (65/65), done.
remote: Total 6081 (delta 46), reused 86 (delta 39), pack-reused 5973
Receiving objects: 100% (6081/6081), 5.09 MiB | 23.60 MiB/s, done.
Resolving deltas: 100% (3825/3825), done.
Enumerating objects: 4471, done.
Counting objects: 100% (4471/4471), done.
Delta compression using up to 4 threads
Compressing objects: 100% (1411/1411), done.
Writing objects: 100% (4471/4471), 1.66 MiB | 9.34 MiB/s, done.
Total 4471 (delta 2759), reused 4470 (delta 2759), pack-reused 0
remote: Resolving deltas: 100% (2759/2759)
To https://source.developers.google.com/p/docai-gen-6623/r/docproc3
 * [new branch]      main -> main
Branch 'canary' set up to track remote branch 'canary' from 'origin'.
Switched to a new branch 'canary'
[canary 18311b1] triple cloud build main
 1 file changed, 31 insertions(+)
 create mode 100644 cloudbuild-prod-main.yaml
Enumerating objects: 276, done.
Counting objects: 100% (268/268), done.
Delta compression using up to 4 threads
Compressing objects: 100% (127/127), done.
Writing objects: 100% (256/256), 61.35 KiB | 20.45 MiB/s, done.
Total 256 (delta 130), reused 246 (delta 122), pack-reused 0
remote: Resolving deltas: 100% (130/130)
To https://source.developers.google.com/p/docai-gen-6623/r/docproc3
 * [new branch]      canary -> canary
create ar
Create request issued for: [docproc3]
Waiting for operation [projects/docai-gen-6623/locations/us-central1/operations/d3833584-41c5-4a78-bcc3-ad126f3db925] to complete...done.     
Created repository [docproc3].
Create Cloud Build - prod
Created [https://cloudbuild.googleapis.com/v1/projects/docai-gen-6623/locations/global/triggers/e5aa7f84-c6f7-441a-a22c-8c74246caacd].
NAME: prod-main
CREATE_TIME: 2023-01-09T01:56:09+00:00
STATUS:
trigger_prod_main_build
Already on 'canary'
Your branch is ahead of 'origin/canary' by 1 commit.
  (use "git push" to publish your local commits)
copy random file empty9193_stub.sh
[canary 9ff52d0] trigger prod with 9193
 1 file changed, 16 insertions(+)
 create mode 100755 empty9193_stub.sh
Enumerating objects: 4, done.
Counting objects: 100% (4/4), done.
Delta compression using up to 4 threads
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 667 bytes | 667.00 KiB/s, done.
Total 3 (delta 1), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (1/1)
To https://source.developers.google.com/p/docai-gen-6623/r/docproc3
   18311b1..9ff52d0  canary -> canary
Updated property [core/project].
Switched back to boot project docproc-old
Use the following command to switch to your new project
gcloud config set project docai-gen-6623
**** Done ****
michael@cloudshell:~/docproc-old/pubsec-declarative-toolkit/solutions/document-processing/gcloud (docproc-old)$

obriensystems added a commit to cloud-quickstart/pubsec-declarative-toolkit that referenced this issue Jan 9, 2023
fmichaelobrien added a commit that referenced this issue Jan 9, 2023
obriensystems added a commit to cloud-quickstart/pubsec-declarative-toolkit that referenced this issue Jan 10, 2023
fmichaelobrien added a commit that referenced this issue Jan 10, 2023
obriensystems added a commit to cloud-quickstart/pubsec-declarative-toolkit that referenced this issue Jan 10, 2023
fmichaelobrien added a commit that referenced this issue Jan 10, 2023
#220 - update python Dockerfile working
obriensystems added a commit to cloud-quickstart/pubsec-declarative-toolkit that referenced this issue Jan 10, 2023
fmichaelobrien added a commit that referenced this issue Jan 10, 2023
#220 - full cloudbuild env substitution
obriensystems added a commit to cloud-quickstart/pubsec-declarative-toolkit that referenced this issue Jan 10, 2023
fmichaelobrien added a commit that referenced this issue Jan 10, 2023
obriensystems added a commit to cloud-quickstart/pubsec-declarative-toolkit that referenced this issue Jan 12, 2023
fmichaelobrien added a commit that referenced this issue Jan 12, 2023
#220 - document ai processor endpoint
fmichaelobrien added a commit that referenced this issue Jan 12, 2023
obriensystems added a commit to cloud-quickstart/pubsec-declarative-toolkit that referenced this issue Jan 13, 2023
fmichaelobrien added a commit that referenced this issue Jan 13, 2023
#220 - document ai create/process/delete OK
@obriensystems
Copy link
Collaborator

obriensystems commented Jan 13, 2023

  • add root and subfolder additons
  • add folder admin role
  • transition document ai example

add bulk resource export to krm
Screen Shot 2023-01-13 at 01 29 35

obriensystems added a commit to cloud-quickstart/pubsec-declarative-toolkit that referenced this issue Jan 13, 2023
fmichaelobrien added a commit that referenced this issue Jan 13, 2023
obriensystems added a commit to cloud-quickstart/pubsec-declarative-toolkit that referenced this issue Jan 13, 2023
fmichaelobrien added a commit that referenced this issue Jan 13, 2023
obriensystems added a commit to cloud-quickstart/pubsec-declarative-toolkit that referenced this issue Jan 13, 2023
fmichaelobrien added a commit that referenced this issue Jan 13, 2023
@obriensystems
Copy link
Collaborator

obriensystems commented Jan 16, 2023

Developer accounts with folder admin and project creation roles
use case 1: move project between folders or from organization to folder
use case 2: create new sub folder off folder
use case 3 variant: cannot create folder off organization
uc 4 - locked creating folder or proj off admin-only

After org removal of BAC and ProjectCreator - (moved to super admin account only)
Verify - restricted user cannot create folders off org - only off designated root folder - ok "You do not have the required "resourcemanager.folders.create" permission to create folders in this location."

Remaining:
restricted user can still create projects anywhere


pre
remove from organization at org root in iam (BillingAccountCreator and ProjectCreator) to one of the org admins


set the org admin with additional FolderAdmin role for the org root
gcloud organizations add-iam-policy-binding 5..5 --member=user:y@landing.systems --role=roles/resourcemanager.folderAdmin

sa creates super folder
gcloud resource-manager folders create --display-name=ch-root --organization=5..
name: 'folders/621660468815'

sa creates project in prep for restricted accounts
ch-restricted-proj1

sa associates restricted accounts as project owner
gcloud projects add-iam-policy-binding ch-restricted-proj1  --member=user:$USER_EMAIL --role=roles/owner


sa adds restricted accounts as owner on super folder
gcloud resource-manager folders add-iam-policy-binding 621660468815 --member=user:restricted@landing.systems --role=roles/resourcemanager.folderAdmin



restricted account creates subfolder
gcloud resource-manager folders create --display-name=ch-root-sub1 --folder=621660468815

 name: 'folders/917483507652'
 parent: 'folders/621660468815'

restricted account creates new projects off subfolder
  gcloud projects create ch-restricted-proj-sub2a --name="restricted-proj-sub2a" --folder 917483507652
  gcloud config set project "${CC_PROJECT_ID}"


restricted user moves projects from org into subfolder

try admin only proj
cloud projects create admin-only2 --name="admin-only2" --folder 694452364990

result
org
+--ch-restricted-proj1 owner
+--ch-root (folder)
      +--ch-root-sub1 (folder)  restricted FolderAadmin
             +--ch-sub-proj1a 

@fmichaelobrien
Copy link
Contributor Author

Thanks Chris - I see I need to switch from beta resource-config to config-connector
config-connector bulk-export

https://cloud.google.com/config-connector/docs/how-to/import-export/bulk-export#exporting_an_inventory_with_config-connector

for

https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit/blob/canary/solutions/document-processing/gcloud/deployment.sh#L588

bulk_resources_export_to_krm() {
    sudo apt-get install google-cloud-sdk-config-connector
    #gcloud services enable cloudasset.googleapis.com
    mkdir ${REPO_TREE_DEPTH_FOR_CD_UP}$RESOURCE_CONFIG_BULK_EXPORT_TO_KRM_SUBDIR
    gcloud beta resource-config bulk-export --path=${REPO_TREE_DEPTH_FOR_CD_UP}$RESOURCE_CONFIG_BULK_EXPORT_TO_KRM_SUBDIR
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants