#103 - workaround for missing cluster policy CRDs

[fmichaelobrien]

Let me just retest this change on another organization
I tested it on gcp.zone, retesting over friday on nuage-cloud.org that had the same issue

[fmichaelobrien]

hold for at least 6 hours until I regression test on a 2nd clean system and post cli results

[obriensystems]

Fully tested in #103 in org obrienlabs.cloud using advanced install

issue is

installing inventory ResourceGroup CRD.
namespace/config-control apply failed: can't adopt an object without the annotation config.k8s.io/owning-inventory
namespace/config-control reconcile skipped
configmap/setters unchanged

using --inventory-policy adopt via kptdev/kpt#1724
works well

before

michael@cloudshell:~/github/GoogleCloudPlatform/20220909-103 (lz-20220910-oldev)$ kpt live apply landing-zone
installing inventory ResourceGroup CRD.
namespace/config-control apply failed: can't adopt an object without the annotation config.k8s.io/owning-inventory
namespace/config-control reconcile skipped
configmap/setters unchanged
..
iampolicymember.iam.cnrm.cloud.google.com/log-sink-writer reconcile skipped
0 resource(s) reconciled, 90 skipped, 0 failed to reconcile, 0 timed out
1 resources failed


after
michael@cloudshell:~/github/GoogleCloudPlatform/20220909-103 (lz-20220910-oldev)$ kpt live apply landing-zone --inventory-policy adopt
installing inventory ResourceGroup CRD.
namespace/config-control configured
namespace/config-control reconcile pending
namespace/config-control reconciled
configmap/setters created
accesscontextmanageraccesslevel.accesscontextmanager.cnrm.cloud.google.com/commonaccesslevels created
accesscontextmanageraccesslevel.accesscontextmanager.cnrm.cloud.google.com/nonprodperimaccesslevel created
accesscontextmanageraccesslevel.accesscontextmanager.cnrm.cloud.google.com/prodaccesslevels created
accesscontextmanageraccesspolicy.accesscontextmanager.cnrm.cloud.google.com/orgaccesspolicy created
computefirewall.compute.cnrm.cloud.google.com/allow-egress-internet created

michael@cloudshell:~ (magellan-01)$ kubectl get gcp
NAME                                                                                          AGE   READY   STATUS         STATUS AGE
accesscontextmanageraccesspolicy.accesscontextmanager.cnrm.cloud.google.com/orgaccesspolicy   21m   False   UpdateFailed   21m

NAME                                                                                                 AGE   READY   STATUS               STATUS AGE
accesscontextmanageraccesslevel.accesscontextmanager.cnrm.cloud.google.com/commonaccesslevels        21m   False   DependencyNotReady   21m
accesscontextmanageraccesslevel.accesscontextmanager.cnrm.cloud.google.com/nonprodperimaccesslevel   21m   False   DependencyNotReady   21m
accesscontextmanageraccesslevel.accesscontextmanager.cnrm.cloud.google.com/prodaccesslevels          21m   False   DependencyNotReady   21m

NAME                                                                        AGE   READY   STATUS         STATUS AGE
computeprojectmetadata.compute.cnrm.cloud.google.com/nonprod-oslogin-meta   21m   False   UpdateFailed   21m

NAME                                                                         AGE   READY   STATUS               STATUS AGE
computesubnetwork.compute.cnrm.cloud.google.com/common-ha-perimeter-subnet   21m   False   DependencyNotReady   21m
computesubnetwork.compute.cnrm.cloud.google.com/management                   21m   False   DependencyNotReady   21m
computesubnetwork.compute.cnrm.cloud.google.com/nonprod-sharedvpc-subnet     21m   False   DependencyNotReady   21m
computesubnetwork.compute.cnrm.cloud.google.com/priv-perimeter-subnet        21m   False   DependencyNotReady   21m
computesubnetwork.compute.cnrm.cloud.google.com/prod-sharedvpc-subnet        21m   False   DependencyNotReady   21m
computesubnetwork.compute.cnrm.cloud.google.com/public-perimeter-subnet      21m   False   DependencyNotReady   21m

NAME                                                                                           AGE   READY   STATUS         STATUS AGE
computesharedvpchostproject.compute.cnrm.cloud.google.com/computesharedvpchostproject-sample   21m   False   UpdateFailed   21m
computesharedvpchostproject.compute.cnrm.cloud.google.com/nonprod-shared-vpc-host              21m   False   UpdateFailed   21m

NAME                                                                 AGE   READY   STATUS         STATUS AGE
computenetwork.compute.cnrm.cloud.google.com/common-ha-perimeter     21m   False   UpdateFailed   21m
computenetwork.compute.cnrm.cloud.google.com/common-mgmt-perimeter   21m   False   UpdateFailed   21m
computenetwork.compute.cnrm.cloud.google.com/nonprod-sharedvpc       21m   False   UpdateFailed   21m
computenetwork.compute.cnrm.cloud.google.com/priv-perimeter          21m   False   UpdateFailed   21m
computenetwork.compute.cnrm.cloud.google.com/prod-sharedvpc          21m   False   UpdateFailed   21m
computenetwork.compute.cnrm.cloud.google.com/public-perimeter        21m   False   UpdateFailed   21m

NAME                                                                 AGE   READY   STATUS               STATUS AGE
computeroute.compute.cnrm.cloud.google.com/egress-internet-nonprod   21m   False   DependencyNotReady   21m
computeroute.compute.cnrm.cloud.google.com/egress-internet-prod      21m   False   DependencyNotReady   21m

NAME                                                                        AGE   READY   STATUS               STATUS AGE
computefirewall.compute.cnrm.cloud.google.com/allow-egress-internet         21m   False   DependencyNotReady   21m
computefirewall.compute.cnrm.cloud.google.com/allow-egress-internet-pr      21m   False   DependencyNotReady   21m
computefirewall.compute.cnrm.cloud.google.com/allow-egress-internet-pu      21m   False   DependencyNotReady   21m
computefirewall.compute.cnrm.cloud.google.com/allow-ssh-ingress-pr          21m   False   DependencyNotReady   21m
computefirewall.compute.cnrm.cloud.google.com/allow-ssh-ingressp            21m   False   DependencyNotReady   21m
computefirewall.compute.cnrm.cloud.google.com/computefirewall-sample-deny   21m   False   DependencyNotReady   21m
computefirewall.compute.cnrm.cloud.google.com/deny-ssh-ingress              21m   False   DependencyNotReady   21m
computefirewall.compute.cnrm.cloud.google.com/prod-firewall-default-deny    21m   False   DependencyNotReady   21m

NAME                                                            AGE   READY   STATUS               STATUS AGE
iampolicymember.iam.cnrm.cloud.google.com/audit-viewer          21m   True    UpToDate             21m
iampolicymember.iam.cnrm.cloud.google.com/billing-iam-member    21m   False   DependencyNotReady   21m
iampolicymember.iam.cnrm.cloud.google.com/log-reader            21m   True    UpToDate             20m
iampolicymember.iam.cnrm.cloud.google.com/log-writer            21m   True    UpToDate             20m
iampolicymember.iam.cnrm.cloud.google.com/organization-viewer   21m   True    UpToDate             21m

NAME                                                                  AGE   READY   STATUS         STATUS AGE
iamserviceaccount.iam.cnrm.cloud.google.com/billing-service-account   21m   False   UpdateFailed   21m

NAME                                                             AGE   READY   STATUS               STATUS AGE
logginglogsink.logging.cnrm.cloud.google.com/audit-bucket-sink   21m   False   DependencyNotReady   21m
logginglogsink.logging.cnrm.cloud.google.com/logs-bucket-sink    21m   False   DependencyNotReady   21m

NAME                                                                                               AGE   READY   STATUS         STATUS AGE
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/allowed-contact-domains                21m   True    UpToDate       21m
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/allowed-policy-member-domain           21m   True    UpToDate       21m
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/disable-guest-attribute-access         21m   True    UpToDate       21m
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/disable-nested-virtualization          21m   True    UpToDate       21m
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/disable-serial-port-access             21m   True    UpToDate       21m
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/disable-serviceaccount-key-creation    21m   True    UpToDate       21m
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/disable-vpc-external-ipv6              21m   True    UpToDate       21m
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/require-shielded-vm                    21m   True    UpToDate       21m
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/require-trusted-images                 21m   True    UpToDate       21m
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/restrict-loadbalancer-creation-types   21m   False   UpdateFailed   21m
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/restrict-os-login                      21m   True    UpToDate       21m
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/restrict-resource-locations            21m   False   UpdateFailed   21m
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/restrict-sql-public-ip                 21m   True    UpToDate       21m
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/restrict-vm-external-access            21m   False   UpdateFailed   21m
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/restrict-vpc-lien-removal              21m   True    UpToDate       21m
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/restrict-vpc-peering                   21m   True    UpToDate       21m
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/skip-default-network-creation          21m   True    UpToDate       21m
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/storage-public-access-prevention       21m   True    UpToDate       21m
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/uniform-bucket-level-access            21m   True    UpToDate       21m
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/vm-can-ip-forward                      21m   True    UpToDate       21m

NAME                                                                                       AGE   READY   STATUS     STATUS AGE
folder.resourcemanager.cnrm.cloud.google.com/audit-and-security                            21m   True    UpToDate   21m
folder.resourcemanager.cnrm.cloud.google.com/audit-and-security.audit                      21m   True    UpToDate   21m
folder.resourcemanager.cnrm.cloud.google.com/audit-and-security.security                   21m   True    UpToDate   21m
folder.resourcemanager.cnrm.cloud.google.com/automation                                    21m   True    UpToDate   21m
folder.resourcemanager.cnrm.cloud.google.com/infrastructure                                21m   True    UpToDate   21m
folder.resourcemanager.cnrm.cloud.google.com/infrastructure.networking                     21m   True    UpToDate   20m
folder.resourcemanager.cnrm.cloud.google.com/infrastructure.networking.nonprodnetworking   21m   True    UpToDate   19m
folder.resourcemanager.cnrm.cloud.google.com/infrastructure.networking.prodnetworking      21m   True    UpToDate   19m
folder.resourcemanager.cnrm.cloud.google.com/infrastructure.sharedinfrastructure           21m   True    UpToDate   19m
folder.resourcemanager.cnrm.cloud.google.com/sandbox                                       21m   True    UpToDate   21m
folder.resourcemanager.cnrm.cloud.google.com/shared-services                               21m   True    UpToDate   21m
folder.resourcemanager.cnrm.cloud.google.com/workloads                                     21m   True    UpToDate   21m
folder.resourcemanager.cnrm.cloud.google.com/workloads.dev                                 21m   True    UpToDate   19m
folder.resourcemanager.cnrm.cloud.google.com/workloads.prod                                21m   True    UpToDate   20m
folder.resourcemanager.cnrm.cloud.google.com/workloads.uat                                 21m   True    UpToDate   20m

NAME                                                                          AGE   READY   STATUS         STATUS AGE
project.resourcemanager.cnrm.cloud.google.com/audit-prj-id-old1               21m   False   UpdateFailed   21m
project.resourcemanager.cnrm.cloud.google.com/guardrails-project-old1         21m   False   UpdateFailed   21m
project.resourcemanager.cnrm.cloud.google.com/net-host-prj-nonprod-old1       21m   False   UpdateFailed   21m
project.resourcemanager.cnrm.cloud.google.com/net-host-prj-prod-old1          21m   False   UpdateFailed   21m
project.resourcemanager.cnrm.cloud.google.com/net-perimeter-prj-common-old1   21m   False   UpdateFailed   21m

NAME                                                                         AGE   READY   STATUS         STATUS AGE
service.serviceusage.cnrm.cloud.google.com/common-nethost-service-compute    21m   False   UpdateFailed   21m
service.serviceusage.cnrm.cloud.google.com/common-nethost-service-logging    21m   False   UpdateFailed   21m
service.serviceusage.cnrm.cloud.google.com/nonprod-nethost-service-compute   21m   False   UpdateFailed   21m
service.serviceusage.cnrm.cloud.google.com/nonprod-nethost-service-dns       21m   False   UpdateFailed   21m
service.serviceusage.cnrm.cloud.google.com/nonprod-nethost-service-logging   21m   False   UpdateFailed   21m
service.serviceusage.cnrm.cloud.google.com/prod-nethost-service-compute      21m   False   UpdateFailed   21m
service.serviceusage.cnrm.cloud.google.com/prod-nethost-service-logging      21m   False   UpdateFailed   21m

NAME                                                                       AGE   READY   STATUS         STATUS AGE
storagebucket.storage.cnrm.cloud.google.com/audit-audit-prj-id-old1        21m   False   UpdateFailed   21m
storagebucket.storage.cnrm.cloud.google.com/log-bucket-audit-prj-id-old1   21m   False   UpdateFailed   21m

[obriensystems]

ready to merge

[fmichaelobrien]

#114 is the full workaround and possible undo of #103

[cartyc]

Taking another pass. I think we should just document this in the README instead of adding it to the .krmignore file. This is because if we modify that file it will impact both the GitOps deployment and the future Skaffold deployment options.

[cartyc]

An alternate temp fix would be to just add constraint.yaml to the .krmignore for the first run so the CRD is installed and then remove the constraint.yaml ref from .krmignore on the second pass which will install the constraint once the CRD is installed.

[fmichaelobrien]

I agree, move to the readme or use something more granular than the entire folder.
I think we need more reproduction and testing and root cause before blocking off these 2 folders - reinvestigating

and your
cartyc commented 1 hour ago
An alternate temp fix would be to just add constraint.yaml to the .krmignore for the first run so the CRD is installed and then remove the constraint.yaml ref from .krmignore on the second pass which will install the constraint once the CRD is installed.

[cartyc]

An alternate solution would be to have the guardrails being deployed as part of a different step and remove it from the Landing Zone package. This would smooth the kpt live apply deployment process but would require additional steps in ci to validate compliance. For example as part of a CI deployment the guardrails package could be fetched via kpt pkg get ... as a ci step instead of directly in the primary kpt fn render.

[fmichaelobrien]

Good workaround for the future. For now working on getting the LZ up first before we start refactoring it.

[cartyc]

Updating documentation to note this in pr #129, closing in favor in of moving discussion there.