Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] add cd .. between kpt fn render and kpt live commands during resource inventory update - or take out the landing-zone folder reference - landing-zone solution #111

Closed
obriensystems opened this issue Sep 10, 2022 · 3 comments
Closed

[BUG] add cd .. between kpt fn render and kpt live commands during resource inventory update - or take out the landing-zone folder reference - landing-zone solution #111

obriensystems opened this issue Sep 10, 2022 · 3 comments
Assignees
@fmichaelobrien
Labels
bug Something isn't working

Comments

@obriensystems
Copy link
Collaborator

Describe the bug
running the landing-zone kpt (non-gitops section) directly will fail on the kpt resource inventory update (step 2 of 3)

The 3 commands in the landing-zone solution section 5 are not run in the same directory - you must .. up a dir when running the 2 kpt live commands or take out the landing-zone folder reference from the kpt resource inventory update and kpt apply of the package to the cluster

Additional context

kpt fn render
kpt live init landing-zone --namespace config-control
kpt live apply landing-zone --reconcile-timeout=2m --output=table

to
cd landing-zone
kpt fn render
cd ..
kpt live init landing-zone --namespace config-control
kpt live apply landing-zone --reconcile-timeout=2m --output=table

or take out the folder
michael@cloudshell:~/wse_github/GoogleCloudPlatform/landing-zone (landing-zone-controller-e4g7d)$ kpt live init landing-zone --namespace config-control
Error: invalid directory argument: landing-zone
michael@cloudshell:~/wse_github/GoogleCloudPlatform/landing-zone (landing-zone-controller-e4g7d)$ kpt live init  --namespace config-control
initializing Kptfile inventory info (namespace: config-control)...failed
Error: Inventory information has already been added to the package Kptfile. Changing it after a package has been applied to the cluster can lead to undesired results. Use the --force flag to suppress this error.
michael@cloudshell:~/wse_github/GoogleCloudPlatform/landing-zone (landing-zone-controller-e4g7d)$ kpt live apply  --reconcile-timeout=2m
installing inventory ResourceGroup CRD.
namespace/config-control apply failed: can't adopt an object without the annotation config.k8s.io/owning-inventory
namespace/config-control reconcile skipped
configmap/setters unchanged
accesscontextmanageraccesslevel.accesscontextmanager.cnrm.cloud.google.com/commonaccesslevels unchanged

**Full run results**
michael@cloudshell:~/wse_github/GoogleCloudPlatform (landing-zone-controller-e4g7d)$ ls
landing-zone  pubsec-declarative-toolkit
michael@cloudshell:~/wse_github/GoogleCloudPlatform (landing-zone-controller-e4g7d)$ cd landing-zone/
michael@cloudshell:~/wse_github/GoogleCloudPlatform/landing-zone (landing-zone-controller-e4g7d)$ kpt fn render
Package "landing-zone/environments/common/guardrails-policies":
Package "landing-zone/environments/common":
....
[PASS] "gcr.io/kpt-fn/kubeval:v0.3.0" in 21s
Successfully executed 9 function(s) in 5 package(s).


michael@cloudshell:~/wse_github/GoogleCloudPlatform/landing-zone (landing-zone-controller-e4g7d)$ kpt live init landing-zone --namespace config-control
Error: invalid directory argument: landing-zone
michael@cloudshell:~/wse_github/GoogleCloudPlatform/landing-zone (landing-zone-controller-e4g7d)$ cd ..
michael@cloudshell:~/wse_github/GoogleCloudPlatform (landing-zone-controller-e4g7d)$ kpt live init landing-zone --namespace config-control
initializing Kptfile inventory info (namespace: config-control)...failed
Error: Inventory information has already been added to the package Kptfile. Changing it after a package has been applied to the cluster can lead to undesired results. Use the --force flag to suppress this error.

or see the alternate landing zone code at

where we leave out the folder name

kpt live init --namespace ${NAMESPACE}"
@obriensystems
Copy link
Collaborator Author

2nd run to verify

After bringing up a new cluster on obrienlabs.dev in #94 (comment) for #104

https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit/tree/main/solutions/landing-zone

michael@cloudshell:~ (lz-20220910-oldev)$ cd github/GoogleCloudPlatform/p
michael@cloudshell:~/github/GoogleCloudPlatform (lz-20220910-oldev)$ mkdir 20220909-103
michael@cloudshell:~/github/GoogleCloudPlatform (lz-20220910-oldev)$ cd 20220909-103/
michael@cloudshell:~/github/GoogleCloudPlatform/20220909-103 (lz-20220910-oldev)$ git clone https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit.git
Cloning into 'pubsec-declarative-toolkit'...
Resolving deltas: 100% (2585/2585), done.

michael@cloudshell:~/github/GoogleCloudPlatform/20220909-103 (lz-20220910-oldev)$ cd pubsec-declarative-toolkit/
michael@cloudshell:~/github/GoogleCloudPlatform/20220909-103/pubsec-declarative-toolkit (lz-20220910-oldev)$ git checkout fmichaelobrien-patch-103
Branch 'fmichaelobrien-patch-103' set up to track remote branch 'fmichaelobrien-patch-103' from 'origin'.
Switched to a new branch 'fmichaelobrien-patch-103'

michael@cloudshell:~/github/GoogleCloudPlatform/20220909-103/pubsec-declarative-toolkit (lz-20220910-oldev)$ cd ..
michael@cloudshell:~/github/GoogleCloudPlatform/20220909-103 (lz-20220910-oldev)$ kpt pkg get https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit.git/solutions/landing-zone landing-zone
Package "landing-zone":
Fetching https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit@main
From https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit
 * branch            main       -> FETCH_HEAD
 + 6596f71...9750a11 main       -> origin/main  (forced update)
Adding package "solutions/landing-zone".

Fetched 1 package(s).


copy the change
michael@cloudshell:~/github/GoogleCloudPlatform/20220909-103/pubsec-declarative-toolkit (lz-20220910-oldev)$ cd ..
michael@cloudshell:~/github/GoogleCloudPlatform/20220909-103 (lz-20220910-oldev)$ cat landing-zone/.krmignore
cicd-examples/
michael@cloudshell:~/github/GoogleCloudPlatform/20220909-103 (lz-20220910-oldev)$ cp pubsec-declarative-toolkit/solutions/landing-zone/.krmignore landing-zone/
michael@cloudshell:~/github/GoogleCloudPlatform/20220909-103 (lz-20220910-oldev)$ cat landing-zone/.krmignore 
cicd-examples/
environments/common/guardrails-policies
environments/common/general-policies/naming-rules
michael@cloudshell:~/github/GoogleCloudPlatform/20220909-103 (lz-20220910-oldev)$

fill in setters.yaml
use 
https://console.cloud.google.com/home/dashboard?project=landing-zone-controller-1z583

  billing-id: "019283-6F1AB5-7AD576"
  org-id: "583675367868"
  management-project-id: landing-zone-controller-1z583
  management-project-number: "453474601356"
  net-host-prj-nonprod-id: net-host-prj-nonprod-old1
  net-host-prj-prod-id: net-host-prj-prod-old1
  net-perimeter-prj-common-id: net-perimeter-prj-common-old1
  audit-prj-id: audit-prj-id-old1
  guardrails-project-id: guardrails-project-old1
  audit-viewer: postmaster@obrienlabs.dev
  log-writer: postmaster@obrienlabs.dev
  log-reader: postmaster@obrienlabs.dev
  organization-viewer: postmaster@obrienlabs.dev

changing the docs to cd in the folder

ichael@cloudshell:~/github/GoogleCloudPlatform/20220909-103 (lz-20220910-oldev)$ kpt fn render
Error: No Kptfile found at "/home/michael/github/GoogleCloudPlatform/20220909-103".
michael@cloudshell:~/github/GoogleCloudPlatform/20220909-103 (lz-20220910-oldev)$ cd landing-zone/
michael@cloudshell:~/github/GoogleCloudPlatform/20220909-103/landing-zone (lz-20220910-oldev)$ kpt fn render

Package "landing-zone/environments/common/guardrails-policies":
Package "landing-zone/environments/common":
[RUNNING] "gcr.io/kpt-fn/set-namespace:v0.4.1"
[PASS] "gcr.io/kpt-fn/set-namespace:v0.4.1" in 3.1s
  Results:
    [info]: namespace "common" updated to "config-control", 23 value(s) changed

Package "landing-zone/environments/nonprod":
[RUNNING] "gcr.io/kpt-fn/set-namespace:v0.4.1"
[PASS] "gcr.io/kpt-fn/set-namespace:v0.4.1" in 400ms
  Results:
    [info]: namespace "nonprod" updated to "config-control", 7 value(s) changed

Package "landing-zone/environments/prod":
[RUNNING] "gcr.io/kpt-fn/enable-gcp-services:v0.1.0"
[FAIL] "gcr.io/kpt-fn/enable-gcp-services:v0.1.0" in 700ms
  Stderr:
    "docker: Error response from daemon: Get \"https://gcr.io/v2/\": dial tcp [2607:f8b0:400c:c03::52]:443: connect: cannot assign requested address."
    "See 'docker run --help'."
  Exit code: 125

run 2nd time

michael@cloudshell:~/github/GoogleCloudPlatform/20220909-103/landing-zone (lz-20220910-oldev)$ kpt fn render
Package "landing-zone/environments/common/guardrails-policies":
Package "landing-zone/environments/common":
[RUNNING] "gcr.io/kpt-fn/set-namespace:v0.4.1"
[PASS] "gcr.io/kpt-fn/set-namespace:v0.4.1" in 500ms
  Results:
    [info]: namespace "common" updated to "config-control", 23 value(s) changed

Package "landing-zone/environments/nonprod":
[RUNNING] "gcr.io/kpt-fn/set-namespace:v0.4.1"
[PASS] "gcr.io/kpt-fn/set-namespace:v0.4.1" in 400ms
  Results:
    [info]: namespace "nonprod" updated to "config-control", 7 value(s) changed

Package "landing-zone/environments/prod":
[RUNNING] "gcr.io/kpt-fn/enable-gcp-services:v0.1.0"
[PASS] "gcr.io/kpt-fn/enable-gcp-services:v0.1.0" in 3.2s
  Results:
    [info] serviceusage.cnrm.cloud.google.com/v1beta1/Service/config-control/prod-nethost-service-compute: generated service
    [info] serviceusage.cnrm.cloud.google.com/v1beta1/Service/config-control/prod-nethost-service-logging: generated service
[RUNNING] "gcr.io/kpt-fn/set-namespace:v0.4.1"
[PASS] "gcr.io/kpt-fn/set-namespace:v0.4.1" in 400ms
  Results:
    [info]: namespace "prod" updated to "config-control", 4 value(s) changed

Package "landing-zone":
[RUNNING] "gcr.io/kpt-fn/apply-setters:v0.2"
[PASS] "gcr.io/kpt-fn/apply-setters:v0.2" in 2s
  Results:
    [info] metadata.annotations.cnrm.cloud.google.com/organization-id: set field value to "583675367868"
    [info] metadata.annotations.cnrm.cloud.google.com/organization-id: set field value to "583675367868"
    [info] spec.projectID: set field value to "net-perimeter-prj-common-old1"
    [info] spec.parentRef.external: set field value to "583675367868"
    ...(87 line(s) truncated, use '--truncate-output=false' to disable)
[RUNNING] "gcr.io/kpt-fn/generate-folders:v0.1.1"
[FAIL] "gcr.io/kpt-fn/generate-folders:v0.1.1" in 700ms
  Stderr:
    "docker: Error response from daemon: Get \"https://gcr.io/v2/\": dial tcp [2607:f8b0:400c:c04::52]:443: connect: cannot assign requested address."
    "See 'docker run --help'."
  Exit code: 125


cluster still warming up after 1 hour idle time (or increase the vCores ram past 2/3

michael@cloudshell:~/github/GoogleCloudPlatform/20220909-103/landing-zone (lz-20220910-oldev)$ kpt fn render
Package "landing-zone/environments/common/guardrails-policies":
Package "landing-zone/environments/common":
[RUNNING] "gcr.io/kpt-fn/set-namespace:v0.4.1"
[PASS] "gcr.io/kpt-fn/set-namespace:v0.4.1" in 500ms
  Results:
    [info]: namespace "common" updated to "config-control", 23 value(s) changed

Package "landing-zone/environments/nonprod":
[RUNNING] "gcr.io/kpt-fn/set-namespace:v0.4.1"
[PASS] "gcr.io/kpt-fn/set-namespace:v0.4.1" in 400ms
  Results:
    [info]: namespace "nonprod" updated to "config-control", 7 value(s) changed

Package "landing-zone/environments/prod":
[RUNNING] "gcr.io/kpt-fn/enable-gcp-services:v0.1.0"
[PASS] "gcr.io/kpt-fn/enable-gcp-services:v0.1.0" in 1.5s
  Results:
    [info] serviceusage.cnrm.cloud.google.com/v1beta1/Service/config-control/prod-nethost-service-compute: generated service
    [info] serviceusage.cnrm.cloud.google.com/v1beta1/Service/config-control/prod-nethost-service-logging: generated service
[RUNNING] "gcr.io/kpt-fn/set-namespace:v0.4.1"
[PASS] "gcr.io/kpt-fn/set-namespace:v0.4.1" in 400ms
  Results:
    [info]: namespace "prod" updated to "config-control", 4 value(s) changed

Package "landing-zone":
[RUNNING] "gcr.io/kpt-fn/apply-setters:v0.2"
[PASS] "gcr.io/kpt-fn/apply-setters:v0.2" in 600ms
  Results:
    [info] metadata.annotations.cnrm.cloud.google.com/organization-id: set field value to "583675367868"
    [info] metadata.annotations.cnrm.cloud.google.com/organization-id: set field value to "583675367868"
    [info] spec.projectID: set field value to "net-perimeter-prj-common-old1"
    [info] spec.parentRef.external: set field value to "583675367868"
    ...(87 line(s) truncated, use '--truncate-output=false' to disable)
[RUNNING] "gcr.io/kpt-fn/generate-folders:v0.1.1"
[PASS] "gcr.io/kpt-fn/generate-folders:v0.1.1" in 5.4s
[RUNNING] "gcr.io/kpt-fn/enable-gcp-services:v0.1.0"
[PASS] "gcr.io/kpt-fn/enable-gcp-services:v0.1.0" in 1.8s
  Results:
    [info] serviceusage.cnrm.cloud.google.com/v1beta1/Service/config-control/nonprod-nethost-service-compute: generated service
    [info] serviceusage.cnrm.cloud.google.com/v1beta1/Service/config-control/nonprod-nethost-service-dns: generated service
    [info] serviceusage.cnrm.cloud.google.com/v1beta1/Service/config-control/nonprod-nethost-service-logging: generated service
    [info] serviceusage.cnrm.cloud.google.com/v1beta1/Service/config-control/prod-nethost-service-compute: recreated service
    ...(3 line(s) truncated, use '--truncate-output=false' to disable)
[RUNNING] "gcr.io/kpt-fn/gatekeeper:v0.2.1"
[PASS] "gcr.io/kpt-fn/gatekeeper:v0.2.1" in 3.5s
[RUNNING] "gcr.io/kpt-fn/kubeval:v0.3.0"
[PASS] "gcr.io/kpt-fn/kubeval:v0.3.0" in 22.5s

Successfully executed 9 function(s) in 5 package(s).

ok now

cd ..

kpt live init landing-zone --namespace config-control
kpt live apply landing-zone --reconcile-timeout=2m --output=table

config-con  StorageBucket/log-bucket-audit-prj-id-ol  Unchanged     NotFound                <None>                                    -       Resource not found

1 resources failed

@obriensystems obriensystems mentioned this issue Sep 10, 2022
Open
@fmichaelobrien fmichaelobrien added the bug Something isn't working label Sep 11, 2022
@fmichaelobrien fmichaelobrien self-assigned this Sep 13, 2022
fmichaelobrien added a commit that referenced this issue Sep 13, 2022
@fmichaelobrien
#111 - fix kpt steps for in/out lz dir
e6cef90 
see #111 (comment)

before
```
michael@cloudshell:~/github/GoogleCloudPlatform/20220909-103/landing-zone (lz-20220910-oldev)$ cd landing-zone/
-bash: cd: landing-zone/: No such file or directory
```

after
```
michael@cloudshell:~/github/GoogleCloudPlatform/20220909-103 (lz-20220910-oldev)$ cd landing-zone/
michael@cloudshell:~/github/GoogleCloudPlatform/20220909-103/landing-zone (lz-20220910-oldev)$ kpt fn render
Package "landing-zone/environments/common/guardrails-policies":
Successfully executed 9 function(s) in 5 package(s).
michael@cloudshell:~/github/GoogleCloudPlatform/20220909-103/landing-zone (lz-20220910-oldev)$ cd ..
michael@cloudshell:~/github/GoogleCloudPlatform/20220909-103 (lz-20220910-oldev)$ kpt live init landing-zone --namespace config-control
initializing Kptfile inventory info (namespace: config-control)...failed
michael@cloudshell:~/github/GoogleCloudPlatform/20220909-103 (lz-20220910-oldev)$ kpt live apply landing-zone --reconcile-timeout=2m
installing inventory ResourceGroup CRD.
```
@fmichaelobrien fmichaelobrien mentioned this issue Sep 13, 2022
Merged
@fmichaelobrien
Copy link
Contributor

right, better - explicit dir

michael@cloudshell:~/github/GoogleCloudPlatform/20220909-103 (lz-20220910-oldev)$ kpt fn render landing-zone
Package "landing-zone/environments/common/guardrails-policies":

fmichaelobrien added a commit that referenced this issue Sep 13, 2022
@fmichaelobrien
#111 - explicit dir is better
af627da 
right, better - explicit dir
```
michael@cloudshell:~/github/GoogleCloudPlatform/20220909-103 (lz-20220910-oldev)$ kpt fn render landing-zone
Package "landing-zone/environments/common/guardrails-policies":
```
cartyc added a commit that referenced this issue Sep 13, 2022
@cartyc
Merge pull request #123 from GoogleCloudPlatform/fmichaelobrien-patch…
38116f6 
…-111-3

#111 - fix kpt steps for in/out lz dir
@obriensystems
Copy link
Collaborator Author

fixed

@obriensystems obriensystems mentioned this issue Sep 13, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@fmichaelobrien fmichaelobrien

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@obriensystems @fmichaelobrien