New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CFE-857: Apply user defined tags on created gcp resources #7279

Merged

openshift-merge-bot merged 3 commits into openshift:master from bharath-b-rh:cfe-857

Mar 1, 2024

Contributor

bharath-b-rh commented Jun 25, 2023 •

edited

PR is for adding user-defined tags to the GCP resources created by installer during cluster creation.

Terraform APIs require the TagValue resource Name and it needs to be derived using the parameters OrganizationID, UserTag.Key, UserTag.Value defined by user in install-config.
Logic for deriving the TagValues Name is implemented using golang, which happens during install-config validation or while generating the terraform variables.
Terraform code has the logic for adding tags to the resources during (for compute resources) and after it's creation(for storage buckets).

bharath-b-rh marked this pull request as draft

June 25, 2023 12:55

Contributor Author

bharath-b-rh commented Jun 25, 2023

/hold openshift/enhancements#1217

openshift-ci bot added do-not-merge/work-in-progress do-not-merge/hold labels

openshift-ci bot requested review from patrickdillon and pawanpinjarkar

June 25, 2023 12:56

bharath-b-rh force-pushed the cfe-857 branch 2 times, most recently from c6b0254 to e67b432 Compare

July 13, 2023 07:33

bharath-b-rh force-pushed the cfe-857 branch 2 times, most recently from 07652d2 to 80da9cd Compare

July 21, 2023 07:47

openshift-merge-robot added the needs-rebase label

bharath-b-rh force-pushed the cfe-857 branch 2 times, most recently from 0170f96 to 5bd3416 Compare

July 24, 2023 11:30

openshift-merge-robot removed the needs-rebase label

Contributor

patrickdillon commented Jul 24, 2023

You will want to separate the go vendoring into a separate commit (like this https://github.com/openshift/installer/blob/master/docs/dev/dependencies.md#go) to make it easier to review

bharath-b-rh force-pushed the cfe-857 branch from 3d65d4a to f81ab43 Compare

July 25, 2023 05:12

Contributor Author

bharath-b-rh commented Jul 25, 2023

You will want to separate the go vendoring into a separate commit (like this https://github.com/openshift/installer/blob/master/docs/dev/dependencies.md#go) to make it easier to review

Thank you for reviewing @patrickdillon!

I have split the go mod changes into two for go.mod update and for adding new vendor code.

bharath-b-rh force-pushed the cfe-857 branch from 7288771 to 6ee6e20 Compare

August 2, 2023 06:46

openshift-merge-robot added the needs-rebase label

bharath-b-rh force-pushed the cfe-857 branch 2 times, most recently from 5658f2b to bfac051 Compare

August 2, 2023 10:42

openshift-merge-robot removed the needs-rebase label

bharath-b-rh changed the title ~~CFE-857 : Apply user defined tags on created gcp resources~~ CFE-857: Apply user defined tags on created gcp resources

openshift-ci-robot added the jira/valid-reference label

Contributor

openshift-ci-robot commented Aug 9, 2023 •

edited by openshift-ci bot

@bharath-b-rh: This pull request references CFE-857 which is a valid jira issue.

In response to this:

PR is for adding user-defined tags to the GCP resources created by installer during cluster creation.

Terraform APIs require the TagValue resource Name and it needs to be derived using the parameters OrganizationID, UserTag.Key, UserTag.Value defined by user in install-config.

Logic for deriving the TagValues Name is implemented using golang, which happens while generating the terraform variables.

Terraform code has the logic for adding tags to the resources after it's creation.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Contributor Author

bharath-b-rh commented Aug 9, 2023

/remove-hold

bharath-b-rh marked this pull request as ready for review

August 9, 2023 12:04

openshift-ci bot removed do-not-merge/hold do-not-merge/work-in-progress labels

openshift-ci bot requested a review from r4f4

August 9, 2023 12:14

bharath-b-rh force-pushed the cfe-857 branch from 3c3ee7d to f01b5a4 Compare

January 30, 2024 07:34

Contributor Author

bharath-b-rh commented Jan 30, 2024

/retest-required

r4f4 reviewed

View reviewed changes

pkg/asset/installconfig/gcp/usertags.go Outdated Show resolved Hide resolved

pkg/asset/installconfig/gcp/usertags.go Outdated Show resolved Hide resolved

pkg/asset/cluster/tfvars/tfvars.go Outdated Show resolved Hide resolved

Contributor

bfournie commented Jan 30, 2024

openshift-ci bot requested a review from bfournie

January 30, 2024 20:55

r4f4 reviewed

View reviewed changes

pkg/asset/installconfig/gcp/usertags.go Outdated Show resolved Hide resolved

pkg/asset/installconfig/gcp/usertags.go Outdated

Comment on lines 187 to 184

+              			if errors.As(err, &gErr) && (gErr.HTTPCode() == http.StatusNotFound ||
+              				gErr.HTTPCode() == http.StatusForbidden) {
+              				logrus.Debugf("does not have permission to access %s tag or does not exist", name)

Contributor

r4f4 Jan 31, 2024

Why give an ambiguous message if you can identify the 2 scenarios?

Suggested change

      
            			if errors.As(err, &gErr) && (gErr.HTTPCode() == http.StatusNotFound ||
          
            				gErr.HTTPCode() == http.StatusForbidden) {
          
            				logrus.Debugf("does not have permission to access %s tag or does not exist", name)
          
            			if errors.As(err, &gErr) {
          
            			    switch gErr.HTTPCode() {
          
            			    case http.StatusNotFound:
          
            			        logrus.Debugf("tag %s does not exist: %s", name, err.Error())
          
            				case http.StatusForbidden:
          
            				    logrus.Debugf("permission issues accessing tag %s: %s", name, err.Error())
          
            				}

Contributor Author

bharath-b-rh Feb 1, 2024 •

edited

Error returned by the GCP APIs is also similar. I think it's because RBAC can be defined on each tag key and value, and with a given service account, it won't be possible to say whether the tag exists and is inaccessible or it doesn't exist at all, so thought to keep the same. Below error was returned for a tag value which didn't exist.

User [xyz] does not have permission to access tagValues instance [namespaced] (or it may not exist): Permission denied on resource 'openshift/key/value' (or it may not exist).

Contributor Author

bharath-b-rh Feb 8, 2024

Code updated, new API of cloudresourcemanager package returns StatusForbidden for both non-existent and inaccessible tags.

pkg/asset/installconfig/gcp/usertags.go Outdated

+              // getProcessedTags returns the list user-provided validated tags.
+              func getProcessedTags(mgr TagManager, ic *types.InstallConfig) (map[string]string, error) {
+              	if !processedTags.getState() {
+              		if err := validateUserTags(mgr, ic); err != nil {

Contributor

r4f4 Jan 31, 2024

It's a bit weird that the definition for validateUserTags lives in the validation.go file and it basically just calls validateAndStoreUserTags which is defined in this file. Also, I don't see why this function has to receive the InstallConfig object if it only needs the project name and the userTags to do its job.

Contributor Author

bharath-b-rh Feb 1, 2024

Yeah, I missed this unnecessary call while incorporating the comments, I will update it.
I thought to pass the InstallConfig object reference and use what's required, instead of passing individual parameters. Please let me know, if needs to be changed.

pkg/asset/installconfig/gcp/usertags.go Outdated Show resolved Hide resolved

pkg/asset/installconfig/gcp/usertags.go Outdated Show resolved Hide resolved

openshift-merge-robot added the needs-rebase label

bharath-b-rh force-pushed the cfe-857 branch from d35d9f0 to 44e22c4 Compare

February 7, 2024 14:44

openshift-merge-robot removed the needs-rebase label

bharath-b-rh force-pushed the cfe-857 branch from 44e22c4 to 8450fef Compare

February 8, 2024 06:32

bharath-b-rh commented

View reviewed changes

pkg/asset/installconfig/gcp/mock/usertags_mock.go

		@@ -0,0 +1,83 @@
		// Code generated by MockGen. DO NOT EDIT.

Contributor Author

bharath-b-rh Feb 8, 2024

This code will be merged with mock code in pkg/asset/installconfig/gcp/mock/gcpclient_generated.go after updating google.golang.org/api/cloudresourcemanager from v1 to v3 in the follow up PR.

bharath-b-rh commented

View reviewed changes

pkg/asset/installconfig/gcp/usertags.go

+              	return dupTags
+              }
+              // getCloudResourceServiceForTags returns the client required for querying resource manager resources.

Contributor Author

bharath-b-rh Feb 8, 2024

Code will be merged with client code in pkg/asset/installconfig/gcp/client.go after updating google.golang.org/api/cloudresourcemanager from v1 to v3 in the follow up PR.

Contributor

openshift-ci-robot commented Feb 8, 2024 •

edited by openshift-ci bot

@bharath-b-rh: This pull request references CFE-857 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.16.0" version, but no target version was set.

In response to this:

PR is for adding user-defined tags to the GCP resources created by installer during cluster creation.

Terraform APIs require the TagValue resource Name and it needs to be derived using the parameters OrganizationID, UserTag.Key, UserTag.Value defined by user in install-config.

Logic for deriving the TagValues Name is implemented using golang, which happens during install-config validation or while generating the terraform variables.

Terraform code has the logic for adding tags to the resources during (for compute resources) and after it's creation(for storage buckets).

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

bharath-b-rh force-pushed the cfe-857 branch from 8450fef to da46904 Compare

February 8, 2024 11:16

Contributor

r4f4 commented Feb 8, 2024

LGTM in terms of the code but I haven't tested it myself. I'll leave it to @patrickdillon to give this a last look and tag it.

Contributor

patrickdillon commented Feb 16, 2024

Generally this looks good to me (but needs a rebase). We are moving away from Terraform and implementing a CAPI-based install pattern (see the enhancement for details). This code is already done so I am fine with merging it, but depending on the progress we make, it is possible we will not be using Terraform for GCP in 4.16. In order to support this feature going forward, it will be necessary for us to implement user tag support in the cluster-api-provider-gcp as it currently does not have support.

openshift-merge-robot added the needs-rebase label

bharath-b-rh added 3 commits

February 17, 2024 13:40


          CFE-857 : Update go mod with new pkgs for GCP tags

e5925c6


          CFE-857 : Add vendor pkgs for GCP tags

905abb4


          CFE-857 : Apply user defined tags on created gcp resources

1363d39

bharath-b-rh force-pushed the cfe-857 branch from da46904 to 1363d39 Compare

February 17, 2024 08:12

openshift-merge-robot removed the needs-rebase label

Contributor

openshift-ci bot commented Feb 17, 2024 •

edited

@bharath-b-rh: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-agent-ha-dualstack	`55b949a`	link	false	`/test e2e-agent-ha-dualstack`
ci/prow/okd-e2e-aws-ovn	`1f9a041`	link	false	`/test okd-e2e-aws-ovn`
ci/prow/okd-scos-e2e-aws-ovn	`1f9a041`	link	false	`/test okd-scos-e2e-aws-ovn`
ci/prow/terraform-images	`1f9a041`	link	true	`/test terraform-images`
ci/prow/terraform-verify-vendor	`1f9a041`	link	true	`/test terraform-verify-vendor`
ci/prow/e2e-aws-ovn-edge-zones	`d35d9f0`	link	false	`/test e2e-aws-ovn-edge-zones`
ci/prow/e2e-aws-custom-security-groups	`d35d9f0`	link	false	`/test e2e-aws-custom-security-groups`
ci/prow/altinfra-e2e-aws-ovn-shared-vpc-edge-zones	`d35d9f0`	link	false	`/test altinfra-e2e-aws-ovn-shared-vpc-edge-zones`
ci/prow/altinfra-e2e-aws-ovn-localzones	`d35d9f0`	link	false	`/test altinfra-e2e-aws-ovn-localzones`
ci/prow/altinfra-e2e-aws-custom-security-groups	`d35d9f0`	link	false	`/test altinfra-e2e-aws-custom-security-groups`
ci/prow/e2e-aws-ovn-shared-vpc-edge-zones	`d35d9f0`	link	false	`/test e2e-aws-ovn-shared-vpc-edge-zones`
ci/prow/altinfra-e2e-aws-ovn-single-node	`d35d9f0`	link	false	`/test altinfra-e2e-aws-ovn-single-node`
ci/prow/e2e-vsphere-zones	`d35d9f0`	link	false	`/test e2e-vsphere-zones`
ci/prow/altinfra-e2e-aws-ovn-wavelengthzones	`d35d9f0`	link	false	`/test altinfra-e2e-aws-ovn-wavelengthzones`
ci/prow/e2e-vsphere-ovn	`d35d9f0`	link	true	`/test e2e-vsphere-ovn`
ci/prow/e2e-aws-ovn-shared-vpc	`d35d9f0`	link	false	`/test e2e-aws-ovn-shared-vpc`
ci/prow/altinfra-e2e-aws-ovn-fips	`d35d9f0`	link	false	`/test altinfra-e2e-aws-ovn-fips`
ci/prow/e2e-aws-ovn-single-node	`d35d9f0`	link	false	`/test e2e-aws-ovn-single-node`
ci/prow/e2e-aws-ovn-imdsv2	`d35d9f0`	link	false	`/test e2e-aws-ovn-imdsv2`
ci/prow/e2e-aws-ovn-fips	`d35d9f0`	link	false	`/test e2e-aws-ovn-fips`
ci/prow/e2e-vsphere-zones-techpreview	`d35d9f0`	link	false	`/test e2e-vsphere-zones-techpreview`
ci/prow/okd-e2e-aws-ovn-upgrade	`1363d39`	link	false	`/test okd-e2e-aws-ovn-upgrade`
ci/prow/e2e-gcp-ovn-xpn	`1363d39`	link	false	`/test e2e-gcp-ovn-xpn`
ci/prow/e2e-gcp-ovn-shared-vpc	`1363d39`	link	false	`/test e2e-gcp-ovn-shared-vpc`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Contributor Author

bharath-b-rh commented Feb 19, 2024

/retest-required

Contributor Author

bharath-b-rh commented Feb 29, 2024

Generally this looks good to me (but needs a rebase). We are moving away from Terraform and implementing a CAPI-based install pattern (see the enhancement for details). This code is already done so I am fine with merging it, but depending on the progress we make, it is possible we will not be using Terraform for GCP in 4.16. In order to support this feature going forward, it will be necessary for us to implement user tag support in the cluster-api-provider-gcp as it currently does not have support.

Thank you! I have rebased the PR.
Tags support was added in capg as part of this issue, where GCP Cluster and Machine Spec API were updated to support tags. And presently, tags are added to Compute Instance created by the provider, need to check for any other resources managed by it.

Contributor

patrickdillon commented Mar 1, 2024

/lgtm

openshift-ci bot added the lgtm label

openshift-merge-bot bot merged commit 11fb96e into openshift:master

23 of 26 checks passed

bharath-b-rh deleted the cfe-857 branch

March 12, 2024 07:03

bharath-b-rh mentioned this pull request

CORS-3254: Update google.golang.org/api/cloudresourcemanager library version #8213

Merged

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment