-
Notifications
You must be signed in to change notification settings - Fork 2k
STOR-1988: add workload identity tests to GCP Filestore operator #56184
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
openshift-merge-bot
merged 1 commit into
openshift:master
from
RomanBednar:add-gcp-wif-tests
Nov 21, 2024
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
14 changes: 14 additions & 0 deletions
14
ci-operator/step-registry/ipi/conf/gcp/filestore-workload-identity-federation-cleanup/OWNERS
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| approvers: | ||
| - dobsonj | ||
| - gnufied | ||
| - jsafrane | ||
| - RomanBednar | ||
| - tsmetana | ||
| - mpatlasov | ||
| reviewers: | ||
| - dobsonj | ||
| - gnufied | ||
| - jsafrane | ||
| - RomanBednar | ||
| - tsmetana | ||
| - mpatlasov |
71 changes: 71 additions & 0 deletions
71
...ederation-cleanup/ipi-conf-gcp-filestore-workload-identity-federation-cleanup-commands.sh
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,71 @@ | ||
| #!/bin/bash | ||
| set -o errexit | ||
| set -o nounset | ||
| set -o pipefail | ||
|
|
||
| # logger function prints standard logs | ||
| logger() { | ||
| local level="$1" | ||
| local message="$2" | ||
| local timestamp | ||
|
|
||
| # Generate a timestamp for the log entry | ||
| timestamp=$(date +"%Y-%m-%d %H:%M:%S") | ||
|
|
||
| # Print the log message with the level and timestamp | ||
| echo "[$timestamp] [$level] $message" | ||
| } | ||
|
|
||
| function run_command() { | ||
| local CMD="$1" | ||
| echo "Running Command: ${CMD}" | ||
| eval "${CMD}" | ||
| } | ||
|
|
||
| logger "INFO" "Starting GCP Filestore Workload Identity Federation cleanup" | ||
|
|
||
| if [ -f "${SHARED_DIR}/gcp-filestore-service-account-email" ]; then | ||
| SERVICE_ACCOUNT_EMAIL=$(cat "${SHARED_DIR}"/gcp-filestore-service-account-email) | ||
| else | ||
| logger "INFO" "Service account email file not found in ${SHARED_DIR} - nothing to clean up." | ||
| exit 0 | ||
| fi | ||
|
|
||
| # For disconnected or otherwise unreachable environments, we want to | ||
| # have steps use an HTTP(S) proxy to reach the API server. This proxy | ||
| # configuration file should export HTTP_PROXY, HTTPS_PROXY, and NO_PROXY | ||
| # environment variables, as well as their lowercase equivalents (note | ||
| # that libcurl doesn't recognize the uppercase variables). | ||
| if test -f "${SHARED_DIR}/proxy-conf.sh" | ||
| then | ||
| # shellcheck disable=SC1091 | ||
| source "${SHARED_DIR}/proxy-conf.sh" | ||
| logger "INFO" "Loaded proxy configuration from ${SHARED_DIR}/proxy-conf.sh" | ||
| fi | ||
|
|
||
| GOOGLE_PROJECT_ID="$(<${CLUSTER_PROFILE_DIR}/openshift_gcp_project)" | ||
| export GCP_SHARED_CREDENTIALS_FILE="${CLUSTER_PROFILE_DIR}/gce.json" | ||
| sa_email=$(jq -r .client_email ${GCP_SHARED_CREDENTIALS_FILE}) | ||
| if ! gcloud auth list | grep -E "\*\s+${sa_email}" | ||
| then | ||
| logger "INFO" "Activating service account: ${sa_email}" | ||
| cmd="gcloud auth activate-service-account --key-file=\"${GCP_SHARED_CREDENTIALS_FILE}\"" | ||
| run_command "$cmd" | ||
| cmd="gcloud config set project \"${GOOGLE_PROJECT_ID}\"" | ||
| run_command "$cmd" | ||
| logger "INFO" "Service account activated and project set to ${GOOGLE_PROJECT_ID}" | ||
| fi | ||
|
|
||
| # Ref: TBD (no Red Hat docs available yet). Google doc: https://cloud.google.com/iam/docs/workload-identity-federation-with-kubernetes#create_the_workload_identity_pool_and_provider | ||
| logger "INFO" "Starting cleanup of GCP Filestore cloud infrastructure for Workload Identity Federation" | ||
|
|
||
| ## TODO: replace cleanup steps with ccoctl automation if this ever gets implemented | ||
| ## TODO: alternatively, this could be documented later in the docs, make sure the code below is aligned with the official procedure | ||
|
|
||
| # Delete the Google cloud service account | ||
| logger "INFO" "Deleting Google cloud service account: ${SERVICE_ACCOUNT_EMAIL}" | ||
| cmd="gcloud --quiet iam service-accounts delete \"$SERVICE_ACCOUNT_EMAIL\"" | ||
| run_command "$cmd" | ||
| logger "INFO" "Service account removed" | ||
|
|
||
| logger "INFO" "GCP Filestore Workload Identity Federation cleanup completed" |
21 changes: 21 additions & 0 deletions
21
...ion-cleanup/ipi-conf-gcp-filestore-workload-identity-federation-cleanup-ref.metadata.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| { | ||
| "path": "ipi/conf/gcp/filestore-workload-identity-federation-cleanup/ipi-conf-gcp-filestore-workload-identity-federation-cleanup-ref.yaml", | ||
| "owners": { | ||
| "approvers": [ | ||
| "dobsonj", | ||
| "gnufied", | ||
| "jsafrane", | ||
| "RomanBednar", | ||
| "tsmetana", | ||
| "mpatlasov" | ||
| ], | ||
| "reviewers": [ | ||
| "dobsonj", | ||
| "gnufied", | ||
| "jsafrane", | ||
| "RomanBednar", | ||
| "tsmetana", | ||
| "mpatlasov" | ||
| ] | ||
| } | ||
| } |
15 changes: 15 additions & 0 deletions
15
...y-federation-cleanup/ipi-conf-gcp-filestore-workload-identity-federation-cleanup-ref.yaml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| ref: | ||
| as: ipi-conf-gcp-filestore-workload-identity-federation-cleanup | ||
| from_image: | ||
| namespace: ocp | ||
| name: "4.12" | ||
| tag: upi-installer | ||
| # inject oc binary | ||
| cli: latest | ||
| commands: ipi-conf-gcp-filestore-workload-identity-federation-cleanup-commands.sh | ||
| resources: | ||
| requests: | ||
| cpu: 10m | ||
| memory: 100Mi | ||
| documentation: |- | ||
| The script cleans up GCP Filestore Workload Identity Federation resources and service accounts. |
14 changes: 14 additions & 0 deletions
14
ci-operator/step-registry/ipi/conf/gcp/filestore-workload-identity-federation/OWNERS
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| approvers: | ||
| - dobsonj | ||
| - gnufied | ||
| - jsafrane | ||
| - RomanBednar | ||
| - tsmetana | ||
| - mpatlasov | ||
| reviewers: | ||
| - dobsonj | ||
| - gnufied | ||
| - jsafrane | ||
| - RomanBednar | ||
| - tsmetana | ||
| - mpatlasov |
110 changes: 110 additions & 0 deletions
110
...kload-identity-federation/ipi-conf-gcp-filestore-workload-identity-federation-commands.sh
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,110 @@ | ||
| #!/bin/bash | ||
| set -o errexit | ||
| set -o nounset | ||
| set -o pipefail | ||
|
|
||
| INFRA_NAME=${NAMESPACE}-${UNIQUE_HASH} | ||
|
|
||
| # logger function prints standard logs | ||
| logger() { | ||
| local level="$1" | ||
| local message="$2" | ||
| local timestamp | ||
|
|
||
| # Generate a timestamp for the log entry | ||
| timestamp=$(date +"%Y-%m-%d %H:%M:%S") | ||
|
|
||
| # Print the log message with the level and timestamp | ||
| echo "[$timestamp] [$level] $message" | ||
| } | ||
|
|
||
| function run_command() { | ||
| local CMD="$1" | ||
| echo "Running Command: ${CMD}" >&2 | ||
| eval "${CMD}" | ||
| } | ||
|
|
||
| logger "INFO" "Starting GCP Filestore Workload Identity Federation configuration" | ||
|
|
||
| # For disconnected or otherwise unreachable environments, we want to | ||
| # have steps use an HTTP(S) proxy to reach the API server. This proxy | ||
| # configuration file should export HTTP_PROXY, HTTPS_PROXY, and NO_PROXY | ||
| # environment variables, as well as their lowercase equivalents (note | ||
| # that libcurl doesn't recognize the uppercase variables). | ||
| if test -f "${SHARED_DIR}/proxy-conf.sh" | ||
| then | ||
| # shellcheck disable=SC1091 | ||
| source "${SHARED_DIR}/proxy-conf.sh" | ||
| logger "INFO" "Loaded proxy configuration from ${SHARED_DIR}/proxy-conf.sh" | ||
| fi | ||
|
|
||
| GOOGLE_PROJECT_ID="$(<${CLUSTER_PROFILE_DIR}/openshift_gcp_project)" | ||
| export GCP_SHARED_CREDENTIALS_FILE="${CLUSTER_PROFILE_DIR}/gce.json" | ||
| GCP_SERVICE_ACCOUNT=$(jq -r .client_email ${GCP_SHARED_CREDENTIALS_FILE}) | ||
| SA_SUFFIX=${GCP_SERVICE_ACCOUNT#*@} | ||
| SA_EMAIL=$(jq -r .client_email ${GCP_SHARED_CREDENTIALS_FILE}) | ||
|
|
||
| if ! gcloud auth list | grep -E "\*\s+${SA_EMAIL}" | ||
| then | ||
| logger "INFO" "Authenticating with GCP service account" | ||
| CMD="gcloud auth activate-service-account --key-file=\"${GCP_SHARED_CREDENTIALS_FILE}\"" | ||
| run_command "${CMD}" | ||
| CMD="gcloud config set project \"${GOOGLE_PROJECT_ID}\"" | ||
| run_command "${CMD}" | ||
| logger "INFO" "Successfully authenticated with GCP service account" | ||
| fi | ||
|
|
||
| # Ref: TBD (no Red Hat docs available yet). Google doc: https://cloud.google.com/iam/docs/workload-identity-federation-with-kubernetes#create_the_workload_identity_pool_and_provider | ||
| logger "INFO" "Create GCP Filestore cloud infrastructure for Workload Identity Federation" | ||
|
|
||
| ## TODO: replace steps to manually create the service account and bindings with ccoctl automation if this ever gets implemented | ||
| ## TODO: alternatively, this could be documented later in the docs, make sure the code below is aligned with the official procedure | ||
|
|
||
| # Create Google cloud service account for GCP Filestore Operator (name length must be between 6 and 30) | ||
| SERVICE_ACCOUNT_NAME="gcp-filestore-sa-${UNIQUE_HASH}"-`echo $RANDOM` | ||
| SERVICE_ACCOUNT_EMAIL="${SERVICE_ACCOUNT_NAME}@${SA_SUFFIX}" | ||
|
|
||
| logger "INFO" "Creating GCP IAM service account: ${SERVICE_ACCOUNT_EMAIL}" | ||
| CMD="gcloud iam service-accounts create \"$SERVICE_ACCOUNT_NAME\" --display-name=\"$SERVICE_ACCOUNT_NAME\"" | ||
| run_command "$CMD" | ||
|
|
||
| # Obtain project number, pool ID, and provider ID | ||
| # We assume the pool ID is the same as the infrastructure name and reuse it. If this won't work well in the future we can create a new pool for the operator. | ||
| logger "INFO" "Obtaining project details and identity pool information" | ||
| CMD="gcloud projects describe \"$GOOGLE_PROJECT_ID\" --format=\"value(projectNumber)\"" | ||
| PROJECT_NUMBER=$(run_command "${CMD}") | ||
| POOL_ID=${INFRA_NAME} | ||
| PROVIDER_ID=${INFRA_NAME} | ||
| logger "INFO" "Project number: ${PROJECT_NUMBER}, Pool ID: ${POOL_ID}, Provider ID: ${PROVIDER_ID}" | ||
|
|
||
| # Set roles for the service account - this should match roles CredentialsRequest of Filestore Operator | ||
| logger "INFO" "Setting IAM roles for the service account" | ||
| CMD="gcloud projects add-iam-policy-binding \"$GOOGLE_PROJECT_ID\" --member=\"serviceAccount:$SERVICE_ACCOUNT_EMAIL\" --role=\"roles/file.editor\" --condition=None" | ||
| run_command "${CMD}" | ||
| CMD="gcloud projects add-iam-policy-binding \"$GOOGLE_PROJECT_ID\" --member=\"serviceAccount:$SERVICE_ACCOUNT_EMAIL\" --role=\"roles/resourcemanager.tagUser\" --condition=None" | ||
| run_command "${CMD}" | ||
| logger "INFO" "IAM roles set successfully" | ||
|
|
||
| # Allow OpenShift service accounts to impersonate Google cloud service account | ||
| logger "INFO" "Configuring Workload Identity Federation for OpenShift service accounts" | ||
| CMD="gcloud iam service-accounts add-iam-policy-binding \"$SERVICE_ACCOUNT_EMAIL\" --member=\"principal://iam.googleapis.com/projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/subject/system:serviceaccount:openshift-cluster-csi-drivers:gcp-filestore-csi-driver-controller-sa\" --role=roles/iam.workloadIdentityUser" | ||
| run_command "${CMD}" | ||
| CMD="gcloud iam service-accounts add-iam-policy-binding \"$SERVICE_ACCOUNT_EMAIL\" --member=\"principal://iam.googleapis.com/projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/subject/system:serviceaccount:openshift-cluster-csi-drivers:gcp-filestore-csi-driver-operator\" --role=roles/iam.workloadIdentityUser" | ||
| run_command "${CMD}" | ||
| logger "INFO" "Workload Identity Federation configured successfully" | ||
|
|
||
| # Store GCP WIF variables to a known location to be used later in chain as OO_CONFIG_ENVVARS used by `optional-operators-subscribe` step | ||
| logger "INFO" "Storing GCP Workload Identity Federation variables" | ||
| echo "$POOL_ID" > "${SHARED_DIR}"/gcp-filestore-pool-id | ||
| echo "$PROVIDER_ID" > "${SHARED_DIR}"/gcp-filestore-provider-id | ||
| echo "$SERVICE_ACCOUNT_EMAIL" > "${SHARED_DIR}"/gcp-filestore-service-account-email | ||
| printf '"%s"' "$PROJECT_NUMBER" > "${SHARED_DIR}"/gcp-filestore-project-number | ||
|
|
||
| logger "INFO" "GCP Workload Identity Federation variables stored successfully in ${SHARED_DIR}:" | ||
| logger "INFO" " Pool ID: $(cat ${SHARED_DIR}/gcp-filestore-pool-id)" | ||
| logger "INFO" " Provider ID: $(cat ${SHARED_DIR}/gcp-filestore-provider-id)" | ||
| logger "INFO" " Service Account Email: $(cat ${SHARED_DIR}/gcp-filestore-service-account-email)" | ||
| logger "INFO" " Project Number: $(cat ${SHARED_DIR}/gcp-filestore-project-number)" | ||
|
|
||
| logger "INFO" "GCP Filestore Workload Identity Federation configuration completed" | ||
|
|
21 changes: 21 additions & 0 deletions
21
...identity-federation/ipi-conf-gcp-filestore-workload-identity-federation-ref.metadata.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| { | ||
| "path": "ipi/conf/gcp/filestore-workload-identity-federation/ipi-conf-gcp-filestore-workload-identity-federation-ref.yaml", | ||
| "owners": { | ||
| "approvers": [ | ||
| "dobsonj", | ||
| "gnufied", | ||
| "jsafrane", | ||
| "RomanBednar", | ||
| "tsmetana", | ||
| "mpatlasov" | ||
| ], | ||
| "reviewers": [ | ||
| "dobsonj", | ||
| "gnufied", | ||
| "jsafrane", | ||
| "RomanBednar", | ||
| "tsmetana", | ||
| "mpatlasov" | ||
| ] | ||
| } | ||
| } |
16 changes: 16 additions & 0 deletions
16
...workload-identity-federation/ipi-conf-gcp-filestore-workload-identity-federation-ref.yaml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| ref: | ||
| as: ipi-conf-gcp-filestore-workload-identity-federation | ||
| from_image: | ||
| namespace: ocp | ||
| name: "4.12" | ||
| tag: upi-installer | ||
| # inject oc binary | ||
| cli: latest | ||
| commands: ipi-conf-gcp-filestore-workload-identity-federation-commands.sh | ||
| resources: | ||
| requests: | ||
| cpu: 10m | ||
| memory: 100Mi | ||
| documentation: |- | ||
| The script automates the setup of GCP Workload Identity Federation for OpenShift's Filestore service by creating and | ||
| configuring necessary service accounts, permissions, and identity bindings between GCP and OpenShift environments. | ||
7 changes: 7 additions & 0 deletions
7
ci-operator/step-registry/ipi/gcp/post/manual-oidc-workload-identity-filestore/OWNERS
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| approvers: | ||
| - dobsonj | ||
| - gnufied | ||
| - jsafrane | ||
| - RomanBednar | ||
| - tsmetana | ||
| - mpatlasov |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't this be a newer version? Where does
upi-installercome from?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The step just needs to call gcp sdk, upi-installer image installed the sdk, but it does not care about the exact version of upi-installer, because all versions of upi-installer installed the gcp sdk.