-
Notifications
You must be signed in to change notification settings - Fork 4
DOC-1105 RPCN on GCP BYOVPC #238
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
16 commits
Select commit
Hold shift + click to select a range
ffcd8b8
draft process
asimms41 da7fc3e
added new page and updated config inline with review
asimms41 dbe3437
dev review comments
asimms41 8165e13
Merge branch 'main' into DOC-1105_GCP_on_BYOVPC
asimms41 cad67b6
update quickstart
asimms41 782dada
Hyperlint reviews
asimms41 8274252
updates for consistency
asimms41 7dbc3a7
review comments
asimms41 2491581
add missing role, reordering, update role bindings to double quotes
asimms41 a57fc8a
add text about placeholders
asimms41 25c8a40
PM review comments
asimms41 e16f4e7
Added detail on GKE master CIDR range
asimms41 04859e9
Add region to addresses command
asimms41 3b9b307
Update feature flag message
asimms41 55c235b
docs review comments
asimms41 29b0eb0
rewording
asimms41 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
124 changes: 124 additions & 0 deletions
124
modules/get-started/pages/cluster-types/byoc/gcp/enable-rpcn-byovpc-gcp.adoc
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,124 @@ | ||
| = Enable Redpanda Connect on an Existing BYOVPC Cluster on GCP | ||
| :description: Add Redpanda Connect to your existing BYOVPC cluster. | ||
| :page-beta: true | ||
|
|
||
| include::shared:partial$feature-flag-rpcn.adoc[] | ||
|
|
||
| To enable Redpanda Connect on an existing BYOVPC cluster, you must update your configuration. You can also create xref:get-started:cluster-types/byoc/gcp/vpc-byo-gcp.adoc[a new BYOVPC cluster] with Redpanda Connect already enabled. | ||
asimms41 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| Replace all `<placeholders>` with your own values. | ||
|
|
||
| . Create two new service accounts with the necessary permissions and roles. | ||
| + | ||
| .Show commands | ||
| [%collapsible] | ||
| ==== | ||
| ```bash | ||
| # Account used to check for and read secrets, which are required to create Redpanda Connect pipelines. | ||
|
|
||
| gcloud iam service-accounts create redpanda-connect-api \ | ||
| --display-name="Redpanda Connect API Service Account" | ||
|
|
||
| cat << EOT > redpanda-connect-api.role | ||
| { | ||
| "name": "redpanda_connect_api_role", | ||
| "title": "Redpanda Connect API Role", | ||
| "description": "Redpanda Connect API Role", | ||
| "includedPermissions": [ | ||
| "resourcemanager.projects.get", | ||
| "secretmanager.secrets.get", | ||
| "secretmanager.versions.access" | ||
asimms41 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| ], | ||
| } | ||
| EOT | ||
|
|
||
| gcloud iam roles create redpanda_connect_api_role --project=<service-project-id> --file redpanda-connect-api.role | ||
|
|
||
| gcloud projects add-iam-policy-binding <service-project-id> \ | ||
| --member="serviceAccount:redpanda-connect-api@<service-project-id>.iam.gserviceaccount.com" \ | ||
| --role="projects/<service-project-id>/roles/redpanda_connect_api_role" | ||
| ``` | ||
|
|
||
| ```bash | ||
| # Account used to retrieve secrets and create Redpanda Connect pipelines. | ||
|
|
||
| gcloud iam service-accounts create redpanda-connect \ | ||
| --display-name="Redpanda Connect Service Account" | ||
|
|
||
| cat << EOT > redpanda-connect.role | ||
| { | ||
| "name": "redpanda_connect_role", | ||
| "title": "Redpanda Connect Role", | ||
| "description": "Redpanda Connect Role", | ||
| "includedPermissions": [ | ||
| "resourcemanager.projects.get", | ||
| "secretmanager.versions.access" | ||
| ], | ||
| } | ||
| EOT | ||
|
|
||
| gcloud iam roles create redpanda_connect_role --project=<service-project-id> --file redpanda-connect.role | ||
|
|
||
| gcloud projects add-iam-policy-binding <service-project-id> \ | ||
| --member="serviceAccount:redpanda-connect@<service-project-id>.iam.gserviceaccount.com" \ | ||
| --role="projects/<service-project-id>/roles/redpanda_connect_role" | ||
| ``` | ||
| ==== | ||
|
|
||
| . Bind the service accounts. | ||
| + | ||
| The account ID of the GCP service account is used to configure service account bindings. This account ID is the local part of the email address for the GCP service account. For example, if the GCP service account is `my-gcp-sa@my-project.iam.gserviceaccount.com`, then the account ID is `my-gcp-sa`. | ||
| + | ||
| .Show commands | ||
| [%collapsible] | ||
| ==== | ||
| ``` | ||
| gcloud iam service-accounts add-iam-policy-binding <redpanda_connect_api-gcp-sa-account-id>@<service-project-id>.iam.gserviceaccount.com \ | ||
| --role roles/iam.workloadIdentityUser \ | ||
| --member "serviceAccount:<service-project-id>.svc.id.goog[redpanda-connect/<redpanda_connect_api-gcp-sa-account-id>]" | ||
| ``` | ||
| ``` | ||
| gcloud iam service-accounts add-iam-policy-binding <redpanda_connect-gcp-sa-account-id>@<service-project-id>.iam.gserviceaccount.com \ | ||
| --role roles/iam.workloadIdentityUser \ | ||
| --member "serviceAccount:<service-project-id>.svc.id.goog[redpanda-connect/<redpanda_connect-gcp-sa-account-id>]" | ||
| ``` | ||
| ==== | ||
|
|
||
| . Make a xref:api:ROOT:cloud-controlplane-api.adoc#patch-/v1/clusters/-cluster.id-[`PATCH /v1/clusters/\{cluster-id}`] request to update the cluster configuration. | ||
| + | ||
| .Show request | ||
| [%collapsible] | ||
| ==== | ||
| ```bash | ||
| export CLUSTER_PATCH_BODY=`cat << EOF | ||
| { | ||
| "customer_managed_resources": { | ||
| "gcp": { | ||
| "redpanda_connect_api_service_account": { | ||
| "email": "<redpanda_connect-api-gcp-sa-account-id>@<service-project-id>.iam.gserviceaccount.com" | ||
| }, | ||
| "redpanda_connect_service_account": { | ||
| "email": "<redpanda_connect-gcp-sa-account-id>@<service-project-id>.iam.gserviceaccount.com" | ||
| } | ||
| } | ||
| } | ||
| } | ||
| EOF` | ||
| curl -v -X PATCH \ | ||
| -H "Content-Type: application/json" \ | ||
| -H "Authorization: Bearer $AUTH_TOKEN" \ | ||
| -d "$CLUSTER_PATCH_BODY" $PUBLIC_API_ENDPOINT/v1/clusters/<cluster-id> | ||
| ``` | ||
| ==== | ||
|
|
||
| . Check Redpanda Connect is available in the Cloud UI. | ||
asimms41 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| .. Log in to https://cloud.redpanda.com[Redpanda Cloud^]. | ||
| .. Go to the **Connect** page and you should see the **Redpanda Connect** tab alongside **Kafka Connect**. | ||
asimms41 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| == Next steps | ||
|
|
||
| * Choose xref:develop:connect/components/catalog.adoc[connectors for your use case]. | ||
| * Learn how to xref:redpanda-connect:guides:getting_started.adoc[configure, test, and run a data pipeline locally]. | ||
| * Try the xref:develop:connect/connect-quickstart.adoc[Redpanda Connect quickstart]. | ||
| * Try one of our xref:cookbooks:index.adoc[Redpanda Connect cookbooks]. | ||
| * Learn how to xref:develop:connect/configuration/secret-management.adoc[add secrets to your pipeline]. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.