-
Notifications
You must be signed in to change notification settings - Fork 27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
PrivateLink and Private Service Connect in Cloud UI #516
Merged
Merged
Changes from all commits
Commits
Show all changes
15 commits
Select commit
Hold shift + click to select a range
a179ed7
Initial draft
kbatuigas 660b0f0
Add new doc to nav tree
kbatuigas 8d69059
Update links to API docs
kbatuigas 870596e
Start to separate out into two different docs
kbatuigas 2cc9b3a
Add DNS instructions for GCP VPC
kbatuigas 00714ed
Edits per feedback
kbatuigas 375a7ab
remove BYOC-specific title
micheleRP 318051c
Update modules/deploy/pages/deployment-option/cloud/configure-private…
micheleRP 6489206
Update modules/deploy/pages/deployment-option/cloud/configure-private…
micheleRP 567d83b
Update modules/deploy/pages/deployment-option/cloud/configure-private…
micheleRP 33192c7
Update modules/deploy/pages/deployment-option/cloud/configure-private…
micheleRP 57637c6
Update modules/deploy/pages/deployment-option/cloud/configure-private…
micheleRP b3859f3
add DNS gcloud steps, review feedback, minor edits
micheleRP feeb1ef
minor edits
micheleRP 06703e0
standardize page titles, nav
micheleRP File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
92 changes: 92 additions & 0 deletions
92
...ages/deployment-option/cloud/configure-private-service-connect-in-cloud-ui.adoc
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,92 @@ | ||
= Configure GCP Private Service Connect in the Cloud UI | ||
:description: Set up GCP Private Service Connect in the Redpanda Cloud UI. | ||
:page-cloud: true | ||
|
||
include::shared:partial$feature-flag.adoc[] | ||
|
||
NOTE: This guide is for configuring GCP Private Service Connect using the Redpanda Cloud UI. See xref:deploy:deployment-option/cloud/gcp-private-service-connect.adoc[] if you want to set up this service using the API. | ||
|
||
The Redpanda GCP Private Service Connect service provides secure access to Redpanda Cloud from your own VPC. Traffic over Private Service Connect does not go through the public internet because these connections are treated as their own private GCP service. While your VPC has access to the Redpanda VPC, Redpanda cannot access your VPC. | ||
|
||
Consider using the endpoint services if you have multiple VPCs and could benefit from a more simplified approach to network management: | ||
|
||
* Private Service Connect allows overlapping xref:./cidr-ranges.adoc[CIDR ranges] in VPC networks. | ||
* Private Service Connect does not limit the number of connections using the service. | ||
* You control which GCP projects are allowed to connect to the service. | ||
|
||
== Requirements | ||
|
||
* Use https://cloud.google.com/sdk/docs/install[gcloud^] to create the consumer-side resources, such as a client VPC and forwarding rule, or modify existing resources to use the Private Service Connect service attachment created for your cluster. | ||
* The client VPC must be in the same region as your Redpanda cluster. | ||
|
||
== Enable endpoint service for existing clusters | ||
|
||
. In the Redpanda Cloud UI, open your https://cloud.redpanda.com/clusters[cluster^], and click **Cluster settings**. | ||
. Under Private Service Connect, click **Enable**. | ||
. For xref:deploy:deployment-option/cloud/vpc-byo-gcp.adoc[BYOC clusters with customer-managed VPC], you need a NAT subnet with the *Purpose* set to `PRIVATE_SERVICE_CONNECT`. You can create the subnet using the `gcloud` command-line interface (CLI): | ||
+ | ||
[,bash] | ||
---- | ||
gcloud compute networks subnets create <subnet-name> \ | ||
--project=<project> \ | ||
--network=<network-name> \ | ||
--region=<region> \ | ||
--range=<subnet-range> \ | ||
--purpose=PRIVATE_SERVICE_CONNECT | ||
---- | ||
+ | ||
Provide your values for the following placeholders: | ||
+ | ||
- `<subnet-name>`: The name of the NAT subnet. | ||
- `<project>`: The **host** GCP project ID. | ||
- `<network-name>`: The name of the VPC being used for your Redpanda Cloud cluster. | ||
- `<region>`: The region of the Redpanda Cloud cluster. | ||
- `<subnet-range>`: The CIDR range of the subnet. The mask should be at least `/29`. Each Private Service Connect connection takes up one IP address from the NAT subnet, so the CIDR must be able to accommodate all projects from which connections to the service attachment will be issued. | ||
+ | ||
See the GCP documentation for https://cloud.google.com/vpc/docs/configure-private-service-connect-producer#add-subnet-psc[creating a subnet for Private Service Connect^]. | ||
. For the accepted consumers list, you need the GCP project IDs from which incoming connections will be accepted. | ||
. It may take several minutes for your cluster to update. When the update is complete, the Private Service Connect status in **Cluster settings** changes from **In progress** to **Enabled**. | ||
|
||
NOTE: For help with issues when enabling Private Service Connect, contact https://support.redpanda.com/hc/en-us/requests/new[Redpanda support^]. | ||
|
||
=== Deploy consumer-side resources | ||
|
||
For each VPC network, you must complete the following steps to successfully connect to the service and use Kafka API and other Redpanda services such as HTTP Proxy. | ||
|
||
. In **Cluster settings**, copy the **Service attachment URL** under **Private Service Connect**. Use this URL to create the Private Service Connect endpoint in GCP. | ||
|
||
. Create a private DNS zone. Use the cluster **DNS zone** value as the DNS name. | ||
+ | ||
[,bash] | ||
---- | ||
gcloud dns --project=<GCP Project ID> managed-zones create <DNS zone name> --description="<description>" --dns-name="<DNS Zone from the UI>" --visibility="private" --networks="<list of fully-qualified name of networks where the DNS zone will be visible>" | ||
---- | ||
|
||
. In the newly-created DNS zone, create a wildcard DNS record using the cluster **DNS record** value. | ||
+ | ||
[,bash] | ||
---- | ||
gcloud dns --project=rp-byoc-juan-0e38 record-sets create '*.<DNS Zone from the UI>' --zone="<DNS zone name>" --type="A" --ttl="300" --rrdatas="<PSC endpoint IP>" | ||
---- | ||
|
||
. Confirm that your GCP VPC firewall allows traffic to and from the Private Service Connect forwarding rule IP address, on the expected ports. | ||
|
||
== Access Redpanda services through VPC endpoint | ||
|
||
After you have enabled Private Service Connect for your cluster, your connection URLs are available in the *How to Connect* section of the cluster overview in the Redpanda Cloud UI. | ||
|
||
include::deploy:partial$cloud/private-links-access-rp-services-through-vpc.adoc[] | ||
|
||
== Test the connection | ||
|
||
You can test the connection to the endpoint service from any VM or container in the consumer VPC. If configuring a client isn't possible right away, you can do these checks using `rpk` or cURL: | ||
|
||
include::deploy:partial$cloud/private-links-test-connection.adoc[] | ||
|
||
== Disable endpoint service | ||
|
||
In **Cluster settings**, click **Disable**. Existing connections are closed after GCP Private Service Connect is disabled. To connect using Private Service Connect again, you must re-enable the service. | ||
|
||
include::shared:partial$suggested-reading.adoc[] | ||
|
||
* xref:deploy:deployment-option/cloud/gcp-private-service-connect.adoc[] |
49 changes: 49 additions & 0 deletions
49
...les/deploy/pages/deployment-option/cloud/configure-privatelink-in-cloud-ui.adoc
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,49 @@ | ||
= Configure AWS PrivateLink in the Cloud UI | ||
:description: Set up AWS PrivateLink in the Redpanda Cloud UI. | ||
:page-cloud: true | ||
|
||
include::shared:partial$feature-flag.adoc[] | ||
|
||
NOTE: This guide is for configuring AWS PrivateLink using the Redpanda Cloud UI. See xref:deploy:deployment-option/cloud/aws-privatelink.adoc[Configure AWS PrivateLink for Redpanda Cloud] if you want to set up the endpoint service using the API. | ||
|
||
The Redpanda AWS PrivateLink endpoint service provides secure access to Redpanda Cloud from your own VPC. Traffic over PrivateLink does not go through the public internet because these connections are treated as their own private AWS service. While your VPC has access to the Redpanda VPC, Redpanda cannot access your VPC. | ||
|
||
Consider using the endpoint service if you have multiple VPCs and could benefit from a more simplified approach to network management: | ||
|
||
* PrivateLink allows overlapping xref:./cidr-ranges.adoc[CIDR ranges] in VPC networks. | ||
* PrivateLink does not limit the number of connections that use the endpoint service. | ||
* You control which AWS principals are allowed to connect to the endpoint service. | ||
|
||
== Requirements | ||
|
||
* Your Redpanda cluster and VPC must be in the same region. | ||
* Use the https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html[AWS CLI] to create a new client VPC or modify an existing one to use the PrivateLink endpoint. | ||
|
||
== Enable endpoint service for existing clusters | ||
|
||
. In the Redpanda Cloud UI, open your https://cloud.redpanda.com/clusters[cluster^], and click **Cluster settings**. | ||
. Under AWS PrivateLink, click **Enable**. | ||
. You need the Amazon Resource Names (ARNs) for the AWS principals allowed to access the endpoint service. For example, for all principals in a specific account, use `arn:aws:iam::<account-id>:root`. See the AWS documentation on https://docs.aws.amazon.com/vpc/latest/privatelink/configure-endpoint-service.html#add-remove-permission[configuring an endpoint service^] for details. | ||
. It may take several minutes for your cluster to update. When the update is complete, the AWS PrivateLink status in **Cluster settings** changes from **In progress** to **Enabled**. | ||
|
||
NOTE: For help with issues when enabling PrivateLink, contact https://support.redpanda.com/hc/en-us/requests/new[Redpanda support^]. | ||
|
||
== Access Redpanda services through VPC endpoint | ||
|
||
After you have enabled PrivateLink for your cluster, your connection URLs are available in the *How to Connect* section of the cluster overview in the Redpanda Cloud UI. | ||
paulzhang97 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
include::deploy:partial$cloud/private-links-access-rp-services-through-vpc.adoc[] | ||
|
||
== Test the connection | ||
|
||
You can test the connection to the endpoint service from any VM or container in the consumer VPC. If configuring a client isn't possible right away, you can do these checks using `rpk` or cURL: | ||
|
||
include::deploy:partial$cloud/private-links-test-connection.adoc[] | ||
|
||
== Disable endpoint service | ||
|
||
In **Cluster settings**, click **Disable**. Existing connections are closed after the AWS PrivateLink service is disabled. To connect using PrivateLink again, you must re-enable the service. | ||
|
||
include::shared:partial$suggested-reading.adoc[] | ||
|
||
* xref:deploy:deployment-option/cloud/aws-privatelink.adoc[] |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@0x5d I put together this section based on the Figma design as well as these original set of docs: https://github.com/redpanda-data/cloudv2/pull/12933/files#diff-b4884caa55c76586fff9375cb88a5e644b0f73f15bf9aa9b5f5d84b4e0fc1292R337 could you confirm if these steps are correct, and if they should also be added to the API guide? In that case, we should probably single source this section to share it across both guides.