-
Notifications
You must be signed in to change notification settings - Fork 181
ECE and ECH to ECK remote clusters setup updated for API keys #3849
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
+318
−35
Merged
Changes from all commits
Commits
Show all changes
13 commits
Select commit
Hold shift + click to select a range
ee287be
ECE and ECH to ECK remote clusters setup updated for API keys
eedugon 2ec2ee6
ECE and ECH to ECK remote clusters setup updated for API keys
eedugon 55379fa
minor refinement
eedugon f77d85e
snippets comments updated
eedugon 7b30283
Apply suggestions from code review
eedugon 67151cc
reviewing suggestions applied
eedugon 4ee69a1
Merge remote-tracking branch 'origin/main' into remote_clusters_ec_eck
eedugon 8f7b12f
simplification of eck 3.3 statement
eedugon 582bebe
linting fixes
eedugon 0757296
Apply suggestions from code review
eedugon 5682aab
targetPort and reviewing suggestions
eedugon 140a208
Apply suggestions from code review
eedugon 1c054b2
Merge branch 'main' into remote_clusters_ec_eck
eedugon File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
15 changes: 15 additions & 0 deletions
15
deploy-manage/remote-clusters/_snippets/eck_expose_transport.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| Expose the transport service (defaults to port `9300`) of your ECK cluster to allow external {{es}} clusters to connect: | ||
|
|
||
| ```yaml | ||
| apiVersion: elasticsearch.k8s.elastic.co/v1 | ||
| kind: Elasticsearch | ||
| metadata: | ||
| name: <cluster-name> | ||
| spec: | ||
| transport: | ||
| service: | ||
| spec: | ||
| type: LoadBalancer <1> | ||
| ``` | ||
| 1. On cloud providers which support external load balancers, setting the type field to `LoadBalancer` provisions a load balancer for your service. Alternatively, expose the service `<cluster-name>-es-transport` through one of the Kubernetes Ingress controllers that support TCP services. |
5 changes: 5 additions & 0 deletions
5
deploy-manage/remote-clusters/_snippets/eck_rcs_connect_intro.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| On the local deployment, add the remote ECK cluster using {{kib}} or the {{es}} API with the following connection settings: | ||
|
|
||
| * **Remote address**: Use the FQDN or IP address of the LoadBalancer service, or similar resource, you created to expose the remote cluster server interface (for API key-based authentication) or the transport interface (for TLS certificate-based authentication). | ||
|
|
||
| * **TLS server name**: You can try leaving this field empty first. If the connection fails, and your environment is presenting the ECK-managed certificates during the TLS handshake, use `<cluster-name>-es-remote-cluster.<namespace>.svc` as the server name. For example, for a cluster named `quickstart` in the `default` namespace, use `quickstart-es-remote-cluster.default.svc`. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| By default, the remote cluster server interface is deactivated on ECK-managed clusters. To use the API key–based security model for cross-cluster connections, you must first enable it on the remote {{es}} cluster: | ||
|
|
||
| ```yaml subs=true | ||
| apiVersion: elasticsearch.k8s.elastic.co/v1 | ||
| kind: Elasticsearch | ||
| metadata: | ||
| name: <cluster-name> | ||
| namespace: <namespace> | ||
| spec: | ||
| version: {{version.stack}} | ||
| remoteClusterServer: | ||
| enabled: true | ||
| nodeSets: | ||
| - name: default | ||
| count: 3 | ||
| ... | ||
| ... | ||
| ``` | ||
|
|
||
| ::::{note} | ||
| Enabling the remote cluster server triggers a restart of the {{es}} cluster. | ||
| :::: | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,52 @@ | ||
| When the remote cluster server is enabled, ECK automatically creates a Kubernetes service named `<cluster-name>-es-remote-cluster` that exposes the server internally on port `9443`. | ||
|
|
||
| To allow clusters running outside your Kubernetes environment to connect to this {{es}} cluster, you must expose this service externally. The way to expose this service depends on your ECK version. | ||
|
|
||
| :::::{applies-switch} | ||
|
|
||
| ::::{applies-item} eck: ga 3.3 | ||
| You can customize how the remote cluster service is exposed by overriding its service specification directly under `spec.remoteClusterServer.service` in the {{es}} resource. By default, this service listens on port 9443. | ||
|
|
||
| ```yaml | ||
| apiVersion: elasticsearch.k8s.elastic.co/v1 | ||
| kind: Elasticsearch | ||
| metadata: | ||
| name: <cluster-name> | ||
| namespace: <namespace> | ||
| spec: | ||
| version: 9.2.1 | ||
| remoteClusterServer: | ||
| enabled: true | ||
| service: | ||
| spec: | ||
| type: LoadBalancer <1> | ||
| nodeSets: | ||
| - name: default | ||
| count: 3 | ||
| ... | ||
| ... | ||
| ``` | ||
| 1. On cloud providers that support external load balancers, setting the type to `LoadBalancer` provisions a load balancer for your service. Alternatively, expose the service `<cluster-name>-es-remote-cluster` through one of the Kubernetes Ingress controllers that support TCP services. | ||
| :::: | ||
|
|
||
| ::::{applies-item} eck: ga 3.0 | ||
eedugon marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| In ECK 3.2 and earlier, you can't customize the service that ECK generates for the remote cluster interface, but you can create your own `LoadBalancer` service, `Ingress` object, or use another method available in your environment. | ||
|
|
||
| For example, for a cluster named `quickstart`, the following command creates a separate `LoadBalancer` service named `quickstart-es-remote-cluster-lb`, pointing to the ECK-managed service `quickstart-es-remote-cluster`: | ||
|
|
||
| ```sh | ||
| kubectl expose service quickstart-es-remote-cluster \ | ||
| --name=quickstart-es-remote-cluster-lb \ | ||
| --type=LoadBalancer \ <1> | ||
| --port=9443 --target-port=9443 | ||
| ``` | ||
| 1. On cloud providers that support external load balancers, setting the type to `LoadBalancer` provisions a load balancer for your service. Alternatively, expose the service `<cluster-name>-es-remote-cluster` through one of the Kubernetes Ingress controllers that support TCP services. | ||
|
|
||
| :::: | ||
| ::::: | ||
|
|
||
| :::{warning} | ||
| If you change the service’s `port`, set `targetPort` explicitly to `9443`, which is the default remote cluster server listening port. Otherwise, Kubernetes uses the same value for both fields, resulting in failed connections. | ||
| ::: | ||
|
|
||
17 changes: 17 additions & 0 deletions
17
deploy-manage/remote-clusters/_snippets/eck_rcs_retrieve_ca.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| The certificate authority (CA) used by ECK to issue certificates for the remote cluster server interface is stored in the `ca.crt` key of the secret named `<cluster_name>-es-transport-certs-public`. | ||
|
|
||
| If the external connections reach the {{es}} Pods on port `9443` without any intermediate TLS termination, you need to retrieve this CA because it is required in the local cluster configuration to establish trust. | ||
|
|
||
| If TLS is terminated by any intermediate component and the certificate presented is not the ECK-managed one, use the CA associated with that component, or omit the CA entirely if it uses a publicly trusted certificate. | ||
|
|
||
| To save the transport CA certificate of a cluster named `quickstart` into a local file, run the following command: | ||
|
|
||
| ```sh | ||
| kubectl get secret quickstart-es-transport-certs-public \ | ||
| -o go-template='{{index .data "ca.crt" | base64decode}}' > eck_transport_ca.crt | ||
| ``` | ||
|
|
||
| ::::{important} | ||
| ECK-managed CA certificates are automatically rotated after one year by default, but you can [configure](/deploy-manage/deploy/cloud-on-k8s/configure-eck.md) a different validity period. When the CA certificate is rotated, ensure that this CA is updated in all environments where it's used to preserve trust. | ||
| :::: | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.