-
Notifications
You must be signed in to change notification settings - Fork 179
Move own CA transport layer mtls guidance to security docs #3932
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
4ef3fbb
3f5b433
c401bfa
c60da3e
b94310e
e43a868
60e11e0
74894f3
141c98d
59aaf4d
d6ecddf
fa7d774
b40ea3b
d2ff6e6
462feef
316536c
89ec319
404c1db
b683d4e
0566b5e
2468cc7
555c0f3
580e632
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1 @@ | ||
| {{es}} transport security and TLS certificates are automatically configured by the operator, but you can still [customize its service and CA certificates](/deploy-manage/security/k8s-transport-settings.md). | ||
| {{es}} transport security and TLS certificates are automatically configured by the operator, but you can still [customize the {{es}} transport service, certificate authority, and certificates](/deploy-manage/security/k8s-transport-settings.md). |
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,5 @@ | ||||||
| :::{warning} | ||||||
| Transport connections between {{es}} nodes are security-critical and you must protect them carefully. Malicious actors who can observe or interfere with node-to-node transport traffic can read or modify cluster data. A malicious actor who can establish a transport connection might be able to invoke system-internal APIs, including APIs that read or modify cluster data. | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is sentence 3 an explanation of sentence 2, or is it a separate thing? If it's an explanation, you could clarify that by saying something like:
Suggested change
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @DaveCTurner can you answer this? not sure if these two statements technically mean two different things
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. They're separate things. Establishing a connection is different from interfering with traffic on existing connections. |
||||||
|
|
||||||
| If you choose to issue node transport certificates using an external CA, then carefully review [](/deploy-manage/security/external-ca-transport.md) to ensure that your certificates meet the security requirements for transport connections. | ||||||
| ::: | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't understand the concept of an
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is defined in these very docs:
|
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,40 @@ | ||||||
| --- | ||||||
| applies_to: | ||||||
| deployment: | ||||||
| self: | ||||||
| eck: | ||||||
| products: | ||||||
| - id: elasticsearch | ||||||
| navigation_title: External CA for TLS | ||||||
| --- | ||||||
|
|
||||||
| # Using an external certificate authority to secure node-to-node connections | ||||||
|
|
||||||
| By default, {{es}} uses mutual TLS (mTLS) to secure node-to-node transport connections. Mutual TLS means that data is encrypted in transit, ensuring confidentiality and integrity, and also that both nodes in a connection must present a valid certificate to the other node when establishing the connection. Each node requires that certificates be issued by a trusted certificate authority (CA), ensuring that only authorized nodes can connect. Configure trusted certificate authorities using settings in the [`xpack.security.transport.ssl.*`](elasticsearch://reference/elasticsearch/configuration-reference/security-settings.md#transport-tls-ssl-settings) namespace, such as `xpack.security.transport.ssl.certificate_authorities` and `xpack.security.transport.ssl.truststore.path`. | ||||||
|
|
||||||
| {{es}} comes with a built-in tool called [`elasticsearch-certutil`](/deploy-manage/security/set-up-basic-security.md), which you can use to create and manage a dedicated certificate authority for each of your clusters, and to issue TLS certificates from this certificate authority. If you prefer not to use `elasticsearch-certutil`, then you must obtain the certificates from another certificate authority using standard TLS tools. Any certificate authority that is not managed using `elasticsearch-certutil` is referred to as an "external certificate authority" or "external CA". | ||||||
|
|
||||||
| This page explains the requirements and best practices to ensure that certificates generated using an external CA work correctly and protect your cluster properly. | ||||||
|
|
||||||
shainaraskas marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
| ::::{warning} | ||||||
| Transport connections between {{es}} nodes are security-critical and you must protect them carefully. Malicious actors who can observe or interfere with unencrypted node-to-node transport traffic can read or modify cluster data. A malicious actor who can establish a transport connection might be able to invoke system-internal APIs, including APIs that read or modify cluster data. | ||||||
| :::: | ||||||
|
|
||||||
| ## mTLS transport certificate requirements for external CAs | ||||||
|
|
||||||
| Obtain your transport certificates from a certificate authority that only issues certificates to {{es}} nodes permitted to connect to your cluster. Do not use a public certificate authority or an organization-wide private certificate authority, because these issue certificates to entities beyond your authorized cluster nodes. Use a dedicated private certificate authority for each {{es}} cluster. | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This first sentence should probably be considered or used for comparison in the section "Transport vs HTTP". We probably want to explain that this is not that relevant for HTTP, and for HTTP, for operational purposes (and because we have extra authentication and authorization mechanisms), it's common to not use dedicated CAs per cluster, and even public / organizational CAs that are automatically trusted by the clients. Of course this will depend on the use case, as in certain use cases it might have sense to have the HTTP layer also super-protected at TLS level. We should end up with the recommendation that as minimum, if they use private CAs, they should create a private CA to generate transport certs, and another private CA to generate HTTP certs. This relates with the comment in the "transport vs HTTP" section.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm not sure what concrete change you're suggesting here. The title of this page is |
||||||
|
|
||||||
| Certificates used for mTLS must either have no Extended Key Usage extension, or include both `clientAuth` and `serverAuth` values in the extension. Public certificate authorities typically omit the `clientAuth` value in the Extended Key Usage extension, making them unsuitable for mTLS. | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
A public certificate without clientAuth is perfectly valid for mTLS in general, but not for ES transport mTLS.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I disagree, it depends whether you read "mTLS" as "everything about setting up a mutually-authenticated TLS connection" or "client-certificate authentication". The general practice seems to be the latter, and that's how we're using it here. |
||||||
|
|
||||||
| ### Transport certificates vs. HTTP certificates | ||||||
|
|
||||||
| Transport certificates have different security requirements than [HTTP certificates](/deploy-manage/security/secure-cluster-communications.md#encrypt-http-communication). HTTP connections don't typically use mTLS because HTTP has its own authentication mechanisms. Because of this, HTTP certificates usually don't need to include the `clientAuth` value in their Extended Key Usage extension. HTTP certificates can come from public or organization-wide certificate authorities, while transport certificates should use a cluster-specific private CA. In most cases, you should not use the same certificate for both HTTP and transport connections. | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We need to rephrase this somehow, because server certs without The transport specific thing is that when it comes to transport layer Elasticsearch node is a client and a server at the same time, and that's why the But in general for mTLS on HTTP (where clients and servers are different entities), the clients and servers use different certificates, and a server cert without
First clause is perfectly ok and true, but it's not related with the second, which is also correct per-se.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Not only the certificates, I think we also want users to use different CAs for HTTP and transport, so:
thoughts? I think this section (which is super important IMO) needs a few paragraphs, bullets and pointers.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. As above I would rather not split hairs about whether mTLS refers just to the client-certificate authentication or the whole connection process. Likewise, you're right that there are many other concerns (mostly addressed elsewhere in this text) about how to obtain these certificates. Yet, a very common mistake is for users to use the same certificate for both uses, and it is vitally important to have a sentence calling out this case. |
||||||
|
|
||||||
| ## Turning off mTLS for transport connections [turn-off-mtls] | ||||||
|
|
||||||
| If your environment has some other way to prevent unauthorized node-to-node connections, you can disable mTLS by setting `xpack.security.transport.ssl.client_authentication: none`. You can still use non-mutual TLS to ensure the confidentiality and integrity of node-to-node traffic by setting `xpack.security.transport.ssl.enabled: true`. With non-mutual TLS, transport certificates don't require the `clientAuth` value in the Extended Key Usage extension. | ||||||
|
|
||||||
| ::::{warning} | ||||||
| Turning off mTLS by setting `xpack.security.transport.ssl.client_authentication` to `optional` or `none` allows anyone with network access to establish transport connections. Malicious actors can use these connections to invoke system-internal APIs that may read or modify cluster data. Use mTLS to | ||||||
| protect your node-to-node transport connections unless you are absolutely certain that unauthorized network access to these nodes cannot occur. | ||||||
| :::: | ||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -10,7 +10,10 @@ products: | |
|
|
||
| # Manage transport certificates on ECK [k8s-transport-settings] | ||
|
|
||
| The transport module in {{es}} is used for internal communication between nodes within the cluster as well as communication between remote clusters. Check the [{{es}} documentation](elasticsearch://reference/elasticsearch/configuration-reference/networking-settings.md) for details. For customization options of the HTTP layer, check [Access services](../deploy/cloud-on-k8s/accessing-services.md) and [HTTP TLS certificates](./k8s-https-settings.md). | ||
| The transport module in {{es}} is used for internal communication between nodes within the cluster as well as communication between remote clusters. For more information, refer to [Networking settings](elasticsearch://reference/elasticsearch/configuration-reference/networking-settings.md). For customization options of the HTTP layer, refer to [Access services](../deploy/cloud-on-k8s/accessing-services.md) and [HTTP TLS certificates](./k8s-https-settings.md). | ||
|
|
||
| :::{include} ./_snippets/own-ca-warning.md | ||
| ::: | ||
|
|
||
| ## Customize the Transport Service [k8s_customize_the_transport_service] | ||
|
|
||
|
|
@@ -36,7 +39,7 @@ When you change the `clusterIP` setting of the service, ECK deletes and re-creat | |
|
|
||
| ## Configure a custom Certificate Authority [k8s-transport-ca] | ||
|
|
||
| {{es}} uses X.509 certificates to establish encrypted and authenticated connections across nodes in the cluster. By default, ECK creates a self-signed CA certificate to issue a certificate [for each node in the cluster](/deploy-manage/security/set-up-basic-security.md#encrypt-internode-communication). | ||
| {{es}} uses X.509 certificates to establish encrypted and authenticated connections across nodes in the cluster. By default, ECK creates a CA to issue a self-signed certificate [for each node in the cluster](/deploy-manage/security/set-up-basic-security.md#encrypt-internode-communication). | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This change isn't accurate. The CA certificate is self-signed but the certificates it issues are signed by the CA certificate and are not self-signed. |
||
|
|
||
| You can use a Kubernetes secret to provide your own CA instead of the self-signed certificate that ECK will then use to create node certificates for transport connections. The CA certificate must be stored in the secret under `ca.crt` and the private key must be stored under `ca.key`. | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -14,14 +14,17 @@ products: | |
| # Same CA [update-node-certs-same] | ||
|
|
||
|
|
||
| This procedure assumes that the you have access to the CA certificate and key that was originally generated (or otherwise held by your organization) and used to sign the node certificates currently in use. It also assumes that the clients connecting to {{es}} on the HTTP layer are configured to trust the CA certificate. | ||
| This procedure assumes that the you have access to the certificate and key that was originally generated (or otherwise held by your organization) and used to sign the node certificates currently in use. It also assumes that the clients connecting to {{es}} on the HTTP layer are configured to trust the certificate. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think we need to be specific that we're talking about CA certificates in this paragraph, rather than the node certificates, so I'd rather we reinstated the |
||
|
|
||
| If you have access to the CA used to sign your existing certificates, you only need to replace the certificates and keys for each node in your cluster. If you replace your existing certificates and keys on each node and use the same filenames, {{es}} reloads the files starts using the new certificates and keys. | ||
| If you have access to the certificate authority (CA) used to sign your existing certificates, you only need to replace the certificates and keys for each node in your cluster. If you replace your existing certificates and keys on each node and use the same filenames, {{es}} reloads the files starts using the new certificates and keys. | ||
|
|
||
| You don’t have to restart each node, but doing so forces new TLS connections and is [a recommended practice](updating-certificates.md#use-rolling-restarts) when updating certificates. Therefore, the following steps include a node restart after updating each certificate. | ||
|
|
||
| The following steps provide instructions for generating new node certificates and keys for both the transport layer and the HTTP layer. You might only need to replace one of these layer’s certificates depending on which of your certificates are expiring. | ||
shainaraskas marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| :::{include} ./_snippets/own-ca-warning.md | ||
| ::: | ||
|
|
||
| ::::{important} | ||
| :name: cert-password-updates | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -4,10 +4,12 @@ mapped_pages: | |
| - https://www.elastic.co/guide/en/elasticsearch/reference/current/security-basic-setup.html | ||
| - https://www.elastic.co/guide/en/kibana/current/elasticsearch-mutual-tls.html | ||
| applies_to: | ||
| serverless: | ||
| deployment: | ||
| self: | ||
| eck: | ||
| ece: | ||
| ess: | ||
|
Comment on lines
+7
to
+12
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't think this applies to serverless or ESS - in the managed setups we manage all this stuff on the users' behalf. |
||
| products: | ||
| - id: elasticsearch | ||
| - id: kibana | ||
|
|
@@ -57,36 +59,43 @@ Securing this layer prevents unauthorized nodes from joining your cluster and pr | |
|
|
||
| The way that transport layer security is managed depends on your deployment type: | ||
|
|
||
| ::::{tab-set} | ||
| :::::{tab-set} | ||
| :group: deployments | ||
|
|
||
| :::{tab-item} ECH and Serverless | ||
| ::::{tab-item} ECH and Serverless | ||
| :sync: ech | ||
| {{es}} transport security is fully managed by Elastic, and no configuration is required. | ||
| ::: | ||
| :::: | ||
|
|
||
| :::{tab-item} ECE | ||
| ::::{tab-item} ECE | ||
| :sync: ece | ||
| {{es}} transport security is fully managed by {{ece}} platform, and no configuration is required. | ||
| ::: | ||
| {{es}} transport security is fully managed by the {{ece}} platform, and no configuration is required. | ||
| :::: | ||
|
|
||
| :::{tab-item} ECK | ||
| ::::{tab-item} ECK | ||
| :sync: eck | ||
|
|
||
| :::{include} ./_snippets/eck-transport.md | ||
shainaraskas marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| ::: | ||
|
|
||
| :::{include} ./_snippets/own-ca-warning.md | ||
| ::: | ||
|
|
||
| :::{tab-item} Self-managed | ||
| :::: | ||
|
|
||
| ::::{tab-item} Self-managed | ||
| :sync: self | ||
| {{es}} transport security can be [automatically configured](self-auto-setup.md), or manually set up by following the steps in [](set-up-basic-security.md). | ||
|
|
||
| For additional TLS configuration options, refer to [](./self-tls.md). | ||
|
|
||
| :::{include} ./_snippets/own-ca-warning.md | ||
| ::: | ||
|
|
||
| :::: | ||
|
|
||
| ::::: | ||
|
|
||
| ### HTTP layer security [encrypt-http-communication] | ||
|
|
||
| The HTTP layer includes the service endpoints exposed by both {{es}} and {{kib}}, supporting communications such as REST API requests, browser access to {{kib}}, and {{kib}}’s own traffic to {{es}}. Securing these endpoints helps prevent unauthorized access and protects sensitive data in transit. | ||
|
|
||
Uh oh!
There was an error while loading. Please reload this page.