From e5772417be9193717a7a53c211256743aa92cf44 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edu=20Gonz=C3=A1lez=20de=20la=20Herr=C3=A1n?= <25320357+eedugon@users.noreply.github.com> Date: Wed, 16 Jul 2025 14:16:26 +0200 Subject: [PATCH 1/9] snapshots clients background added for Azure and S3 --- .../tools/snapshot-and-restore/azure-repository.md | 8 +++++--- deploy-manage/tools/snapshot-and-restore/s3-repository.md | 8 +++++--- 2 files changed, 10 insertions(+), 6 deletions(-) diff --git a/deploy-manage/tools/snapshot-and-restore/azure-repository.md b/deploy-manage/tools/snapshot-and-restore/azure-repository.md index 5063831155..1233b7477c 100644 --- a/deploy-manage/tools/snapshot-and-restore/azure-repository.md +++ b/deploy-manage/tools/snapshot-and-restore/azure-repository.md @@ -12,19 +12,21 @@ products: You can use [Azure Blob storage](https://docs.microsoft.com/en-us/azure/storage/blobs/storage-blobs-introduction) as a repository for [Snapshot and restore](../snapshot-and-restore.md). +To communicate with Azure Blob storage, {{es}} internally uses a client module, referred to as the *Azure client* or *Azure repository client* in this document. Each client is configured through a combination of [secure settings](../../security/secure-settings.md) and [stack settings](/deploy-manage/stack-settings.md) defined in `elasticsearch.yml`. + ## Setup [repository-azure-usage] To enable Azure repositories, first configure an Azure repository client by specifying one or more settings of the form `azure.client.CLIENT_NAME.SETTING_NAME`. By default, `azure` repositories use a client named `default`, but you may specify a different client name when registering each repository. -The only mandatory Azure repository client setting is `account`, which is a [secure setting](../../security/secure-settings.md) defined in the [{{es}} keystore](../../security/secure-settings.md). To provide this setting, use the `elasticsearch-keystore` tool on each node: +The only mandatory setting for an Azure repository client is `account`, which is a [secure setting](../../security/secure-settings.md) defined in the {{es}} keystore. To provide this setting, use the `elasticsearch-keystore` tool on each node: ```sh bin/elasticsearch-keystore add azure.client.default.account ``` -If you adjust this setting after a node has started, call the [Nodes reload secure settings API](../../security/secure-settings.md) to reload the new value. +If you adjust this setting after a node has started, call the [Nodes reload secure settings API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-reload-secure-settings) to reload the new value. -You may define more than one client by setting their `account` values. For instance, to set the `default` client and another client called `secondary`, run the following commands on each node: +You may define more than one client by setting their `account` values. For example, to set the `default` client and another client called `secondary`, run the following commands on each node: ```sh bin/elasticsearch-keystore add azure.client.default.account diff --git a/deploy-manage/tools/snapshot-and-restore/s3-repository.md b/deploy-manage/tools/snapshot-and-restore/s3-repository.md index b5b9e644c4..5fcfd1a292 100644 --- a/deploy-manage/tools/snapshot-and-restore/s3-repository.md +++ b/deploy-manage/tools/snapshot-and-restore/s3-repository.md @@ -18,6 +18,8 @@ If you are looking for a hosted solution of {{es}} on AWS, visit [https://www.el See [this video](https://www.youtube.com/watch?v=ACqfyzWf-xs) for a walkthrough of connecting an AWS S3 repository. +{{es}} communicates with S3 through a dedicated *S3 client* module. Each client is configured using a mix of [secure settings](../../security/secure-settings.md) and [standard `elasticsearch.yml` settings](/deploy-manage/stack-settings.md). If you don't provide explicit S3 client configuration, {{es}} will try to obtain credentials from the environment it's running in. + ## Getting started [repository-s3-usage] To register an S3 repository, specify the type as `s3` when creating the repository. The repository defaults to using [ECS IAM Role](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-iam-roles.html) credentials for authentication. You can also use [Kubernetes service accounts](#iam-kubernetes-service-accounts) for authentication. @@ -37,7 +39,7 @@ PUT _snapshot/my_s3_repository ## Client settings [repository-s3-client] -The client that you use to connect to S3 has a number of settings available. The settings have the form `s3.client.CLIENT_NAME.SETTING_NAME`. By default, `s3` repositories use a client named `default`, but this can be modified using the [repository setting](#repository-s3-repository) `client`. For example, to use a client named `my-alternate-client`, register the repository as follows: +The S3 client that you use to connect to S3 has a number of settings available. The settings have the form `s3.client.CLIENT_NAME.SETTING_NAME`. By default, `s3` repositories use a client named `default`, but this can be modified using the [repository setting](#repository-s3-repository) `client`. For example, to use an S3 client named `my-alternate-client`, register the repository as follows: ```console PUT _snapshot/my_s3_repository @@ -50,7 +52,7 @@ PUT _snapshot/my_s3_repository } ``` -Most client settings can be added to the [`elasticsearch.yml`](/deploy-manage/stack-settings.md) configuration file with the exception of the secure settings, which you add to the {{es}} keystore. For more information about creating and updating the {{es}} keystore, see [Secure settings](../../security/secure-settings.md). +Most S3 client settings can be added to the [`elasticsearch.yml`](/deploy-manage/stack-settings.md) configuration file with the exception of the secure settings, which you add to the {{es}} keystore. For more information about creating and updating the {{es}} keystore, see [Secure settings](../../security/secure-settings.md). For example, if you want to use specific credentials to access S3 then run the following commands to add these credentials to the keystore. @@ -77,7 +79,7 @@ bin/elasticsearch-keystore remove s3.client.default.session_token Define the relevant secure settings in each node’s keystore before starting the node. The secure settings described here are all [reloadable](../../security/secure-settings.md#reloadable-secure-settings) so you may update the keystore contents on each node while the node is running and then call the [Nodes reload secure settings API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-reload-secure-settings) to apply the updated settings to the nodes in the cluster. After this API completes, {{es}} will use the updated setting values for all future snapshot operations, but ongoing operations may continue to use older setting values. -The following list contains the available client settings. Those that must be stored in the keystore are marked as "secure" and are **reloadable**; the other settings belong in the [`elasticsearch.yml`](/deploy-manage/stack-settings.md) file. +The following list contains the available S3 client settings. Those that must be stored in the keystore are marked as "secure" and are **reloadable**; the other settings belong in the [`elasticsearch.yml`](/deploy-manage/stack-settings.md) file. `region` : Specifies the region to use. When set, determines the signing region and regional endpoint to use, unless the endpoint is overridden via the `endpoint` setting. If not set, {{es}} will attempt to determine the region automatically using the AWS SDK. From 104d1061e9dc066fc8a5b865c5aadde86c8f7dd5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edu=20Gonz=C3=A1lez=20de=20la=20Herr=C3=A1n?= <25320357+eedugon@users.noreply.github.com> Date: Wed, 16 Jul 2025 14:20:19 +0200 Subject: [PATCH 2/9] snapshots clients background added for Azure and S3 --- deploy-manage/tools/snapshot-and-restore/azure-repository.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/deploy-manage/tools/snapshot-and-restore/azure-repository.md b/deploy-manage/tools/snapshot-and-restore/azure-repository.md index 1233b7477c..7aaabba2ba 100644 --- a/deploy-manage/tools/snapshot-and-restore/azure-repository.md +++ b/deploy-manage/tools/snapshot-and-restore/azure-repository.md @@ -12,7 +12,7 @@ products: You can use [Azure Blob storage](https://docs.microsoft.com/en-us/azure/storage/blobs/storage-blobs-introduction) as a repository for [Snapshot and restore](../snapshot-and-restore.md). -To communicate with Azure Blob storage, {{es}} internally uses a client module, referred to as the *Azure client* or *Azure repository client* in this document. Each client is configured through a combination of [secure settings](../../security/secure-settings.md) and [stack settings](/deploy-manage/stack-settings.md) defined in `elasticsearch.yml`. +{{es}} uses an internal *Azure repository client* to connect to Azure Blob storage. Each client is configured using both [secure settings](../../security/secure-settings.md) (set in the keystore) and [stack settings](/deploy-manage/stack-settings.md) (set in `elasticsearch.yml`). ## Setup [repository-azure-usage] From 382df18038c85edf730e2434a8bedbc819f96237 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edu=20Gonz=C3=A1lez=20de=20la=20Herr=C3=A1n?= <25320357+eedugon@users.noreply.github.com> Date: Wed, 16 Jul 2025 14:21:10 +0200 Subject: [PATCH 3/9] snapshots clients background added for Azure and S3 --- deploy-manage/tools/snapshot-and-restore/azure-repository.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/deploy-manage/tools/snapshot-and-restore/azure-repository.md b/deploy-manage/tools/snapshot-and-restore/azure-repository.md index 7aaabba2ba..3531679e52 100644 --- a/deploy-manage/tools/snapshot-and-restore/azure-repository.md +++ b/deploy-manage/tools/snapshot-and-restore/azure-repository.md @@ -12,7 +12,7 @@ products: You can use [Azure Blob storage](https://docs.microsoft.com/en-us/azure/storage/blobs/storage-blobs-introduction) as a repository for [Snapshot and restore](../snapshot-and-restore.md). -{{es}} uses an internal *Azure repository client* to connect to Azure Blob storage. Each client is configured using both [secure settings](../../security/secure-settings.md) (set in the keystore) and [stack settings](/deploy-manage/stack-settings.md) (set in `elasticsearch.yml`). +{{es}} uses an internal *Azure repository client* module to connect to Azure Blob storage. Each client is configured using both [secure settings](../../security/secure-settings.md) (set in the keystore) and [stack settings](/deploy-manage/stack-settings.md) (set in `elasticsearch.yml`). ## Setup [repository-azure-usage] From 2e672f2fb84914960d7b32a6fdee09541a0d6b02 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edu=20Gonz=C3=A1lez=20de=20la=20Herr=C3=A1n?= <25320357+eedugon@users.noreply.github.com> Date: Wed, 16 Jul 2025 14:27:06 +0200 Subject: [PATCH 4/9] snapshots clients background added for Azure and S3 --- deploy-manage/tools/snapshot-and-restore/azure-repository.md | 2 +- deploy-manage/tools/snapshot-and-restore/s3-repository.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/deploy-manage/tools/snapshot-and-restore/azure-repository.md b/deploy-manage/tools/snapshot-and-restore/azure-repository.md index 3531679e52..5727f26dab 100644 --- a/deploy-manage/tools/snapshot-and-restore/azure-repository.md +++ b/deploy-manage/tools/snapshot-and-restore/azure-repository.md @@ -12,7 +12,7 @@ products: You can use [Azure Blob storage](https://docs.microsoft.com/en-us/azure/storage/blobs/storage-blobs-introduction) as a repository for [Snapshot and restore](../snapshot-and-restore.md). -{{es}} uses an internal *Azure repository client* module to connect to Azure Blob storage. Each client is configured using both [secure settings](../../security/secure-settings.md) (set in the keystore) and [stack settings](/deploy-manage/stack-settings.md) (set in `elasticsearch.yml`). +{{es}} uses an internal *Azure repository client* module to connect to Azure Blob storage. Clients are configured through a combination of [secure settings](../../security/secure-settings.md) defined in the {{es}} keystore, and [standard settings](/deploy-manage/stack-settings.md) defined in `elasticsearch.yml`. ## Setup [repository-azure-usage] diff --git a/deploy-manage/tools/snapshot-and-restore/s3-repository.md b/deploy-manage/tools/snapshot-and-restore/s3-repository.md index 5fcfd1a292..62b2fb0089 100644 --- a/deploy-manage/tools/snapshot-and-restore/s3-repository.md +++ b/deploy-manage/tools/snapshot-and-restore/s3-repository.md @@ -18,7 +18,7 @@ If you are looking for a hosted solution of {{es}} on AWS, visit [https://www.el See [this video](https://www.youtube.com/watch?v=ACqfyzWf-xs) for a walkthrough of connecting an AWS S3 repository. -{{es}} communicates with S3 through a dedicated *S3 client* module. Each client is configured using a mix of [secure settings](../../security/secure-settings.md) and [standard `elasticsearch.yml` settings](/deploy-manage/stack-settings.md). If you don't provide explicit S3 client configuration, {{es}} will try to obtain credentials from the environment it's running in. +{{es}} communicates with S3 through a dedicated *S3 client* module. Clients are configured through a combination of [secure settings](../../security/secure-settings.md) defined in the {{es}} keystore, and [standard settings](/deploy-manage/stack-settings.md) defined in `elasticsearch.yml`. If you don't provide explicit S3 client configuration, {{es}} will try to obtain credentials from the environment it's running in. ## Getting started [repository-s3-usage] From 81d43c3e8835db6618c99589d0a0d5acf53f4d6c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edu=20Gonz=C3=A1lez=20de=20la=20Herr=C3=A1n?= <25320357+eedugon@users.noreply.github.com> Date: Wed, 16 Jul 2025 14:31:25 +0200 Subject: [PATCH 5/9] snapshots clients background added for Azure and S3 --- deploy-manage/tools/snapshot-and-restore/azure-repository.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/deploy-manage/tools/snapshot-and-restore/azure-repository.md b/deploy-manage/tools/snapshot-and-restore/azure-repository.md index 5727f26dab..78bfa883c1 100644 --- a/deploy-manage/tools/snapshot-and-restore/azure-repository.md +++ b/deploy-manage/tools/snapshot-and-restore/azure-repository.md @@ -14,6 +14,8 @@ You can use [Azure Blob storage](https://docs.microsoft.com/en-us/azure/storage/ {{es}} uses an internal *Azure repository client* module to connect to Azure Blob storage. Clients are configured through a combination of [secure settings](../../security/secure-settings.md) defined in the {{es}} keystore, and [standard settings](/deploy-manage/stack-settings.md) defined in `elasticsearch.yml`. +{{es}} uses an internal client module to connect to Azure Blob storage, referred to in this document as the *Azure repository client* or the *Azure client*. Clients are configured through a combination of [secure settings](../../security/secure-settings.md) defined in the {{es}} keystore and [standard settings](/deploy-manage/stack-settings.md) defined in `elasticsearch.yml`. + ## Setup [repository-azure-usage] To enable Azure repositories, first configure an Azure repository client by specifying one or more settings of the form `azure.client.CLIENT_NAME.SETTING_NAME`. By default, `azure` repositories use a client named `default`, but you may specify a different client name when registering each repository. From 469b55dd38877c68d01c7a3a772a2b1975398f5c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edu=20Gonz=C3=A1lez=20de=20la=20Herr=C3=A1n?= <25320357+eedugon@users.noreply.github.com> Date: Wed, 16 Jul 2025 14:31:48 +0200 Subject: [PATCH 6/9] snapshots clients background added for Azure and S3 --- deploy-manage/tools/snapshot-and-restore/azure-repository.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/deploy-manage/tools/snapshot-and-restore/azure-repository.md b/deploy-manage/tools/snapshot-and-restore/azure-repository.md index 78bfa883c1..c2f9aa992a 100644 --- a/deploy-manage/tools/snapshot-and-restore/azure-repository.md +++ b/deploy-manage/tools/snapshot-and-restore/azure-repository.md @@ -12,8 +12,6 @@ products: You can use [Azure Blob storage](https://docs.microsoft.com/en-us/azure/storage/blobs/storage-blobs-introduction) as a repository for [Snapshot and restore](../snapshot-and-restore.md). -{{es}} uses an internal *Azure repository client* module to connect to Azure Blob storage. Clients are configured through a combination of [secure settings](../../security/secure-settings.md) defined in the {{es}} keystore, and [standard settings](/deploy-manage/stack-settings.md) defined in `elasticsearch.yml`. - {{es}} uses an internal client module to connect to Azure Blob storage, referred to in this document as the *Azure repository client* or the *Azure client*. Clients are configured through a combination of [secure settings](../../security/secure-settings.md) defined in the {{es}} keystore and [standard settings](/deploy-manage/stack-settings.md) defined in `elasticsearch.yml`. ## Setup [repository-azure-usage] From 867a19a6b5834bc9b6ccf746c87024f8d244748a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edu=20Gonz=C3=A1lez=20de=20la=20Herr=C3=A1n?= <25320357+eedugon@users.noreply.github.com> Date: Wed, 16 Jul 2025 14:33:33 +0200 Subject: [PATCH 7/9] snapshots clients background added for Azure and S3 --- deploy-manage/tools/snapshot-and-restore/azure-repository.md | 2 +- deploy-manage/tools/snapshot-and-restore/s3-repository.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/deploy-manage/tools/snapshot-and-restore/azure-repository.md b/deploy-manage/tools/snapshot-and-restore/azure-repository.md index c2f9aa992a..fdaa6cff21 100644 --- a/deploy-manage/tools/snapshot-and-restore/azure-repository.md +++ b/deploy-manage/tools/snapshot-and-restore/azure-repository.md @@ -12,7 +12,7 @@ products: You can use [Azure Blob storage](https://docs.microsoft.com/en-us/azure/storage/blobs/storage-blobs-introduction) as a repository for [Snapshot and restore](../snapshot-and-restore.md). -{{es}} uses an internal client module to connect to Azure Blob storage, referred to in this document as the *Azure repository client* or the *Azure client*. Clients are configured through a combination of [secure settings](../../security/secure-settings.md) defined in the {{es}} keystore and [standard settings](/deploy-manage/stack-settings.md) defined in `elasticsearch.yml`. +{{es}} uses an internal client module to connect to Azure Blob storage, referred to in this document as the *Azure client* or the *Azure repository client*. Clients are configured through a combination of [secure settings](../../security/secure-settings.md) defined in the {{es}} keystore and [standard settings](/deploy-manage/stack-settings.md) defined in `elasticsearch.yml`. ## Setup [repository-azure-usage] diff --git a/deploy-manage/tools/snapshot-and-restore/s3-repository.md b/deploy-manage/tools/snapshot-and-restore/s3-repository.md index 62b2fb0089..f5d868ab03 100644 --- a/deploy-manage/tools/snapshot-and-restore/s3-repository.md +++ b/deploy-manage/tools/snapshot-and-restore/s3-repository.md @@ -18,7 +18,7 @@ If you are looking for a hosted solution of {{es}} on AWS, visit [https://www.el See [this video](https://www.youtube.com/watch?v=ACqfyzWf-xs) for a walkthrough of connecting an AWS S3 repository. -{{es}} communicates with S3 through a dedicated *S3 client* module. Clients are configured through a combination of [secure settings](../../security/secure-settings.md) defined in the {{es}} keystore, and [standard settings](/deploy-manage/stack-settings.md) defined in `elasticsearch.yml`. If you don't provide explicit S3 client configuration, {{es}} will try to obtain credentials from the environment it's running in. +{{es}} communicates with S3 through a dedicated S3 client module. Clients are configured through a combination of [secure settings](../../security/secure-settings.md) defined in the {{es}} keystore, and [standard settings](/deploy-manage/stack-settings.md) defined in `elasticsearch.yml`. If you don't provide explicit S3 client configuration, {{es}} will try to obtain credentials from the environment it's running in. ## Getting started [repository-s3-usage] From c5729c798550c0573f3195bbc7f918c05a2c26ac Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edu=20Gonz=C3=A1lez=20de=20la=20Herr=C3=A1n?= <25320357+eedugon@users.noreply.github.com> Date: Wed, 16 Jul 2025 14:52:38 +0200 Subject: [PATCH 8/9] snapshots clients background added for Azure and S3 --- deploy-manage/tools/snapshot-and-restore/azure-repository.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/deploy-manage/tools/snapshot-and-restore/azure-repository.md b/deploy-manage/tools/snapshot-and-restore/azure-repository.md index fdaa6cff21..bf3f7fd0f1 100644 --- a/deploy-manage/tools/snapshot-and-restore/azure-repository.md +++ b/deploy-manage/tools/snapshot-and-restore/azure-repository.md @@ -12,7 +12,7 @@ products: You can use [Azure Blob storage](https://docs.microsoft.com/en-us/azure/storage/blobs/storage-blobs-introduction) as a repository for [Snapshot and restore](../snapshot-and-restore.md). -{{es}} uses an internal client module to connect to Azure Blob storage, referred to in this document as the *Azure client* or the *Azure repository client*. Clients are configured through a combination of [secure settings](../../security/secure-settings.md) defined in the {{es}} keystore and [standard settings](/deploy-manage/stack-settings.md) defined in `elasticsearch.yml`. +{{es}} uses an internal client module to connect to Azure Blob storage, referred to in this document as the *Azure client* or the *Azure repository client*. Clients are configured through a combination of [secure settings](../../security/secure-settings.md) defined in the {{es}} keystore, and [standard settings](/deploy-manage/stack-settings.md) defined in `elasticsearch.yml`. ## Setup [repository-azure-usage] From 0e37a83b0322a6224383273871370b1e933cde91 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edu=20Gonz=C3=A1lez=20de=20la=20Herr=C3=A1n?= <25320357+eedugon@users.noreply.github.com> Date: Thu, 17 Jul 2025 13:09:57 +0200 Subject: [PATCH 9/9] Update deploy-manage/tools/snapshot-and-restore/azure-repository.md Co-authored-by: David Kilfoyle <41695641+kilfoyle@users.noreply.github.com> --- deploy-manage/tools/snapshot-and-restore/azure-repository.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/deploy-manage/tools/snapshot-and-restore/azure-repository.md b/deploy-manage/tools/snapshot-and-restore/azure-repository.md index bf3f7fd0f1..f0bb83cc3e 100644 --- a/deploy-manage/tools/snapshot-and-restore/azure-repository.md +++ b/deploy-manage/tools/snapshot-and-restore/azure-repository.md @@ -12,7 +12,7 @@ products: You can use [Azure Blob storage](https://docs.microsoft.com/en-us/azure/storage/blobs/storage-blobs-introduction) as a repository for [Snapshot and restore](../snapshot-and-restore.md). -{{es}} uses an internal client module to connect to Azure Blob storage, referred to in this document as the *Azure client* or the *Azure repository client*. Clients are configured through a combination of [secure settings](../../security/secure-settings.md) defined in the {{es}} keystore, and [standard settings](/deploy-manage/stack-settings.md) defined in `elasticsearch.yml`. +{{es}} uses an internal client module to connect to Azure Blob storage, referred to in this document as the *Azure client* or the *Azure repository client*. Clients are configured through a combination of [secure settings](../../security/secure-settings.md) defined in the {{es}} keystore, and [standard settings](/deploy-manage/stack-settings.md) defined in the `elasticsearch.yml` configuration file. ## Setup [repository-azure-usage]