From 73c85862cdb47a7b45445b77cd0f638137f018c0 Mon Sep 17 00:00:00 2001 From: Michael Montgomery Date: Mon, 13 Oct 2025 13:46:01 -0500 Subject: [PATCH 1/8] wip Signed-off-by: Michael Montgomery --- .../deployments/cloud-on-k8s/common-problems.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/troubleshoot/deployments/cloud-on-k8s/common-problems.md b/troubleshoot/deployments/cloud-on-k8s/common-problems.md index 7359f9108f..a6a5fe829c 100644 --- a/troubleshoot/deployments/cloud-on-k8s/common-problems.md +++ b/troubleshoot/deployments/cloud-on-k8s/common-problems.md @@ -306,3 +306,14 @@ Check that the role mapping was deleted: GET /_security/role_mapping/ {} ``` + + +## If a volume expansion failed [k8s-common-problems-volume-failed-expansion] + +If you attempted an expansion of an Elasticsearch volume via its volume claim template, you may have encountered scenarios where the operation failed such as Azure not allowing volume expansion without shutting down the Virtual Machine to which it is attached. To troubleshoot, refer to the official documentation at https://www.elastic.co/docs/deploy-manage/deploy/cloud-on-k8s/volume-claim-templates#k8s-volume-claim-templates-update. If you try to adjust the volume claim template back to the original size you will encounter a failure + +``` +Failed to apply spec change: handle volume expansion: decreasing storage size is not supported: an attempt was made to decrease storage size for claim elasticsearch-data +``` + +In this scenario the best course of action is to rename the existing `nodeSet` to a new name while simultaneously updating the volume claim template to the original size. From d9ac2e5f9fde03faf662fc434da7a2f168397c5e Mon Sep 17 00:00:00 2001 From: Michael Montgomery Date: Mon, 13 Oct 2025 14:10:42 -0500 Subject: [PATCH 2/8] Update wording. Signed-off-by: Michael Montgomery --- troubleshoot/deployments/cloud-on-k8s/common-problems.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/troubleshoot/deployments/cloud-on-k8s/common-problems.md b/troubleshoot/deployments/cloud-on-k8s/common-problems.md index a6a5fe829c..0798c37cb7 100644 --- a/troubleshoot/deployments/cloud-on-k8s/common-problems.md +++ b/troubleshoot/deployments/cloud-on-k8s/common-problems.md @@ -310,10 +310,10 @@ GET /_security/role_mapping/ ## If a volume expansion failed [k8s-common-problems-volume-failed-expansion] -If you attempted an expansion of an Elasticsearch volume via its volume claim template, you may have encountered scenarios where the operation failed such as Azure not allowing volume expansion without shutting down the Virtual Machine to which it is attached. To troubleshoot, refer to the official documentation at https://www.elastic.co/docs/deploy-manage/deploy/cloud-on-k8s/volume-claim-templates#k8s-volume-claim-templates-update. If you try to adjust the volume claim template back to the original size you will encounter a failure +If you attempted an expansion of an Elasticsearch volume via its (volume claim template)[/deploy-manage/deploy/cloud-on-k8s/volume-claim-templates#k8s-volume-claim-templates-update], you may have encountered scenarios where the operation failed such as Azure not allowing volume expansion without shutting down the Virtual Machine to which it is attached. If you try to adjust the volume claim template back to the original size you will encounter a failure: ``` Failed to apply spec change: handle volume expansion: decreasing storage size is not supported: an attempt was made to decrease storage size for claim elasticsearch-data ``` -In this scenario the best course of action is to rename the existing `nodeSet` to a new name while simultaneously updating the volume claim template to the original size. +In this scenario the best course of action is to rename the existing `nodeSet` to a new name while simultaneously updating the volume claim template to the original size. This operation will bring a new `StatefulSet` online while moving all existing indices to the new volumes and will delete the old `StatefulSet` and its volumes once the operation is complete. From db5deaf601c5c5645cb38f576651de4d7c172448 Mon Sep 17 00:00:00 2001 From: Michael Montgomery Date: Mon, 13 Oct 2025 14:20:46 -0500 Subject: [PATCH 3/8] Fix the url Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- troubleshoot/deployments/cloud-on-k8s/common-problems.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/troubleshoot/deployments/cloud-on-k8s/common-problems.md b/troubleshoot/deployments/cloud-on-k8s/common-problems.md index 0798c37cb7..da95ece6c0 100644 --- a/troubleshoot/deployments/cloud-on-k8s/common-problems.md +++ b/troubleshoot/deployments/cloud-on-k8s/common-problems.md @@ -310,7 +310,7 @@ GET /_security/role_mapping/ ## If a volume expansion failed [k8s-common-problems-volume-failed-expansion] -If you attempted an expansion of an Elasticsearch volume via its (volume claim template)[/deploy-manage/deploy/cloud-on-k8s/volume-claim-templates#k8s-volume-claim-templates-update], you may have encountered scenarios where the operation failed such as Azure not allowing volume expansion without shutting down the Virtual Machine to which it is attached. If you try to adjust the volume claim template back to the original size you will encounter a failure: +If you attempted an expansion of an Elasticsearch volume via its [volume claim template](/deploy-manage/deploy/cloud-on-k8s/volume-claim-templates#k8s-volume-claim-templates-update), you may have encountered scenarios where the operation failed such as Azure not allowing volume expansion without shutting down the Virtual Machine to which it is attached. If you try to adjust the volume claim template back to the original size you will encounter a failure: ``` Failed to apply spec change: handle volume expansion: decreasing storage size is not supported: an attempt was made to decrease storage size for claim elasticsearch-data From 9049817162e276329f4fa7631663609375cf61a7 Mon Sep 17 00:00:00 2001 From: Michael Montgomery Date: Tue, 14 Oct 2025 09:31:53 -0500 Subject: [PATCH 4/8] Add missing .md Signed-off-by: Michael Montgomery --- troubleshoot/deployments/cloud-on-k8s/common-problems.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/troubleshoot/deployments/cloud-on-k8s/common-problems.md b/troubleshoot/deployments/cloud-on-k8s/common-problems.md index da95ece6c0..0b97eb824b 100644 --- a/troubleshoot/deployments/cloud-on-k8s/common-problems.md +++ b/troubleshoot/deployments/cloud-on-k8s/common-problems.md @@ -310,7 +310,7 @@ GET /_security/role_mapping/ ## If a volume expansion failed [k8s-common-problems-volume-failed-expansion] -If you attempted an expansion of an Elasticsearch volume via its [volume claim template](/deploy-manage/deploy/cloud-on-k8s/volume-claim-templates#k8s-volume-claim-templates-update), you may have encountered scenarios where the operation failed such as Azure not allowing volume expansion without shutting down the Virtual Machine to which it is attached. If you try to adjust the volume claim template back to the original size you will encounter a failure: +If you attempted an expansion of an Elasticsearch volume via its [volume claim template](/deploy-manage/deploy/cloud-on-k8s/volume-claim-templates.md#k8s-volume-claim-templates-update), you may have encountered scenarios where the operation failed such as Azure not allowing volume expansion without shutting down the Virtual Machine to which it is attached. If you try to adjust the volume claim template back to the original size you will encounter a failure: ``` Failed to apply spec change: handle volume expansion: decreasing storage size is not supported: an attempt was made to decrease storage size for claim elasticsearch-data From 337db6263196ffd8e6c180cf91c983b87a11bd2d Mon Sep 17 00:00:00 2001 From: Michael Montgomery Date: Wed, 15 Oct 2025 07:25:37 -0500 Subject: [PATCH 5/8] Review changes Signed-off-by: Michael Montgomery --- troubleshoot/deployments/cloud-on-k8s/common-problems.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/troubleshoot/deployments/cloud-on-k8s/common-problems.md b/troubleshoot/deployments/cloud-on-k8s/common-problems.md index 0b97eb824b..41a7de6f5e 100644 --- a/troubleshoot/deployments/cloud-on-k8s/common-problems.md +++ b/troubleshoot/deployments/cloud-on-k8s/common-problems.md @@ -310,10 +310,10 @@ GET /_security/role_mapping/ ## If a volume expansion failed [k8s-common-problems-volume-failed-expansion] -If you attempted an expansion of an Elasticsearch volume via its [volume claim template](/deploy-manage/deploy/cloud-on-k8s/volume-claim-templates.md#k8s-volume-claim-templates-update), you may have encountered scenarios where the operation failed such as Azure not allowing volume expansion without shutting down the Virtual Machine to which it is attached. If you try to adjust the volume claim template back to the original size you will encounter a failure: +If you attempted an expansion of an {{es}} volume via its [volume claim template](/deploy-manage/deploy/cloud-on-k8s/volume-claim-templates.md#k8s-volume-claim-templates-update), you may have encountered scenarios where the operation failed such as Azure not allowing volume expansion without shutting down the Virtual Machine to which it is attached. If you try to adjust the volume claim template back to the original size you will encounter a failure: ``` Failed to apply spec change: handle volume expansion: decreasing storage size is not supported: an attempt was made to decrease storage size for claim elasticsearch-data ``` -In this scenario the best course of action is to rename the existing `nodeSet` to a new name while simultaneously updating the volume claim template to the original size. This operation will bring a new `StatefulSet` online while moving all existing indices to the new volumes and will delete the old `StatefulSet` and its volumes once the operation is complete. +In this scenario the best course of action is to rename the existing `nodeSet` to a new name while simultaneously updating the volume claim template to the original size. This operation will bring a new `StatefulSet` online while moving all existing indices to the new volumes, and will delete the old `StatefulSet` and its volumes once the operation is complete. From 6dc6eb72c98e4405ab2ee0c455c2663944c1d210 Mon Sep 17 00:00:00 2001 From: Michael Montgomery Date: Tue, 28 Oct 2025 08:38:36 -0500 Subject: [PATCH 6/8] Adjust wording for volume upsize errors Co-authored-by: Peter Brachwitz --- troubleshoot/deployments/cloud-on-k8s/common-problems.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/troubleshoot/deployments/cloud-on-k8s/common-problems.md b/troubleshoot/deployments/cloud-on-k8s/common-problems.md index 41a7de6f5e..0dd76f8161 100644 --- a/troubleshoot/deployments/cloud-on-k8s/common-problems.md +++ b/troubleshoot/deployments/cloud-on-k8s/common-problems.md @@ -310,7 +310,7 @@ GET /_security/role_mapping/ ## If a volume expansion failed [k8s-common-problems-volume-failed-expansion] -If you attempted an expansion of an {{es}} volume via its [volume claim template](/deploy-manage/deploy/cloud-on-k8s/volume-claim-templates.md#k8s-volume-claim-templates-update), you may have encountered scenarios where the operation failed such as Azure not allowing volume expansion without shutting down the Virtual Machine to which it is attached. If you try to adjust the volume claim template back to the original size you will encounter a failure: +If you attempted an expansion of an {{es}} data volume via its [volume claim template](/deploy-manage/deploy/cloud-on-k8s/volume-claim-templates.md#k8s-volume-claim-templates-update), you may have encountered scenarios where the operation failed. For example older versions of the Azure Disk CSI driver did not allow volume expansion without shutting down the Virtual Machine to which it is attached. If you try to adjust the volume claim template back to the original size you will encounter an error: ``` Failed to apply spec change: handle volume expansion: decreasing storage size is not supported: an attempt was made to decrease storage size for claim elasticsearch-data From 344ca9c81b2b28cf84bc01358bb2756f0a6959f1 Mon Sep 17 00:00:00 2001 From: Michael Montgomery Date: Tue, 28 Oct 2025 10:10:38 -0500 Subject: [PATCH 7/8] Review comments Signed-off-by: Michael Montgomery --- troubleshoot/deployments/cloud-on-k8s/common-problems.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/troubleshoot/deployments/cloud-on-k8s/common-problems.md b/troubleshoot/deployments/cloud-on-k8s/common-problems.md index 0dd76f8161..aa4b6d4b71 100644 --- a/troubleshoot/deployments/cloud-on-k8s/common-problems.md +++ b/troubleshoot/deployments/cloud-on-k8s/common-problems.md @@ -310,10 +310,10 @@ GET /_security/role_mapping/ ## If a volume expansion failed [k8s-common-problems-volume-failed-expansion] -If you attempted an expansion of an {{es}} data volume via its [volume claim template](/deploy-manage/deploy/cloud-on-k8s/volume-claim-templates.md#k8s-volume-claim-templates-update), you may have encountered scenarios where the operation failed. For example older versions of the Azure Disk CSI driver did not allow volume expansion without shutting down the Virtual Machine to which it is attached. If you try to adjust the volume claim template back to the original size you will encounter an error: +If you attempted an expansion of an {{es}} data volume via its [volume claim template](/deploy-manage/deploy/cloud-on-k8s/volume-claim-templates.md#k8s-volume-claim-templates-update), you may have encountered scenarios where the operation failed. For example older versions of the Azure Disk CSI driver did not allow volume expansion without shutting down the Virtual Machine to which the volume was attached. If you try to adjust the volume claim template back to the original size you will encounter an error: ``` Failed to apply spec change: handle volume expansion: decreasing storage size is not supported: an attempt was made to decrease storage size for claim elasticsearch-data ``` -In this scenario the best course of action is to rename the existing `nodeSet` to a new name while simultaneously updating the volume claim template to the original size. This operation will bring a new `StatefulSet` online while moving all existing indices to the new volumes, and will delete the old `StatefulSet` and its volumes once the operation is complete. +In this scenario the best course of action is to rename the existing `nodeSet` to a new name while simultaneously updating the volume claim template to the original size. This operation will bring a new `StatefulSet` online while moving all existing data to the new volumes, and will delete the old `StatefulSet` and its volumes once the operation is complete. From 6b32cf7633e9f88e7e154c01e14ec9096d7c5fc2 Mon Sep 17 00:00:00 2001 From: Michael Montgomery Date: Tue, 4 Nov 2025 09:03:00 -0600 Subject: [PATCH 8/8] Update wording/header for volume expansion failure. Co-authored-by: David Kilfoyle <41695641+kilfoyle@users.noreply.github.com> --- troubleshoot/deployments/cloud-on-k8s/common-problems.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/troubleshoot/deployments/cloud-on-k8s/common-problems.md b/troubleshoot/deployments/cloud-on-k8s/common-problems.md index aa4b6d4b71..4e8dda5785 100644 --- a/troubleshoot/deployments/cloud-on-k8s/common-problems.md +++ b/troubleshoot/deployments/cloud-on-k8s/common-problems.md @@ -308,7 +308,7 @@ GET /_security/role_mapping/ ``` -## If a volume expansion failed [k8s-common-problems-volume-failed-expansion] +## Volume expansion failed [k8s-common-problems-volume-failed-expansion] If you attempted an expansion of an {{es}} data volume via its [volume claim template](/deploy-manage/deploy/cloud-on-k8s/volume-claim-templates.md#k8s-volume-claim-templates-update), you may have encountered scenarios where the operation failed. For example older versions of the Azure Disk CSI driver did not allow volume expansion without shutting down the Virtual Machine to which the volume was attached. If you try to adjust the volume claim template back to the original size you will encounter an error: