diff --git a/deploy-manage/_snippets/serverless-upgrade.md b/deploy-manage/_snippets/serverless-upgrade.md new file mode 100644 index 0000000000..4551488cb6 --- /dev/null +++ b/deploy-manage/_snippets/serverless-upgrade.md @@ -0,0 +1,5 @@ +::::{note} +With [{{serverless-full}}](/deploy-manage/deploy/elastic-cloud/serverless.md), upgrades of the Elastic-managed infrastructure and project components are fully handled by Elastic. Users automatically receive the latest features and improvements, with no need to plan or perform project-level upgrades. + +Keep in mind that the [ingest components](/deploy-manage/upgrade/ingest-components.md) or clients you manage externally, such as {{agent}}, {{beats}}, {{ls}}, or custom applications, still need to be updated manually. +:::: \ No newline at end of file diff --git a/deploy-manage/deploy/cloud-on-k8s.md b/deploy-manage/deploy/cloud-on-k8s.md index 9eb3c31581..4dfb7e487b 100644 --- a/deploy-manage/deploy/cloud-on-k8s.md +++ b/deploy-manage/deploy/cloud-on-k8s.md @@ -91,7 +91,7 @@ ECK should work with all conformant **installers** listed in these [FAQs](https: Alpha, beta, and stable API versions follow the same [conventions used by Kubernetes](https://kubernetes.io/docs/concepts/overview/kubernetes-api/#api-versioning). -### {{stack}} compatibility +### {{stack}} compatibility [stack-compatibility] ECK is compatible with the following {{stack}} applications: diff --git a/deploy-manage/toc.yml b/deploy-manage/toc.yml index 2a95ea81a0..637eafd4b0 100644 --- a/deploy-manage/toc.yml +++ b/deploy-manage/toc.yml @@ -797,12 +797,6 @@ toc: - file: upgrade.md children: - file: upgrade/plan-upgrade.md - - file: upgrade/orchestrator.md - children: - - file: upgrade/orchestrator/upgrade-cloud-enterprise.md - children: - - file: upgrade/orchestrator/re-running-the-ece-upgrade.md - - file: upgrade/orchestrator/upgrade-cloud-on-k8s.md - file: upgrade/prepare-to-upgrade.md children: - file: upgrade/prepare-to-upgrade/upgrade-assistant.md @@ -823,6 +817,12 @@ toc: - file: upgrade/deployment-or-cluster/kibana-roll-back.md - file: upgrade/deployment-or-cluster/enterprise-search.md - file: upgrade/ingest-components.md + - file: upgrade/orchestrator.md + children: + - file: upgrade/orchestrator/upgrade-cloud-enterprise.md + children: + - file: upgrade/orchestrator/re-running-the-ece-upgrade.md + - file: upgrade/orchestrator/upgrade-cloud-on-k8s.md - file: uninstall.md children: - file: uninstall/uninstall-elastic-cloud-enterprise.md diff --git a/deploy-manage/upgrade.md b/deploy-manage/upgrade.md index 6d4b1e5ab3..a227a8e6fb 100644 --- a/deploy-manage/upgrade.md +++ b/deploy-manage/upgrade.md @@ -1,14 +1,99 @@ +--- +applies_to: + stack: + deployment: + eck: + ess: + ece: + self: +products: + - id: kibana + - id: cloud-enterprise + - id: cloud-hosted + - id: cloud-kubernetes + - id: elasticsearch +--- + # Upgrade -Upgrading to the latest version provides access to the newest Elastic features, enhancements, performance improvements, and bug fixes. These updates reduce costs, speed up threat response, and improve investigative and analytical data tools. +Upgrading to the latest version provides access to the newest Elastic features, security patches, performance improvements, and bug fixes. These updates reduce costs, speed up threat response, and improve investigative and analytical data tools. -:::{important} -In {{stack}} 9.0.0 and beyond, Enterprise Search is unavailable. For more information, refer to [Migrating to 9.x from Enterprise Search 8.x versions](https://www.elastic.co/guide/en/enterprise-search/8.18/upgrading-to-9-x.html). +As Elastic releases new versions, older versions of Elastic products reach their end of life on a set schedule. To keep your deployment supported, stay up to date. For more information, refer to [Product End of Life Dates](https://www.elastic.co/support/eol). + +:::{include} /deploy-manage/_snippets/serverless-upgrade.md ::: -When Elastic releases new versions, older versions reach their end of life on a set schedule. To keep your deployment supported, stay up to date. For more information, refer to [Product End of Life Dates](https://www.elastic.co/support/eol). +The upgrade procedure depends on how your deployment is managed. If you're using {{ech}} or {{ece}}, upgrades can often be performed with a single click in the {{ecloud}} UI. For self-managed deployments, upgrades must be carried out manually using a rolling upgrade process, upgrading the nodes one by one to minimize downtime and ensure cluster stability. + +This section provides guidance to help you plan and safely perform upgrades of your {{stack}} components, with a primary focus on {{es}} and {{kib}} as the core of the stack. It also covers upgrades for orchestration platforms like {{ece}} and {{eck}}, as well as related components such as APM, {{beats}}, {{agent}}, and {{ls}}. -:::{note} -Upgrading from a release candidate build, such as 9.0.0-rc1, is unsupported. Use pre-releases only for testing in a temporary environment. +:::{important} +In {{stack}} 9.0.0 and beyond, Enterprise Search is unavailable. For more information, refer to [Migrating to 9.x from Enterprise Search 8.x versions](https://www.elastic.co/guide/en/enterprise-search/8.19/upgrading-to-9-x.html). ::: +## Upgrade overview + +Upgrading your Elastic cluster or deployment involves several stages, including planning, preparation, and execution. This section guides you through the full upgrade process: + +- [Plan your upgrade](./upgrade/plan-upgrade.md): Review compatibility, define your upgrade path and order, and understand important pre-upgrade considerations. + +- [Prepare to upgrade](./upgrade/prepare-to-upgrade.md): Follow detailed preparation steps for major, minor, and patch upgrades. Identify breaking changes, run the Upgrade Assistant (for major upgrades), and verify readiness. + +- [Upgrade your deployment or cluster](./upgrade/deployment-or-cluster.md): Step-by-step instructions for performing the upgrade, organized by deployment type: + + - [Upgrade deployments on {{ech}}](/deploy-manage/upgrade/deployment-or-cluster/upgrade-on-ech.md) + - [Upgrade deployments on {{ece}}](/deploy-manage/upgrade/deployment-or-cluster/upgrade-on-ece.md) + - [Upgrade deployments on {{eck}}](/deploy-manage/upgrade/deployment-or-cluster/upgrade-on-eck.md) + - [Upgrade self-managed clusters](/deploy-manage/upgrade/deployment-or-cluster/self-managed.md) + +- [Upgrade ingest components](./upgrade/ingest-components.md): Covers supporting components such as {{beats}}, {{agent}}, {{ls}}, and APM Server. + +Additionally, if you're using a self-managed orchestration platform such as {{ece}} or {{eck}}, refer to [Upgrade your ECE or ECK orchestrator](/deploy-manage/upgrade/orchestrator.md) to keep the orchestrator up to date. + +## Availability during upgrades + +{{es}} supports rolling upgrades, where nodes are upgraded one at a time to minimize downtime and ensure cluster stability. If your deployment or cluster follows high availability best practices, such as using replicated indices and distributing nodes across multiple availability zones, it should remain available throughout the entire upgrade process. + +{{kib}} upgrades, however, require downtime. All running instances must be shut down before starting the upgrade. + +::::{important} +Ensure your deployment is properly sized and configured for high availability to reduce the risk of service interruption during upgrades. + +If you want to learn more about scaling, high availability, and performance, refer to: +* [Run {{es}} in production](/deploy-manage/production-guidance/elasticsearch-in-production-environments.md) +* [Run {{kib}} in production](/deploy-manage/production-guidance/kibana-in-production-environments.md) +* [{{ech}} > Plan for production](/deploy-manage/deploy/elastic-cloud/elastic-cloud-hosted-planning.md) +:::: + +You’ll find more information about availability considerations for specific deployment types in the related upgrade guides within this section. + +## Upgrade paths [upgrade-paths] + +You can upgrade to a higher version if the target version was released *after* your current version. Upgrades to versions released *before* your current version are not supported, even if the version number is higher. Refer to [out-of-order releases](/deploy-manage/upgrade/deployment-or-cluster.md#out-of-order-releases) for more information. + +For example: +- ✅ Upgrade allowed: From 9.0.4 to 9.1.0 (9.1.0 released *after* 9.0.4) +- ❌ Not allowed: From 9.0.5 to 9.1.0 (9.1.0 released *before* 9.0.5) → wait for 9.1.1 to be released + + +### Upgrade paths from 8.x [upgrade-paths-8.x] + +To perform a major upgrade from 8.x, the required starting version depends on the target release: + +- To upgrade to the **9.0.x series**, you must be on **8.18.x**. +- To upgrade to **9.1.0 or later**, you must be on **8.19.x**, which is the latest minor release of the 8.x series. + +::::{note} +While 8.19 is the final minor release in the 8.x series, 8.18 was released at the same time as 9.0, enabling a supported upgrade path between the 8.18.x and 9.0.x series. This compatibility also applies to other features and clients. +:::: + +If you're upgrading to the current {{version.stack}} release from an earlier 8.x version, first upgrade to the latest available 8.19 release. + +#### Ingest tools and clients considerations + +For flexible upgrade scheduling, 8.19 {{agent}}, {{beats}}, and {{ls}} are compatible with all 9.x versions of {{es}}. + +By default, 8.x {{es}} clients are compatible with 9.x and use [REST API compatibility](elasticsearch://reference/elasticsearch/rest-apis/compatibility.md) to maintain compatibility with the 9.x cluster. diff --git a/deploy-manage/upgrade/deployment-or-cluster.md b/deploy-manage/upgrade/deployment-or-cluster.md index 0fcb4103ce..d53cfa2055 100644 --- a/deploy-manage/upgrade/deployment-or-cluster.md +++ b/deploy-manage/upgrade/deployment-or-cluster.md @@ -28,17 +28,38 @@ products: # Upgrade your deployment or cluster [upgrade-deployment-cluster] -When upgrading an existing cluster, you perform a minor or major upgrade. For example, a minor upgrade takes you from version 9.0.0 to 9.1.0, while a major upgrade takes you from version 8.0.0 to 9.0.0. +This section contains the upgrade instructions for {{es}} clusters and {{kib}} instances. Upgrade procedures depend on whether you installed Elastic components using Elastic-managed or self-managed infrastructure. -Upgrade procedures depend on whether you installed Elastic components using Elastic-managed or self-managed infrastructure. +## Prerequisites + +Before proceeding with the upgrade, review the [Plan your upgrade](/deploy-manage/upgrade/plan-upgrade.md) guidance to understand compatibility and timing considerations, and follow the steps in [Prepare to upgrade](/deploy-manage/upgrade/prepare-to-upgrade.md) to get your environment ready for the upgrade. + +## Out-of-order releases [out-of-order-releases] + +Elastic maintains several minor versions of {{es}} at the same time. This means releases do not always happen in order of their version numbers. You can only upgrade to {{version.stack}} if the version you are currently running meets both of these conditions: + +* Has a lower version number than {{version.stack}} +* Has an earlier release date than {{version.stack}} + +If you are currently running a version with a lower version number but a later release date than {{version.stack}}, wait for a newer release before upgrading. + +Additionally, upgrading from a release candidate build, such as 9.0.0-rc1, is unsupported. Use pre-releases only for testing in a temporary environment. + +## Upgrade methods If you’re using Elastic-managed infrastructure, use the following options: * [Upgrade on {{ech}}](/deploy-manage/upgrade/deployment-or-cluster/upgrade-on-ech.md) -* Upgrade on [{{serverless-full}}](/deploy-manage/deploy/elastic-cloud/serverless.md), which is automatically performed by Elastic and requires no user management If you’re using self-managed infrastructure - either on-prem or public cloud - use the following options: * [Upgrade the {{stack}} on a self-managed cluster](/deploy-manage/upgrade/deployment-or-cluster/self-managed.md) -* [Upgrade on {{ece}} (ECE)](/deploy-manage/upgrade/deployment-or-cluster/upgrade-on-ece.md) -* [Upgrade on {{eck}} (ECK)](/deploy-manage/upgrade/deployment-or-cluster/upgrade-on-eck.md) +* [Upgrade your deployment on {{ece}} (ECE)](/deploy-manage/upgrade/deployment-or-cluster/upgrade-on-ece.md) +* [Upgrade your deployment on {{eck}} (ECK)](/deploy-manage/upgrade/deployment-or-cluster/upgrade-on-eck.md) + +:::{include} /deploy-manage/_snippets/serverless-upgrade.md +::: + +## Next steps + +Once you've successfully upgraded your deployment, you can [upgrade your ingest components](./ingest-components.md), such as {{ls}}, {{agent}}, or {{beats}}. \ No newline at end of file diff --git a/deploy-manage/upgrade/deployment-or-cluster/archived-settings.md b/deploy-manage/upgrade/deployment-or-cluster/archived-settings.md index 693da00a27..12d49515f7 100644 --- a/deploy-manage/upgrade/deployment-or-cluster/archived-settings.md +++ b/deploy-manage/upgrade/deployment-or-cluster/archived-settings.md @@ -1,6 +1,13 @@ --- mapped_pages: - https://www.elastic.co/guide/en/elasticsearch/reference/current/archived-settings.html +applies_to: + stack: + deployment: + eck: + ess: + ece: + self: products: - id: elasticsearch --- diff --git a/deploy-manage/upgrade/deployment-or-cluster/elasticsearch.md b/deploy-manage/upgrade/deployment-or-cluster/elasticsearch.md index 2d458b2a28..d8adc4e5de 100644 --- a/deploy-manage/upgrade/deployment-or-cluster/elasticsearch.md +++ b/deploy-manage/upgrade/deployment-or-cluster/elasticsearch.md @@ -3,20 +3,39 @@ applies_to: stack: deployment: self: all +products: + - id: elasticsearch --- # Upgrade {{es}} [upgrading-elasticsearch] An {{es}} cluster can be upgraded one node at a time so upgrading does not interrupt service. Running multiple versions of {{es}} in the same cluster beyond the duration of an upgrade is not supported, as shards cannot be replicated from upgraded nodes to nodes running the older version. -Before you start, [take the upgrade preparation steps](/deploy-manage/upgrade/prepare-to-upgrade.md). When performing a [rolling upgrade](#rolling-upgrades): +:::{note} +Upgrading from a release candidate build, such as 9.0.0-rc1, is unsupported. Use pre-releases only for testing in a temporary environment. +::: -1. Upgrade the data nodes first, tier-by-tier, starting with the frozen tier, then the cold tier, then the warm tier, then the hot tier, and finally any other data nodes which are not in a tier. Complete the upgrade for all nodes in each data tier before moving to the next. This ensures {{ilm-init}} can continue to move data through the tiers during the upgrade. You can get the list of nodes in a specific tier with a `GET /_nodes` request, for example: `GET /_nodes/data_frozen:true/_none`. +Before you start the rolling upgrade procedure, [plan your upgrade](/deploy-manage/upgrade/plan-upgrade.md) and [take the upgrade preparation steps](/deploy-manage/upgrade/prepare-to-upgrade.md). + +## {{es}} nodes upgrade order + +When performing a [rolling upgrade](#rolling-upgrades): + +1. Upgrade the data nodes first, tier-by-tier, in the following order: + 1. The frozen tier + 2. The cold tier + 3. The warm tier + 4. The hot tier + 5. Any other data nodes which are not in a tier. + + Complete the upgrade for all nodes in each data tier before moving to the next. This ensures {{ilm-init}} can continue to move data through the tiers during the upgrade. You can get the list of nodes in a specific tier with a `GET /_nodes` request, for example: `GET /_nodes/data_frozen:true/_none`. 2. Upgrade all remaining nodes that are neither master-eligible nor data nodes. This includes dedicated ML nodes, dedicated ingest nodes, and dedicated coordinating nodes. 3. Upgrade the master-eligible nodes last. You can retrieve a list of these nodes with `GET /_nodes/master:true/_none`. This order ensures that all nodes can join the cluster during the upgrade. Upgraded nodes can join a cluster with an older master, but older nodes cannot always join a cluster with a upgraded master. +## Upgrade process + To upgrade a cluster: 1. **Disable shard allocation**. @@ -45,10 +64,9 @@ To upgrade a cluster: It is possible to leave your {{ml}} jobs running during the upgrade, but it puts increased load on the cluster. When you shut down a {{ml}} node, its jobs automatically move to another node and restore the model states. ::::{note} - Any {{ml}} indices created before 8.x must be reindexed before upgrading, which you can initiate from the **Upgrade Assistant** in 8.18. For more information, refer to [Anomaly detection results migration] + Any {{ml}} indices created before 8.x must be reindexed before upgrading, which you can initiate from the **Upgrade Assistant** in 8.19. For more information, refer to [Anomaly detection results migration] :::: - * Temporarily halt the tasks associated with your {{ml}} jobs and {{dfeeds}} and prevent new jobs from opening by using the [set upgrade mode API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ml-set-upgrade-mode): ```console @@ -187,7 +205,7 @@ To upgrade a cluster: -## Rolling upgrades [rolling-upgrades] +## Rolling upgrades considerations [rolling-upgrades] During a rolling upgrade, the cluster continues to operate normally. However, any new functionality is disabled or operates in a backward compatible mode until all nodes in the cluster are upgraded. New functionality becomes operational once the upgrade is complete and all nodes are running the new version. Once that has happened, there’s no way to return to operating in a backward compatible mode. Nodes running the previous version will not be allowed to join the fully-updated cluster. diff --git a/deploy-manage/upgrade/deployment-or-cluster/enterprise-search.md b/deploy-manage/upgrade/deployment-or-cluster/enterprise-search.md index 4d08480834..4733c70c49 100644 --- a/deploy-manage/upgrade/deployment-or-cluster/enterprise-search.md +++ b/deploy-manage/upgrade/deployment-or-cluster/enterprise-search.md @@ -12,4 +12,4 @@ applies_to: In {{stack}} 9.0.0 and beyond, Enterprise Search is unavailable. We recommend migrating to {{es}} for improved performance, scalability, and flexibility. For information about what {{es}} features you can use, check out this [blog post](https://www.elastic.co/blog/app-search-to-elasticsearch). -For complete information about resources and initiatives related to App Search migrations, future enhancements, and decommissioning, refer to [Migrating to 9.x from Enterprise Search 8.x versions](https://www.elastic.co/guide/en/enterprise-search/8.18/upgrading-to-9-x.html). \ No newline at end of file +For complete information about resources and initiatives related to App Search migrations, future enhancements, and decommissioning, refer to [Migrating to 9.x from Enterprise Search 8.x versions](https://www.elastic.co/guide/en/enterprise-search/8.19/upgrading-to-9-x.html). \ No newline at end of file diff --git a/deploy-manage/upgrade/deployment-or-cluster/kibana-roll-back.md b/deploy-manage/upgrade/deployment-or-cluster/kibana-roll-back.md index 42e25bf135..944559493d 100644 --- a/deploy-manage/upgrade/deployment-or-cluster/kibana-roll-back.md +++ b/deploy-manage/upgrade/deployment-or-cluster/kibana-roll-back.md @@ -2,6 +2,9 @@ navigation_title: Roll back to a previous version mapped_pages: - https://www.elastic.co/guide/en/kibana/current/upgrade-migrations-rolling-back.html +applies_to: + deployment: + self: all products: - id: kibana --- diff --git a/deploy-manage/upgrade/deployment-or-cluster/kibana.md b/deploy-manage/upgrade/deployment-or-cluster/kibana.md index 12a05204cf..1f1c723b78 100644 --- a/deploy-manage/upgrade/deployment-or-cluster/kibana.md +++ b/deploy-manage/upgrade/deployment-or-cluster/kibana.md @@ -3,20 +3,22 @@ applies_to: stack: deployment: self: all +products: + - id: kibana --- # Upgrade {{kib}} [upgrade-kibana] When you upgrade {{kib}}, you also upgrade the {{observability}} and {{elastic-sec}} solutions, which use {{kib}} as their main interface. +{{kib}} must always be upgraded after {{es}}, and to the same version. Version mismatches or upgrading in the wrong order can result in failures or unexpected behavior. + ::::{warning} {{kib}} automatically runs saved object migrations when required. To roll back to an earlier version in case of an upgrade failure, you **must** have a [backup snapshot](../../tools/snapshot-and-restore.md) that includes the `kibana` feature state. Snapshots include this feature state by default. For more information, refer to [Migrate saved objects](saved-object-migrations.md). - :::: - ## Upgrading multiple {{kib}} instances [_upgrading_multiple_kib_instances] When upgrading several {{kib}} instances connected to the same {{es}} cluster, ensure that all outdated instances are shut down before starting the upgrade. @@ -27,8 +29,7 @@ For large deployments with more than 10 {{kib}} instances, and more than 10,000 ## Preparing for upgrading [preventing-migration-failures] -Before you start, ensure you [take the upgrade preparation steps](/deploy-manage/upgrade/prepare-to-upgrade.md). Then, take these extra steps to ensure you are ready to upgrade. - +Before you start, ensure that you’ve followed the [Plan your upgrade](/deploy-manage/upgrade/plan-upgrade.md) guidelines, completed the [upgrade preparation steps](/deploy-manage/upgrade/prepare-to-upgrade.md), and [upgraded the {{es}} cluster](./elasticsearch.md). ### Ensure your {{es}} cluster is healthy [_ensure_your_es_cluster_is_healthy] @@ -53,21 +54,20 @@ To upgrade {{kib}}: 1. Shut down all {{kib}} instances. {{kib}} does not support rolling upgrades. **Upgrading while older {{kib}} instances are running can cause data loss or upgrade failures.** 2. To install the `deb` or `rpm` package: - - a. Use `rpm` or `dpkg`. This installs all files in their proper locations and will not overwrite the config files. - b. Upgrade any plugins by removing the existing plugin and reinstalling the appropriate version using the `kibana-plugin` script. For more information, refer to [{{kib}} plugins](kibana://reference/kibana-plugins.md). + 1. Use `rpm` or `dpkg`. This installs all files in their proper locations and will not overwrite the config files. + 2. Upgrade any plugins by removing the existing plugin and reinstalling the appropriate version using the `kibana-plugin` script. For more information, refer to [{{kib}} plugins](kibana://reference/kibana-plugins.md). 3. To install from a `zip` or `tar.gz` archive: - a. **Extract the archive to a new directory** to be sure that you don’t overwrite the `config` or `data` directories. - b. Copy the files from the `config` directory from your old installation to your new installation. - c. Copy the files from the `data` directory from your old installation to your new installation. + 1. **Extract the archive to a new directory** to be sure that you don’t overwrite the `config` or `data` directories. + 2. Copy the files from the `config` directory from your old installation to your new installation. + 3. Copy the files from the `data` directory from your old installation to your new installation. ::::{important} If you use {{monitor-features}}, you must re-use the data directory when you upgrade {{kib}}. Otherwise, the {{kib}} instance is assigned a new persistent UUID and becomes a new instance in the monitoring data. :::: - d. Install the appropriate versions of all your plugins for your new installation using the `kibana-plugin` script. For more information, refer to [{{kib}} plugins](kibana://reference/kibana-plugins.md). + 4. Install the appropriate versions of all your plugins for your new installation using the `kibana-plugin` script. For more information, refer to [{{kib}} plugins](kibana://reference/kibana-plugins.md). 4. Start {{kib}}. diff --git a/deploy-manage/upgrade/deployment-or-cluster/reading-indices-from-older-elasticsearch-versions.md b/deploy-manage/upgrade/deployment-or-cluster/reading-indices-from-older-elasticsearch-versions.md index ce92584833..afc9d495d2 100644 --- a/deploy-manage/upgrade/deployment-or-cluster/reading-indices-from-older-elasticsearch-versions.md +++ b/deploy-manage/upgrade/deployment-or-cluster/reading-indices-from-older-elasticsearch-versions.md @@ -6,6 +6,8 @@ applies_to: deployment: ess: self: + eck: + ece: products: - id: elasticsearch --- diff --git a/deploy-manage/upgrade/deployment-or-cluster/self-managed.md b/deploy-manage/upgrade/deployment-or-cluster/self-managed.md index f855f3219d..94d108095b 100644 --- a/deploy-manage/upgrade/deployment-or-cluster/self-managed.md +++ b/deploy-manage/upgrade/deployment-or-cluster/self-managed.md @@ -4,6 +4,9 @@ applies_to: stack: deployment: self: +products: + - id: elasticsearch + - id: kibana --- # Upgrade the {{stack}} on a self-managed cluster diff --git a/deploy-manage/upgrade/deployment-or-cluster/upgrade-on-ece.md b/deploy-manage/upgrade/deployment-or-cluster/upgrade-on-ece.md index fc91767ef2..72bd3320bb 100644 --- a/deploy-manage/upgrade/deployment-or-cluster/upgrade-on-ece.md +++ b/deploy-manage/upgrade/deployment-or-cluster/upgrade-on-ece.md @@ -4,6 +4,10 @@ applies_to: stack: deployment: ece: +products: + - id: kibana + - id: cloud-enterprise + - id: elasticsearch --- # Upgrade your deployment on {{ece}} (ECE) @@ -12,10 +16,8 @@ A single click in the {{ecloud}} console can upgrade a deployment running on ECE Once you're [prepared to upgrade](/deploy-manage/upgrade/prepare-to-upgrade.md), do the following: -% Note: Add a link once confirmed where prepare to upgrade will reside in TOC. - 1. Ensure your current ECE and Docker or Podman versions are [compatible](https://www.elastic.co/support/matrix/#elastic-cloud-enterprise) with the {{stack}} version you're upgrading to. For example, if you're upgrading to 9.0.0, the minimum required version is ECE 4.0. If you don’t have a compatible version installed, [upgrade your orchestrator](/deploy-manage/upgrade/orchestrator/upgrade-cloud-enterprise.md). -2. Download the most recent [stack pack](/deploy-manage/deploy/cloud-enterprise/manage-elastic-stack-versions.md#ece_most_recent_elastic_stack_packs) for the version you’re upgrading to, then [add the stack pack](/deploy-manage/deploy/cloud-enterprise/manage-elastic-stack-versions.md#ece-manage-elastic-stack-add) to your installation using the Cloud UI. +2. Download the most recent [stack pack](/deploy-manage/deploy/cloud-enterprise/manage-elastic-stack-versions.md#ece_most_recent_elastic_stack_packs) for the version you’re upgrading to, then [add the stack pack](/deploy-manage/deploy/cloud-enterprise/manage-elastic-stack-versions.md#ece-manage-elastic-stack-add) to your ECE installation using the Cloud UI. 3. If not configured already, [assign a snapshots repository](/deploy-manage/tools/snapshot-and-restore/cloud-enterprise.md) to your deployment to enable snapshots and back up your data. Although this is optional, we recommend this step. ## Perform the upgrade diff --git a/deploy-manage/upgrade/deployment-or-cluster/upgrade-on-ech.md b/deploy-manage/upgrade/deployment-or-cluster/upgrade-on-ech.md index d19c5d75fd..aee8cf5ea1 100644 --- a/deploy-manage/upgrade/deployment-or-cluster/upgrade-on-ech.md +++ b/deploy-manage/upgrade/deployment-or-cluster/upgrade-on-ech.md @@ -10,13 +10,15 @@ applies_to: ess: products: - id: cloud-hosted + - id: elasticsearch + - id: kibana --- # Upgrade on {{ech}} (ECH) Once you are [prepared to upgrade](/deploy-manage/upgrade/prepare-to-upgrade.md), a single click in the {{ecloud}} console can upgrade a deployment to a newer version, add more processing capacity, change plugins, and enable or disable high availability, all at the same time. During the upgrade process, {{es}}, {{kib}}, Elastic APM, and all of your deployment components are upgraded simultaneously. -{{ecloud}} only supports upgrades to released versions. Release candidate builds and master snapshots are not supported. +{{ecloud}} only supports and enables upgrades to released versions that comply with the supported [upgrade paths](/deploy-manage/upgrade.md#upgrade-paths). Release candidate builds and master snapshots are not supported. ::::{important} Although it’s simple to upgrade an {{ecloud}} deployment, the new version might include breaking changes that affect your application. Ensure you review breaking changes and deprecation logs, make any necessary changes, and test against the new version before upgrading your production deployment. @@ -24,7 +26,7 @@ Although it’s simple to upgrade an {{ecloud}} deployment, the new version migh ## Availability during upgrades -For deployments with nodes in more than one availability zone, you can perform minor version upgrades, upgrades from 8.18 to 9.0.0, and cluster configuration changes with no downtime. Deployments with nodes in only one zone might experience downtime during these operations. +For deployments with nodes in more than one availability zone, you can perform upgrades and cluster configuration changes with no downtime. Deployments with nodes in only one zone might experience downtime during these operations. When {{kib}} instances are upgraded, all instances are shut down simultaneously, making {{kib}} temporarily inaccessible, even if it's deployed across multiple zones. diff --git a/deploy-manage/upgrade/deployment-or-cluster/upgrade-on-eck.md b/deploy-manage/upgrade/deployment-or-cluster/upgrade-on-eck.md index 7c543807a5..16c0f1f560 100644 --- a/deploy-manage/upgrade/deployment-or-cluster/upgrade-on-eck.md +++ b/deploy-manage/upgrade/deployment-or-cluster/upgrade-on-eck.md @@ -4,29 +4,33 @@ applies_to: stack: deployment: eck: +products: + - id: kibana + - id: cloud-kubernetes + - id: elasticsearch --- # Upgrade your deployment on {{eck}} (ECK) The ECK orchestrator can safely perform upgrades to newer versions of the {{stack}}. -Once you're [prepared to upgrade](/deploy-manage/upgrade/prepare-to-upgrade.md), ensure the ECK version is [compatible](/deploy-manage/deploy/cloud-on-k8s.md) with the {{stack}} version you’re upgrading to. For example, if you're upgrading to 9.0.0, the minimum required ECK version is 3.0.0. If it's incompatible, [upgrade your orchestrator](/deploy-manage/upgrade/orchestrator/upgrade-cloud-on-k8s.md). +Before you start the upgrade, [plan your upgrade](/deploy-manage/upgrade/plan-upgrade.md), [take the upgrade preparation steps](/deploy-manage/upgrade/prepare-to-upgrade.md), and ensure your ECK version is [compatible](/deploy-manage/deploy/cloud-on-k8s.md#stack-compatibility) with the {{stack}} version you’re upgrading to. If it's incompatible, [upgrade your orchestrator](/deploy-manage/upgrade/orchestrator/upgrade-cloud-on-k8s.md) first. ## Perform the upgrade 1. In the resource spec file, modify the `version` field for the desired {{stack}} version. 2. Save your changes. The orchestrator will start the upgrade process automatically. -In this example, we’re modifying the version to `9.0.0`. +In this example, we’re modifying the version to {{version.stack}}. -```yaml +```yaml subs=true apiVersion: elasticsearch.k8s.elastic.co/v1 kind: Elasticsearch metadata: name: elasticsearch-sample namespace: production spec: - version: 9.0.0 + version: {{version.stack}} monitoring: metrics: elasticsearchRefs: @@ -117,7 +121,7 @@ metadata: name: kibana-sample namespace: production spec: - version: 9.0.0 + version: {{version.stack}} monitoring: metrics: elasticsearchRefs: @@ -142,4 +146,4 @@ Check out [Nodes orchestration](/deploy-manage/deploy/cloud-on-k8s/nodes-orchest ## Next steps -Once you've successfully upgraded your deployment, [upgrade your ingest components](/deploy-manage/upgrade/ingest-components.md), such as {{ls}}, {{agents}}, or {{beats}}. +Once you've successfully upgraded your deployment, you can [upgrade your ingest components](/deploy-manage/upgrade/ingest-components.md), such as {{ls}}, {{agents}}, or {{beats}}. diff --git a/deploy-manage/upgrade/ingest-components.md b/deploy-manage/upgrade/ingest-components.md index 27375b753c..53072f74f3 100644 --- a/deploy-manage/upgrade/ingest-components.md +++ b/deploy-manage/upgrade/ingest-components.md @@ -6,12 +6,18 @@ applies_to: ess: ece: self: + serverless: --- # Upgrade your ingest components Once you've successfully [upgraded your deployment or cluster](/deploy-manage/upgrade/deployment-or-cluster.md), the final step is to update your ingest components and clients in the following order: 1. {{agent}}: [Upgrade instructions](../../reference/fleet/upgrade-elastic-agent.md) + + ::::{note} + If you plan to upgrade {{fleet}}-managed {{agent}}s, start by upgrading the agent running as the {{fleet-server}}. + :::: + 2. {{beats}}: [Upgrade instructions](beats://reference/libbeat/upgrading.md) 3. {{ls}}: [Upgrade instructions](logstash://reference/upgrading-logstash.md) -4. Custom clients \ No newline at end of file +4. Custom clients diff --git a/deploy-manage/upgrade/orchestrator.md b/deploy-manage/upgrade/orchestrator.md index 5cc71f3a22..9f5b466442 100644 --- a/deploy-manage/upgrade/orchestrator.md +++ b/deploy-manage/upgrade/orchestrator.md @@ -4,6 +4,9 @@ applies_to: deployment: ece: eck: +products: + - id: cloud-enterprise + - id: cloud-kubernetes --- # Upgrade your orchestrator @@ -14,7 +17,7 @@ The topics in this section apply to customers running the {{stack}} on {{ece}} ( Elastic provides customers with two major self-managed orchestrators to manage the {{stack}}. Before upgrading the products in the {{stack}}, ensure your orchestrator is running a compatible version. If your orchestrator is running an incompatible with the {{stack}} version you’re upgrading to, you must upgrade the orchestrator first. -If your orchestrator is up to date, you can skip this step and proceed with [Upgrading on ECE](/deploy-manage/upgrade/deployment-or-cluster/upgrade-on-ece.md) or [Upgrading on ECK](/deploy-manage/upgrade/deployment-or-cluster/upgrade-on-eck.md). +If your orchestrator is up to date, you can skip this step and proceed with [Upgrading your deployment on ECE](/deploy-manage/upgrade/deployment-or-cluster/upgrade-on-ece.md) or [Upgrading your deployment on ECK](/deploy-manage/upgrade/deployment-or-cluster/upgrade-on-eck.md). To learn how to upgrade your orchestrator, refer to one of these topics: diff --git a/deploy-manage/upgrade/orchestrator/re-running-the-ece-upgrade.md b/deploy-manage/upgrade/orchestrator/re-running-the-ece-upgrade.md index 8ecf23fa17..416a8d0382 100644 --- a/deploy-manage/upgrade/orchestrator/re-running-the-ece-upgrade.md +++ b/deploy-manage/upgrade/orchestrator/re-running-the-ece-upgrade.md @@ -2,6 +2,8 @@ applies_to: deployment: ece: +products: + - id: cloud-enterprise --- # Re-running the ECE upgrade [re-running-ece-upgrade] diff --git a/deploy-manage/upgrade/plan-upgrade.md b/deploy-manage/upgrade/plan-upgrade.md index 86c9df0cba..b11645a945 100644 --- a/deploy-manage/upgrade/plan-upgrade.md +++ b/deploy-manage/upgrade/plan-upgrade.md @@ -1,30 +1,82 @@ +--- +applies_to: + stack: + deployment: + eck: + ess: + ece: + self: +products: + - id: kibana + - id: cloud-enterprise + - id: cloud-hosted + - id: cloud-kubernetes + - id: elasticsearch +--- # Plan your upgrade -There are several things you need to plan for before performing the actual upgrade, so create a test plan. Consider the following recommendations: +There are several important factors to consider before starting the upgrade process. Use the following recommendations to build your upgrade plan: -* Plan for an appropriate amount of time to complete the upgrade. Depending on your configuration and the size of your cluster, the process can take up to a few weeks or more to complete. -* Consider opening a [support case](https://support.elastic.co/) with Elastic to alert our Elastic Support team of your system change. If you need additional assistance, [Elastic Consulting Services](https://www.elastic.co/consulting) provides the technical expertise and step-by-step approach for upgrading your Elastic deployment. +* Plan for an appropriate amount of time to complete the upgrade. Depending on your configuration and the size of your cluster, the process might take just a few minutes or several hours. In more complex environments, the process can take up to a few weeks or more to complete. +* Consider opening a [support case](https://support.elastic.co/) with Elastic to alert our Elastic Support team of your system change. If you need additional assistance, [Elastic Consulting Services](https://www.elastic.co/consulting) provides the technical expertise and step-by-step approach for upgrading your environment. * Schedule a system maintenance window within your organization. +* When possible, perform testing of the upgrade process in a non-production environment. -## Check system requirements [check-system-requirements] +The objective of this section is to facilitate the creation of an upgrade plan that addresses all the necessary steps and preparations needed for upgrading your deployment or cluster. -Ensure the version you’re upgrading to for {{es}}, {{kib}}, and any ingest components supports your current operating system. Refer to the [Product and Operating System support matrix](https://www.elastic.co/support/matrix#matrix_os). +## Compatibility checks -### OpenJDK compatibility and FIPS compliance +Before upgrading, verify that your current environment supports the version you plan to upgrade to. If not, identify any required intermediate upgrades or component changes and include them in your upgrade plan. -By default, {{es}} is built using Java and includes a bundled version of [OpenJDK](https://openjdk.java.net/) within each distribution. While we strongly recommend using the bundled Java Virtual Machine (JVM) in all installations of {{es}}, if you choose to use your own JVM, ensure it’s compatible by reviewing the [Product and JVM support matrix](https://www.elastic.co/support/matrix#matrix_jvm). +**{{es}} upgrade path** +: Check the [upgrade paths](../upgrade.md#upgrade-paths) to determine whether you must upgrade through an intermediate version (such as 8.19.x before moving 9.1+), or if you can upgrade directly to the target version. + +**System requirements** +: Ensure your current operating system is supported by the target versions of {{es}}, {{kib}}, and any ingest components. Refer to the [Product and Operating System support matrix](https://www.elastic.co/support/matrix#matrix_os). + +**OpenJDK compatibility and FIPS compliance** +: By default, {{es}} is built using Java and includes a bundled version of [OpenJDK](https://openjdk.java.net/) in each distribution. While we strongly recommend using the bundled Java Virtual Machine (JVM) in all installations of {{es}}, if you choose to use your own JVM, ensure it’s compatible by reviewing the [Product and JVM support matrix](https://www.elastic.co/support/matrix#matrix_jvm). + + If you’re running {{es}} in FIPS 140-2 mode, we recommend using [Bouncy Castle](https://www.bouncycastle.org/java.html) as a Java security provider when running {{es}}. + +**Ingest component compatibility** +: Confirm that your ingest components, such as {{beats}}, {{ls}}, or {{agent}}, are compatible with the target {{es}} version. If they’re not, upgrade them first. Refer to [conduct a component inventory](#conduct-a-component-inventory) for guidance. + +**Rest API compatibility** +: If you use custom-developed applications or clients, ensure the [{{es}} client libraries](/reference/elasticsearch-clients/index.md) are compatible with the target version. If your applications use deprecated or removed APIs, then you might need to update the client code first. + + :::{note} + By default, 8.x {{es}} clients are compatible with 9.x and use [REST API compatibility](elasticsearch://reference/elasticsearch/rest-apis/compatibility.md) to maintain compatibility with the 9.x {{es}} cluster. + REST API compatibility is a per-request opt-in feature that can help REST clients mitigate non-compatible (breaking) changes to the REST API. + ::: + +**Index compatibility** +: {{es}} provides full query and write support for indices created in the previous major version. If you have indices created in 7.x or earlier, you must reindex, delete, or [archive](/deploy-manage/upgrade/deployment-or-cluster/reading-indices-from-older-elasticsearch-versions.md) them before upgrading to 9.x. + + This topic is covered during the [upgrade preparations](prepare-to-upgrade.md#prepare-upgrade-from-8.x), with help from the Upgrade Assistant. + +**Orchestrator compatibility** +: If you’re using an orchestrator like {{ece}} or {{eck}}, verify that it supports the target {{stack}} version. If not, [upgrade the orchestrator](/deploy-manage/upgrade/orchestrator.md) before upgrading your cluster. Refer to: + * [ECE – Stack packs compatibility](/deploy-manage/deploy/cloud-enterprise/manage-elastic-stack-versions.md#ece_most_recent_elastic_stack_packs) + * [ECK – {{stack}} compatibility](/deploy-manage/deploy/cloud-on-k8s.md#stack-compatibility) -If you’re running {{es}} in FIPS 140-2 mode, we recommend using [Bouncy Castle](https://www.bouncycastle.org/java.html) as a Java security provider when running {{es}}. ## Conduct a component inventory -It is very important to map all the components that are being used on the {{stack}}. When you upgrade your deployment, you also may need to upgrade all the other components. You should record whether any users use each component, and if so, also record the current version. While not comprehensive, here’s a list of components you should check: +When you plan to upgrade your deployment, it is very important to map all the components that are being used in the {{stack}}, and check if they are compatible with the {{es}} version you plan to upgrade to by reviewing the [Product compatibility support matrix](https://www.elastic.co/support/matrix#matrix_compatibility). + +::::{note} +If any of your ingest components does not support the {{es}} version you plan to upgrade to, you need to upgrade that component to a version that supports the desired {{es}} version before upgrading {{es}}. +:::: + +As part of the upgrade plan, you will also have to determine if you want to upgrade the ingest components to the same version as {{es}}, after the upgrade of {{es}} and {{kib}}. + +While not comprehensive, here’s a list of the components you should check: * {{es}} * {{es}} Hadoop * {{es}} plugins * {{es}} clients -* {{kib}} * {{ls}} * {{ls}} plugins * {{beats}} @@ -41,9 +93,9 @@ It is very important to map all the components that are being used on the {{stac When you do your inventory, you can [enable audit logging](/deploy-manage/security/logging-configuration/enabling-audit-logs.md) to evaluate resources accessing your deployment. ::: -## Test your development environment +## Test in a non-production environment -We highly recommend testing and upgrading in your development environment before your production environment. Therefore, it is crucial to ensure that both your development and production environments have the same settings. Consider checking the following components beforehand: +We highly recommend testing the upgrade process in a non-production environment before applying changes to your production environment. To ensure meaningful results, your test and production environments should be configured as similarly as possible. Consider validating the following areas: * Enrichment information * Plugins @@ -60,23 +112,47 @@ We highly recommend testing and upgrading in your development environment before * Alerts * Authentication -## Choose your upgrade path [choose-upgrade-path] +During your upgrade tests, pay particular attention to the following aspects: + +**Cluster stability** +: Does the new version of {{es}} form a stable healthy cluster? + +**Indexing and search performance** +: Does the new version of {{es}} perform the same (or better) than the current one on your specific workload and data? + +**Snapshots** +: Do all of your snapshot repositories work correctly and pass [repository analysis](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-snapshot-repository-analyze)? + +## Upgrade order + +When upgrading the {{stack}}, the process begins with {{es}}, followed by {{kib}}, which must always be aligned in terms of versioning. Other components can remain on earlier versions as long as they are compatible with the target {{es}} version, though we recommend upgrading them as well to benefit from the latest features and fixes. + +In general, you should upgrade the components of your {{stack}} in the following order: + +1. {{es}} +2. {{kib}} (must be kept aligned with the {{es}} version) +3. {{fleet-server}} and Elastic APM (if used) +4. Ingest tools ({{beats}}, {{agent}}, {{ls}}, etc.) and {{es}} client libraries + +If your deployment runs on {{ech}} or {{ece}}, the platform handles the upgrade and component order automatically in a single plan execution, including the {{fleet-server}} and APM as part of the Integrations Server. You only need to upgrade any external ingest tools afterward. -The procedures you follow to upgrade depend on your infrastructure and deployment method. You’ve installed Elastic components using either Elastic-managed infrastructure or self-managed infrastructure. +::::{note} +If you use a separate [monitoring cluster](/deploy-manage/monitor/stack-monitoring/elasticsearch-monitoring-self-managed.md), upgrade the monitoring cluster before the production cluster. -### Elastic-managed infrastructure +The monitoring cluster should be running the same version, or a newer one, than the clusters being monitored. It cannot monitor clusters running a newer version of the {{stack}}. If necessary, the monitoring cluster can monitor clusters running the latest release of the previous major version. +:::: -Elastic-managed infrastructure includes {{ecloud}} – the umbrella term for {{ech}} (ECH) and {{serverless-full}}. {{serverless-full}} (“Serverless”) is a fully managed cloud offering with three products: {{es-serverless}}, {{obs-serverless}}, and {{sec-serverless}}. All serverless products are built on top of the Search AI Lake. Customers on serverless receive the latest features automatically when updates are published and do not need to choose an upgrade path. +## Example of an upgrade plan -{{ech}} is Elastic’s cloud offering for managing {{stack}} deployments, built on top of {{es}}. A single click in the {{ecloud}} console can upgrade a deployment to a newer version. +Let's assume you are running all {{stack}} components in version 8.14 and your main goal is to upgrade {{es}} and {{kib}} to the latest {{version.stack}}, without requiring to upgrade the ingest components ({{beats}}, {{agent}}, and {{ls}}) except when required by the [upgrade paths](../upgrade.md#upgrade-paths). -### Self-managed infrastructure +The minimum steps your plan should include are: -Self-managed infrastructure – either on-prem or on public cloud, includes: -* {{stack}} -* {{ece}} (ECE) -* {{eck}} (ECK) +1. Upgrade {{es}} and {{kib}} to the latest 8.19 version, as a requirement for the major upgrade to {{version.stack}}. +2. Upgrade all ingest components to the latest 8.19 version, as otherwise they won't be compatible with {{es}} running {{version.stack}}. +3. Follow all [preparation steps](./prepare-to-upgrade.md) and use the upgrade assistant to ensure the cluster is ready for a major upgrade. +4. {{es}} and {{kib}} to {{version.stack}}. -For ECE and ECK, ensure the operator is running a version compatible with the {{stack}} version you’re upgrading to. If not, you need to upgrade that before [upgrading your cluster](/deploy-manage/upgrade/deployment-or-cluster.md). +## Next steps -If you’re running the {{stack}} on your own self-managed infrastructure, you must [upgrade each component individually](/deploy-manage/upgrade/deployment-or-cluster/self-managed.md). +After you’ve planned your upgrade and defined a clear upgrade path for all the components, you can proceed to the [upgrade preparations](/deploy-manage/upgrade/prepare-to-upgrade.md). \ No newline at end of file diff --git a/deploy-manage/upgrade/prepare-to-upgrade.md b/deploy-manage/upgrade/prepare-to-upgrade.md index f9e547fac3..270f57c6f2 100644 --- a/deploy-manage/upgrade/prepare-to-upgrade.md +++ b/deploy-manage/upgrade/prepare-to-upgrade.md @@ -1,4 +1,5 @@ --- +navigation_title: Preparation steps applies_to: stack: deployment: @@ -6,106 +7,157 @@ applies_to: ess: ece: self: +products: + - id: kibana + - id: cloud-enterprise + - id: cloud-hosted + - id: cloud-kubernetes + - id: elasticsearch --- # Prepare to upgrade -Before you upgrade, review and complete the necessary preparation steps, which vary by version. +This document describes the preparation steps for upgrading {{es}}, which vary depending on the type of upgrade. These steps follow the [upgrade planning](./plan-upgrade.md) phase, and should be completed before proceeding to [upgrade your deployment or cluster](./deployment-or-cluster.md). + +When upgrading an existing cluster, you perform a major, minor, or patch upgrade. A minor upgrade, for example, can take you from version 9.0.2 to 9.1.x; a major upgrade from 8.19.x to {{version.stack}}; and a patch upgrade from 9.0.1 to 9.0.4. :::{important} Upgrading from a release candidate build, such as 9.0.0-rc1, is unsupported. Use pre-releases only for testing in a temporary environment. ::: -## Prepare to upgrade from 8.x [prepare-upgrade-from-8.x] +## Common preparation steps for all upgrades + +The following steps and recommendations are common to all types of upgrades, regardless if you are upgrading from 8.x (major upgrade), or if you are already running a 9.x version. + +:::::{stepper} -To upgrade from 8.17.0 or earlier to 9.0.0, you must first upgrade to the latest 8.18 patch release. This allows you to use the [Upgrade Assistant](prepare-to-upgrade/upgrade-assistant.md) to identify and resolve issues, reindex indices created before 8.0.0, and perform a rolling upgrade. Upgrading to the latest 8.18 patch release is required even if you choose a full {{es}} cluster restart. If you're using 7.x and earlier, you may need to complete multiple upgrades or perform a full-cluster restart to reach the latest 8.18 patch release before upgrading to 9.0.0. +::::{step} Review breaking changes +Although breaking changes typically affect [major upgrades](#prepare-upgrade-from-8.x), they can also occur in minor or patch releases. Review the [breaking changes](../../release-notes/index.md) for each product you use to learn more about potential impacts on your applications. Ensure you test with the new version before upgrading production deployments. -If you are already running an 8.18.x version, it's also recommended to upgrade to the latest 8.18 patch release before upgrading to 9.x. This ensures that the latest version of the upgrade assistant is used, and any bug fixes that could have implications for the upgrade are applied. +If you are affected by a breaking change, you have to take action before upgrading. This can include updating your code, change configuration settings, or other steps. +:::: -As an alternative to upgrading the cluster, you can create a new 9.0.0 deployment and reindex from the original cluster. For more information, refer to [Reindex to upgrade](#reindex-to-upgrade). +::::{step} Verify plugin compatibility +If you use [{{es}} plugins](elasticsearch://reference/elasticsearch-plugins/index.md), ensure each plugin is compatible with the {{es}} version you're upgrading to. +:::: + +::::{step} Create a snapshot for backup +Take a [snapshot](/deploy-manage/tools/snapshot-and-restore/create-snapshots.md) of your cluster before starting the upgrade. This provides a recovery point in case the upgrade needs to be rolled back. + +:::{important} +After you start to upgrade your {{es}} cluster, you cannot downgrade any of its nodes. If you can't complete the upgrade process, you must [restore from a snapshot](/deploy-manage/tools/snapshot-and-restore/restore-snapshot.md) which was taken before starting the upgrade. +::: +:::: + +::::{step} Test in a non-production environment +Before upgrading your production deployment, test the upgrade using a non-production environment. Make sure the test environment mirrors production as closely as possible, including configuration and client interactions. Refer to [](./plan-upgrade.md#test-in-a-non-production-environment) for more details and recommendations. :::{note} -For flexible upgrade scheduling, 8.18.0 {{beats}} and {{ls}} are compatible with 9.x {{es}}. -By default, 8.x {{es}} clients are compatible with 9.x and use [REST API compatibility](elasticsearch://reference/elasticsearch/rest-apis/compatibility.md) to maintain compatibility with the 9.x {{es}} server. +The upgraded version of {{es}} may interact with its environment in different ways from the version you are currently running. It is possible that your environment behaves incorrectly in a way that does not matter to the version of {{es}} that you are currently running, but which does matter to the upgraded version. In this case, the upgraded version will not work correctly until you address the incorrect behavior in your environment. ::: -Review the following best practices to upgrade your deployments. +:::: -1. Run the [Upgrade Assistant](prepare-to-upgrade/upgrade-assistant.md), which identifies deprecated settings in your configuration and guides you through resolving issues that could prevent a successful upgrade. The Upgrade Assistant also helps resolve issues with older indices created before version 8.0.0, providing the option to reindex older indices or mark them as read-only. To prevent upgrade failures, we strongly recommend you **do not** skip this step. +::::{step} Upgrade your monitoring cluster first +If you use a separate [monitoring cluster](/deploy-manage/monitor/stack-monitoring/elasticsearch-monitoring-self-managed.md), upgrade the monitoring cluster before the production cluster. - :::{note} - Depending on your setup, reindexing can change your indices, and you may need to update alerts, transforms, or other code targeting the old index. - ::: +The monitoring cluster should be running the same version, or a newer one, than the clusters being monitored. It cannot monitor clusters running a newer version of the {{stack}}. If necessary, the monitoring cluster can monitor clusters running the latest release of the previous major version. +:::: -1. Before you change configurations or reindex, ensure you have a current [snapshot](/deploy-manage/tools/snapshot-and-restore/create-snapshots.md). +::::{step} Upgrade remote clusters first +If you use {{ccs}}, versions 9.0.0 and later can search only remote clusters running the previous minor version, the same version, or a newer minor version in the same major version. For more information, refer to [{{ccs-cap}}](../../solutions/search/cross-cluster-search.md). - :::{tip} - Tip: In 8.3.0 and later, snapshots are generally available as simple archives. Use the [archive functionality](/deploy-manage/upgrade/deployment-or-cluster/reading-indices-from-older-elasticsearch-versions.md) to search snapshots from 5.0.0 and later without needing an old {{es}} cluster. This ensures that your {{es}} data remains accessible after upgrading, without requiring a reindex process. - ::: +If you use {{ccr}}, a cluster that contains follower indices must run the same or newer (compatible) version as the remote cluster. For more information and to view the version compatibility matrix, refer to [{{ccr-cap}}](/deploy-manage/tools/cross-cluster-replication.md). - To successfully upgrade, resolve all critical issues. If you make additional changes, create a snapshot to back up your data. +To view your remote clusters in {{kib}}, go to **Stack Management > Remote Clusters**. +:::: -1. To identify if your applications use unsupported features or behave differently in 9.x, review the deprecation logs in the Upgrade Assistant. +::::{step} (Optional) Close machine learning jobs +To reduce overhead on the cluster during the upgrade, close [{{ml}} jobs](/explore-analyze/machine-learning/anomaly-detection/ml-ad-run-jobs.md) before starting the upgrade, and open them after the upgrade is complete. Although {{ml}} jobs can run during a rolling upgrade, doing so increases the cluster workload. +:::: -4. Major version upgrades can include breaking changes that require additional steps to ensure your applications function as expected. Review the [breaking changes](../../release-notes/index.md) for each product you use to learn more about potential impacts on your application. Ensure you test with the new version before upgrading existing deployments. +::::: -1. Make the recommended changes to ensure your clients continue operating as expected after the upgrade. +If you are preparing a minor or patch upgrade, you're ready to [upgrade your deployment or cluster](./deployment-or-cluster.md). If you are preparing a major upgrade, continue with the [preparations to upgrade from 8.x](#prepare-upgrade-from-8.x). - :::{note} - As a temporary solution, use the 8.x syntax to submit requests to 9.x with REST API compatibility mode. While this allows you to submit requests using the old syntax, it doesn’t guarantee the same behavior. REST API compatibility should serve as a bridge during the upgrade, not a long-term solution. For more details on how to effectively use REST API compatibility during an upgrade, refer to [REST API compatibility](elasticsearch://reference/elasticsearch/rest-apis/compatibility.md). - ::: +## Additional preparation steps to upgrade from 8.x [prepare-upgrade-from-8.x] -1. If you use {{es}} plugins, ensure each plugin is compatible with the {{es}} version you're upgrading. +Major upgrades require additional planning and preparation, as they often introduce a significant number of breaking changes and require additional steps to ensure a smooth transition. -1. Before upgrading your production deployment, test the upgrade using an isolated environment. Ensure the test and production environments use the same settings. +To assist with this process, use the [Upgrade Assistant](prepare-to-upgrade/upgrade-assistant.md), which helps detect deprecated settings, highlights upgrade blockers, and guides you through the required actions. - :::{note} - The upgraded version of {{es}} may interact with its environment in different ways from the version you are currently running. It is possible that your environment behaves incorrectly in a way that does not matter to the version of {{es}} that you are currently running, but which does matter to the upgraded version. In this case, the upgraded version will not work correctly until you address the incorrect behavior in your environment. - ::: +Follow these steps to prepare for a successful major upgrade from 8.x to 9.x: - :::{tip} - During your upgrade tests, pay particular attention to the following aspects: +:::::{stepper} - **Cluster stability** - : Does the new version of {{es}} form a stable healthy cluster? +::::{step} Upgrade to the latest 8.19 patch release - **Indexing and search performance** - : Does the new version of {{es}} perform the same (or better) than the current one on your specific workload and data? +To perform a major upgrade from 8.x to 9.x of {{es}}, you must first upgrade to 8.19.x. This allows you to use the [Upgrade Assistant](prepare-to-upgrade/upgrade-assistant.md) to identify and resolve issues, reindex indices created before 8.0.0, and prepare the cluster for the actual upgrade. Upgrading to 8.19 is required regardless of whether you perform a rolling upgrade or a full cluster restart to upgrade. - **Snapshots** - : Do all of your snapshot repositories work correctly and pass [repository analysis](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-snapshot-repository-analyze)? - ::: +:::{note} +Because 8.18.0 and 9.0.0 were released simultaneously, upgrading from 8.18.x to 9.0.x is supported, as long as the versions comply with the supported [upgrade paths](../upgrade.md#upgrade-paths). However, upgrading to 9.1.0 or later requires starting from 8.19.x. -1. Create a snapshot of your production deployment before starting the upgrade. +If you're upgrading to the current {{version.stack}} release from an earlier 8.x version, first upgrade to the latest available 8.19 release. +::: - :::{important} - After you start to upgrade your {{es}} cluster, you cannot downgrade any of its nodes. If you can't complete the upgrade process, you must [restore from a snapshot](/deploy-manage/tools/snapshot-and-restore/restore-snapshot.md) which was taken before starting the upgrade. - ::: +If you are already running an 8.19.x version, it's also recommended to upgrade to the latest 8.19 patch release before upgrading to 9.x. This ensures that the latest version of the upgrade assistant is used, and any bug fixes that could have implications for the upgrade are applied. -1. If you use a separate [monitoring cluster](/deploy-manage/monitor/stack-monitoring/elasticsearch-monitoring-self-managed.md), upgrade the monitoring cluster before the production cluster. The monitoring cluster and the clusters being monitored should be running the same version of the {{stack}}. Monitoring clusters cannot monitor production clusters running newer versions of the {{stack}}. If necessary, the monitoring cluster can monitor production clusters running the latest release of the previous major version. +If you're using 7.x and earlier, you may need to complete multiple upgrades to reach the latest 8.19 patch release before upgrading to 9.x. As an alternative method to upgrading the cluster, you can create a new 9.x deployment and reindex from the original cluster. For more information, refer to [Reindex to upgrade](#reindex-to-upgrade). - :::{note} - If you use {{ccs}}, versions 9.0.0 and later can search only remote clusters running the previous minor version, the same version, or a newer minor version in the same major version. For more information, refer to [{{ccs-cap}}](../../solutions/search/cross-cluster-search.md). +:::{note} +For flexible upgrade scheduling, 8.19.x {{agent}}, {{beats}} and {{ls}} are compatible with all 9.x versions of {{es}}. - If you use {{ccr}}, a cluster that contains follower indices must run the same or newer (compatible) version as the remote cluster. For more information and to view the version compatibility matrix, refer to [{{ccr-cap}}](/deploy-manage/tools/cross-cluster-replication.md). To view your remote clusters in {{kib}}, go to **Stack Management > Remote Clusters**. +By default, 8.x {{es}} clients are compatible with 9.x and use [REST API compatibility](elasticsearch://reference/elasticsearch/rest-apis/compatibility.md) to maintain compatibility with the 9.x {{es}} server. +::: +:::: - In addition, if you have {{ccr-init}} data streams, refer to [Upgrade uni-directional {{ccr}} clusters with followed data streams](#upgrade-ccr-data-streams) for specific instructions on reindexing. - :::: +::::{step} Run the Upgrade Assistant -1. To reduce overhead on the cluster during the upgrade, close {{ml}} jobs. Although {{ml}} jobs can run during a rolling upgrade, doing so increases the cluster workload. +The [Upgrade Assistant](prepare-to-upgrade/upgrade-assistant.md) identifies deprecated settings in your configuration and guides you through resolving issues that could prevent a successful upgrade. The Upgrade Assistant also helps resolve issues with older indices created before version 8.0.0, providing the option to reindex older indices or mark them as read-only. To prevent upgrade failures, we strongly recommend you **do not** skip this step. -1. If you have `.ml-anomalies-*`anomaly detection result indices created in {{es}} 7.x, reindex them, mark them as read-only, or delete them before you upgrade to 9.x. For more information, refer to [Migrate anomaly detection results](#anomaly-migration). +:::{note} + Depending on your setup, reindexing can change your indices, and you may need to update alerts, transforms, or other code targeting the old index. +::: -1. If you have transform destination indices created in {{es}} 7.x, reset, reindex, or delete them before you upgrade to 9.x. For more information, refer to [Migrate transform destination indices](#transform-migration). +Considerations when using the upgrade assistant: +* For a successful upgrade, **resolve all critical issues reported by the assistant**. {{es}} nodes will fail to start if incompatible indices are present. -## Reindex to upgrade [reindex-to-upgrade] +* Before you apply configuration changes or reindex, ensure you have a current [snapshot](/deploy-manage/tools/snapshot-and-restore/create-snapshots.md). -Optionally create a 9.0.0 deployment and reindex from remote: +* Indices created in 7.x or earlier must be reindexed, deleted, or [archived](/deploy-manage/upgrade/deployment-or-cluster/reading-indices-from-older-elasticsearch-versions.md) (marked as read-only) before upgrading to 9.x. -1. Provision an additional deployment running 9.0.0. -2. To reindex your data into the new {{es}} cluster, use the [reindex documents API](https://www.elastic.co/docs/api/doc/elasticsearch/v8/operation/operation-reindex) and temporarily send new index requests to both clusters. -3. Verify the new cluster performs as expected, fix any problems, and then permanently swap in the new cluster. -4. Delete the old deployment. On {{ecloud}}, you are billed only for the time the new deployment runs in parallel with your old deployment. Usage is billed on an hourly basis. + :::{tip} + In Elasticsearch 9.x, you can also use the [archive functionality](/deploy-manage/upgrade/deployment-or-cluster/reading-indices-from-older-elasticsearch-versions.md) to access snapshots of 7.x or earlier indices, without needing to reindex or run an older cluster. This provides a convenient option to retain historical data in case you choose to delete those indices and keep them only in existing snapshots. + ::: + +* Review the deprecation logs from the Upgrade Assistant to determine if your applications are using features that are not supported or behave differently in 9.x. See the [breaking changes](elasticsearch://release-notes/breaking-changes.md) for more information about changes in 9.x that could affect your application. + + :::{note} + Make sure you check the breaking changes for each 9.x release up to your target release. + ::: + +* Make the recommended changes to ensure your clients continue operating as expected after the upgrade. + + :::{note} + As a temporary solution, use the 8.x syntax to submit requests to 9.x with REST API compatibility mode. While this allows you to submit requests using the old syntax, it doesn’t guarantee the same behavior. REST API compatibility should serve as a bridge during the upgrade, not a long-term solution. For more details on how to effectively use REST API compatibility during an upgrade, refer to [REST API compatibility](elasticsearch://reference/elasticsearch/rest-apis/compatibility.md). + ::: +:::: + +::::{step} Manage CCR follower data streams +If you have {{ccr-cap}} (CCR) data streams, and your indices require reindexing, refer to [Upgrade uni-directional {{ccr}} clusters with followed data streams](#upgrade-ccr-data-streams) for specific instructions. +:::: + +::::{step} Manage old machine learning indices +If you have `.ml-anomalies-*` anomaly detection result indices created in {{es}} 7.x, reindex them, mark them as read-only, or delete them before you upgrade to 9.x. For more information, refer to [Migrate anomaly detection results](#anomaly-migration). +:::: + +::::{step} Manage old transform indices +If you have transform destination indices created in {{es}} 7.x, reset, reindex, or delete them before you upgrade to 9.x. For more information, refer to [Migrate transform destination indices](#transform-migration). +:::: + +::::: + +After completing all the preparation steps, you're ready to [upgrade your deployment or cluster](./deployment-or-cluster.md). ## Upgrade uni-directional {{ccr}} clusters with followed data streams [upgrade-ccr-data-streams] @@ -134,9 +186,6 @@ If you need to write directly to non-write backing indices of data streams in a 3. Once the reindexing is complete and the leader cluster is upgraded, re-add the newly reindexed backing indexes as follower indices on the {{ccr-init}} follower. - - - ## Migrate anomaly detection results [anomaly-migration] Reindex, mark as read-only, or delete the `.ml-anomalies-*` {{anomaly-detect}} result indices created in {{es}} 7.x. @@ -759,3 +808,12 @@ If the destination index is no longer needed, it can be deleted with the transfo DELETE _transform/my-transform?delete_dest_index ``` ::: + +## Reindex to upgrade [reindex-to-upgrade] + +If you are running a pre-8.x version, you might need to perform multiple upgrades before being able to upgrade to 9.x. As an alternative method to upgrading the cluster, you can create a new deployment in the target version and reindex from remote: + +1. Provision an additional deployment running the desired version, such as {{version.stack}}. +2. To reindex your data into the new {{es}} cluster, use the [reindex documents API](https://www.elastic.co/docs/api/doc/elasticsearch/v8/operation/operation-reindex) and temporarily send new indexing requests to both clusters. +3. Verify the new cluster performs as expected, fix any problems, and then permanently swap in the new cluster. +4. Delete the old deployment. On {{ecloud}}, you are billed for the time the new deployment runs in parallel with your old deployment. Usage is billed on an hourly basis. diff --git a/deploy-manage/upgrade/prepare-to-upgrade/upgrade-assistant.md b/deploy-manage/upgrade/prepare-to-upgrade/upgrade-assistant.md index ba882da472..33bedc4b00 100644 --- a/deploy-manage/upgrade/prepare-to-upgrade/upgrade-assistant.md +++ b/deploy-manage/upgrade/prepare-to-upgrade/upgrade-assistant.md @@ -10,11 +10,20 @@ applies_to: self: products: - id: kibana + - id: cloud-enterprise + - id: cloud-hosted + - id: cloud-kubernetes + - id: elasticsearch --- # Upgrade Assistant [upgrade-assistant] -The Upgrade Assistant helps you [prepare to upgrade](/deploy-manage/upgrade/prepare-to-upgrade.md) to the next version of the {{stack}}. To access the assistant, go to **{{stack-manage-app}} → Upgrade Assistant**. +The Upgrade Assistant helps you [prepare to upgrade](/deploy-manage/upgrade/prepare-to-upgrade.md) to the next major version of the {{stack}}. To access the assistant, go to **{{stack-manage-app}} → Upgrade Assistant**. + +::::{tip} +Upgrade assistant should be run from the latest minor release before a major upgrade. When upgrading to 9.x, ensure you run 8.19.latest, and run the assistant there. +Running the latest patched version of 8.19 will apply latest version of the upgrade assistant logic. +:::: The assistant identifies deprecated settings in your configuration, and if any of those settings are enabled, it guides you through resolving issues that could prevent a successful upgrade. The Upgrade Assistant also helps resolve issues with older indices created before version 8.0.0, providing options to reindex older indices or mark them as read-only.