From 5afe568343bc9eb7d5dca0cc0a35c54237e44151 Mon Sep 17 00:00:00 2001 From: lcawl Date: Wed, 27 Aug 2025 16:31:35 -0700 Subject: [PATCH 01/11] Clarify usage of Elasticsearch Serverless configuration profiles --- solutions/search/get-started/semantic-search.md | 6 +----- solutions/search/serverless-elasticsearch-get-started.md | 4 ++-- solutions/search/vector/bring-own-vectors.md | 2 +- 3 files changed, 4 insertions(+), 8 deletions(-) diff --git a/solutions/search/get-started/semantic-search.md b/solutions/search/get-started/semantic-search.md index 2aa6fd33c5..487470d772 100644 --- a/solutions/search/get-started/semantic-search.md +++ b/solutions/search/get-started/semantic-search.md @@ -22,15 +22,11 @@ By playing with a simple use case, you'll take the first steps toward understand ## Prerequisites -- If you're using [{{es-serverless}}](/solutions/search/serverless-elasticsearch-get-started.md), create a project that is optimized for vectors. To add the sample data, you must have a `developer` or `admin` predefined role or an equivalent custom role. +- If you're using [{{es-serverless}}](/solutions/search/serverless-elasticsearch-get-started.md), create a project with a general purpose configuration. To add the sample data, you must have a `developer` or `admin` predefined role or an equivalent custom role. - If you're using [{{ech}}](/deploy-manage/deploy/elastic-cloud/cloud-hosted.md) or [running {{es}} locally](/solutions/search/run-elasticsearch-locally.md), start {{es}} and {{kib}}. To add the sample data, log in with a user that has the `superuser` built-in role. To learn about role-based access control, check out [](/deploy-manage/users-roles/cluster-or-deployment-auth/user-roles.md). - - ## Create a vector database When you create vectors (or _vectorize_ your data), you convert complex and nuanced documents into multidimensional numerical representations. diff --git a/solutions/search/serverless-elasticsearch-get-started.md b/solutions/search/serverless-elasticsearch-get-started.md index 8f68135775..4b80feb853 100644 --- a/solutions/search/serverless-elasticsearch-get-started.md +++ b/solutions/search/serverless-elasticsearch-get-started.md @@ -48,8 +48,8 @@ Use your {{ecloud}} account to create a fully-managed {{es}} project: 3. Choose the {{es}} project type. 4. Select a **configuration** for your project, based on your use case. - * **General purpose**: For general search use cases across various data types. - * **Optimized for Vectors**: For search use cases using vectors and near real-time retrieval. + * **General purpose**: This option is suitable for all standard search and vector workloads. It is performant and less expensive than the other option for most use cases. + * **Optimized for vectors**: This option provides great performance on vector search for uncompressed vectors with high dimensionality. 5. Provide a name for the project and optionally edit the project settings, such as the cloud platform [region](../../deploy-manage/deploy/elastic-cloud/regions.md). Select **Create project** to continue. 6. Once the project is ready, select **Continue**. diff --git a/solutions/search/vector/bring-own-vectors.md b/solutions/search/vector/bring-own-vectors.md index e648e08638..d82aaab97a 100644 --- a/solutions/search/vector/bring-own-vectors.md +++ b/solutions/search/vector/bring-own-vectors.md @@ -20,7 +20,7 @@ You'll also learn the syntax for searching these documents using a [k-nearest ne ## Prerequisites -- If you're using {{es-serverless}}, create a project that is optimized for vectors. To add the sample data, you must have a `developer` or `admin` predefined role or an equivalent custom role. +- If you're using {{es-serverless}}, create a project with a general purpose configuration. The "optimized for vectors" configuration is not required unless you choose to use uncompressed vectors. To add the sample data, you must have a `developer` or `admin` predefined role or an equivalent custom role. - If you're using {{ech}} or a self-managed cluster, start {{es}} and {{kib}}. The simplest method to complete the steps in this guide is to log in with a user that has the `superuser` built-in role. To learn about role-based access control, check out [](/deploy-manage/users-roles/cluster-or-deployment-auth/user-roles.md). From cb1503c649bc9aea175d8c0e0cd843c6ac3e0f88 Mon Sep 17 00:00:00 2001 From: lcawl Date: Wed, 27 Aug 2025 17:21:54 -0700 Subject: [PATCH 02/11] Update billing page --- .../elasticsearch-billing-dimensions.md | 45 ++++++++++--------- solutions/search/vector/bring-own-vectors.md | 2 +- 2 files changed, 24 insertions(+), 23 deletions(-) diff --git a/deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md b/deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md index 73c224d16d..4b8fb55535 100644 --- a/deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md +++ b/deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md @@ -3,14 +3,18 @@ navigation_title: Elasticsearch mapped_pages: - https://www.elastic.co/guide/en/serverless/current/elasticsearch-billing.html applies_to: - serverless: all + serverless: + elasticsearch: ga products: - id: cloud-serverless +description: The cost of {{es-serverless}} projects is affected by the type and number of virtual compute units used. --- # {{es}} billing dimensions [elasticsearch-billing] -{{es}} is priced based on consumption of the underlying infrastructure that supports your use case, with the performance characteristics you need. Measurements are in Virtual Compute Units (VCUs). Each VCU represents a fraction of RAM, CPU, and local disk for caching. +{{es-serverless}} projects are priced based on consumption of the underlying infrastructure that supports your use case with the performance characteristics you need. +Measurements are in virtual compute units (VCUs). +Each VCU represents a fraction of RAM, CPU, and local disk for caching. The number of VCUs you need is determined by: @@ -20,40 +24,37 @@ The number of VCUs you need is determined by: * Search Power setting * Machine learning usage -For detailed {{es-serverless}} project rates, see the [{{es-serverless}} pricing page](https://www.elastic.co/pricing/serverless-search). +The number of VCUs is also affected by your choice of configuration: +* The general purpose configuration is suitable for all standard search and vector workloads. It is performant and less expensive than the vector optimized configuration for most use cases. +* The optimized for vectors configuration provides great performance on vector search for uncompressed vectors with high dimensionality. However, it uses approximately four times more VCUs than the general purpose configuration. -## VCU types: Search, Indexing, and ML [elasticsearch-billing-information-about-the-vcu-types-search-ingest-and-ml] +For detailed {{es-serverless}} project rates, refer to the [{{es-serverless}} pricing page](https://www.elastic.co/pricing/serverless-search). -{{es}} uses three VCU types: +## VCU types: search, indexing, and ML [elasticsearch-billing-information-about-the-vcu-types-search-ingest-and-ml] -* **Indexing:** The VCUs used to index incoming documents. Indexing VCUs account for compute resources consumed for ingestion. This is based on ingestion rate, and amount of data ingested at any given time. Transforms and ingest pipelines also contribute to ingest VCU consumption. -* **Search:** The VCUs used to return search results, with the latency and queries per second (QPS) you require. Search VCUs are calculated as a factor of the compute resources needed to run search queries, search throughput and latency. Search VCUs are not charged per search request, but instead are a factor of the compute resources that scale up and down based on amount of searchable data, search load (QPS) and performance (latency and availability). -* **Machine learning:** The VCUs used to perform inference, NLP tasks, and other ML activities. ML VCUs are a factor of the models deployed, and number of ML operations such as inference for search and ingest. ML VCUs are typically consumed for generating embeddings during ingestion, and during semantic search or reranking. -* **Tokens:** The Elastic Managed LLM is charged per 1Mn Input and Output tokens. The LLM powers all AI Search features such as Playground and AI Assistant for Search, and is enabled by default. +{{es-serverless}} uses three VCU types: +* **Indexing:** The VCUs used to index incoming documents. Indexing VCUs account for compute resources consumed for ingestion. This is based on ingestion rate and amount of data ingested at any given time. Transforms and ingest pipelines also contribute to ingest VCU consumption. +* **Search:** The VCUs used to return search results with the latency and queries per second (QPS) you require. Search VCUs are calculated as a factor of the compute resources needed to run search queries, search throughput, and latency. Search VCUs are not charged per search request. Instead, they are a factor of the compute resources that scale up and down based on amount of searchable data, search load (QPS), and performance (latency and availability). +* **Machine learning:** The VCUs used to perform inference, NLP tasks, and other ML activities. ML VCUs are a factor of the models deployed and number of ML operations such as inference for search and ingest. ML VCUs are typically consumed for generating embeddings during ingestion and during semantic search or reranking. +* **Tokens:** The Elastic Managed LLM is charged per 1 million input and output tokens. The LLM powers all AI Search features such as Playground and AI Assistant for Search and is enabled by default. ## Data storage and billing [elasticsearch-billing-information-about-the-search-ai-lake-dimension-gb] -{{es-serverless}} projects store data in the [Search AI Lake](../../deploy/elastic-cloud/project-settings.md#elasticsearch-manage-project-search-ai-lake-settings). You are charged per GB of stored data at rest. Note that if you perform operations at ingest such as vectorization or enrichment, the size of your stored data will differ from the size of the original source data. - +{{es-serverless}} projects store data in the [Search AI Lake](/deploy/elastic-cloud/project-settings.md#elasticsearch-manage-project-search-ai-lake-settings). You are charged per GB of stored data at rest. Note that if you perform operations at ingest such as vectorization or enrichment, the size of your stored data will differ from the size of the original source data. ## Managing {{es}} costs [elasticsearch-billing-managing-elasticsearch-costs] You can control costs using the following strategies: -* **Search Power setting:** [Search Power](../../deploy/elastic-cloud/project-settings.md#elasticsearch-manage-project-search-power-settings) controls the speed of searches against your data. With Search Power, you can improve search performance by adding more resources for querying, or you can reduce provisioned resources to cut costs. -* **Search boost window**: By limiting the number of days of [time series data](../../../solutions/search/ingest-for-search.md#elasticsearch-ingest-time-series-data) that are available for caching, you can reduce the number of search VCUs required. +* **Search Power setting:** [Search Power](/deploy/elastic-cloud/project-settings.md#elasticsearch-manage-project-search-power-settings) controls the speed of searches against your data. With Search Power, you can improve search performance by adding more resources for querying or you can reduce provisioned resources to cut costs. +* **Search boost window**: By limiting the number of days of [time series data](/solutions/search/ingest-for-search.md#elasticsearch-ingest-time-series-data) that are available for caching, you can reduce the number of search VCUs required. * **Machine learning trained model autoscaling:** [Trained model autoscaling](/deploy-manage/autoscaling/trained-model-autoscaling.md) is always enabled and cannot be disabled, ensuring efficient resource usage, reduced costs, and optimal performance without manual configuration. Trained model deployments automatically scale down to zero allocations after 24 hours without any inference requests. When they scale up again, they remain active for 5 minutes before they can scale down. During these cooldown periods, you will continue to be billed for the active resources. - -* **Indexing Strategies:** Consider your indexing strategies and how they might impact overall VCU usage and costs: +* **Indexing strategies:** Consider your indexing strategies and how they might impact overall VCU usage and costs: - * To ensure optimal performance and cost-effectiveness for your project, it’s important to consider how you structure your data. - * Consolidate small indices for better efficiency. We recommend avoiding a design where your project contains hundreds of very small indices, specifically those under 1GB each. - * Why is this important? - * Every index in Elasticsearch has a certain amount of resource overhead. This is because Elasticsearch needs to maintain metadata for each index to keep it running smoothly. When you have a very large number of small indices, the combined overhead from all of them can consume more CPU resources than if the same data were stored in fewer, larger indices. This can lead to higher resource consumption and hence higher costs and potentially impact the overall performance of your project. - - * Recommended Approach - * If your use case naturally generates many small, separate streams of data, we advise implementing a process to consolidate them into fewer, larger indices. This practice leads to more efficient resource utilization. By grouping your data into larger indices, you can ensure a more performant and cost-efficient experience with Elasticsearch Serverless. + * To ensure optimal performance and cost-effectiveness for your project, it's important to consider how you structure your data. Consolidate small indices for better efficiency. In general, avoid a design where your project contains hundreds of very small indices, specifically those under 1GB each. + * Why is this important? Every index in {{es}} has a certain amount of resource overhead. This is because {{es}} needs to maintain metadata for each index to keep it running smoothly. When you have a very large number of small indices, the combined overhead from all of them can consume more CPU resources than if the same data were stored in fewer, larger indices. This can lead to higher resource consumption and hence higher costs and potentially impact the overall performance of your project. + * If your use case naturally generates many small, separate streams of data, the recommended approach is to implement a process to consolidate them into fewer, larger indices. This practice leads to more efficient resource utilization. By grouping your data into larger indices, you can ensure a more performant and cost-efficient experience with {{es-serverless}}. diff --git a/solutions/search/vector/bring-own-vectors.md b/solutions/search/vector/bring-own-vectors.md index d82aaab97a..cb5150e516 100644 --- a/solutions/search/vector/bring-own-vectors.md +++ b/solutions/search/vector/bring-own-vectors.md @@ -20,7 +20,7 @@ You'll also learn the syntax for searching these documents using a [k-nearest ne ## Prerequisites -- If you're using {{es-serverless}}, create a project with a general purpose configuration. The "optimized for vectors" configuration is not required unless you choose to use uncompressed vectors. To add the sample data, you must have a `developer` or `admin` predefined role or an equivalent custom role. +- If you're using {{es-serverless}}, create a project with the general purpose configuration. The optimized for vectors configuration is not required unless you choose to use uncompressed vectors with high dimensionality. To add the sample data, you must have a `developer` or `admin` predefined role or an equivalent custom role. - If you're using {{ech}} or a self-managed cluster, start {{es}} and {{kib}}. The simplest method to complete the steps in this guide is to log in with a user that has the `superuser` built-in role. To learn about role-based access control, check out [](/deploy-manage/users-roles/cluster-or-deployment-auth/user-roles.md). From bd76e72640fde38d6488f5df7ac3835ce4ed71b1 Mon Sep 17 00:00:00 2001 From: lcawl Date: Wed, 27 Aug 2025 17:22:30 -0700 Subject: [PATCH 03/11] Revert changes to serverless-elasticsearch-get-started.md --- solutions/search/serverless-elasticsearch-get-started.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/solutions/search/serverless-elasticsearch-get-started.md b/solutions/search/serverless-elasticsearch-get-started.md index 4b80feb853..8f68135775 100644 --- a/solutions/search/serverless-elasticsearch-get-started.md +++ b/solutions/search/serverless-elasticsearch-get-started.md @@ -48,8 +48,8 @@ Use your {{ecloud}} account to create a fully-managed {{es}} project: 3. Choose the {{es}} project type. 4. Select a **configuration** for your project, based on your use case. - * **General purpose**: This option is suitable for all standard search and vector workloads. It is performant and less expensive than the other option for most use cases. - * **Optimized for vectors**: This option provides great performance on vector search for uncompressed vectors with high dimensionality. + * **General purpose**: For general search use cases across various data types. + * **Optimized for Vectors**: For search use cases using vectors and near real-time retrieval. 5. Provide a name for the project and optionally edit the project settings, such as the cloud platform [region](../../deploy-manage/deploy/elastic-cloud/regions.md). Select **Create project** to continue. 6. Once the project is ready, select **Continue**. From 99811a76bd36412fde85d665de7b46f37859d8f0 Mon Sep 17 00:00:00 2001 From: lcawl Date: Wed, 27 Aug 2025 17:40:52 -0700 Subject: [PATCH 04/11] Fix link paths --- .../billing/elasticsearch-billing-dimensions.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md b/deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md index 4b8fb55535..9ad41319af 100644 --- a/deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md +++ b/deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md @@ -42,13 +42,13 @@ For detailed {{es-serverless}} project rates, refer to the [{{es-serverless}} pr ## Data storage and billing [elasticsearch-billing-information-about-the-search-ai-lake-dimension-gb] -{{es-serverless}} projects store data in the [Search AI Lake](/deploy/elastic-cloud/project-settings.md#elasticsearch-manage-project-search-ai-lake-settings). You are charged per GB of stored data at rest. Note that if you perform operations at ingest such as vectorization or enrichment, the size of your stored data will differ from the size of the original source data. +{{es-serverless}} projects store data in the [Search AI Lake](/deploy-manage/deploy/elastic-cloud/project-settings.md#elasticsearch-manage-project-search-ai-lake-settings). You are charged per GB of stored data at rest. Note that if you perform operations at ingest such as vectorization or enrichment, the size of your stored data will differ from the size of the original source data. ## Managing {{es}} costs [elasticsearch-billing-managing-elasticsearch-costs] You can control costs using the following strategies: -* **Search Power setting:** [Search Power](/deploy/elastic-cloud/project-settings.md#elasticsearch-manage-project-search-power-settings) controls the speed of searches against your data. With Search Power, you can improve search performance by adding more resources for querying or you can reduce provisioned resources to cut costs. +* **Search Power setting:** [Search Power](/deploy-manage/deploy/elastic-cloud/project-settings.md#elasticsearch-manage-project-search-power-settings) controls the speed of searches against your data. With Search Power, you can improve search performance by adding more resources for querying or you can reduce provisioned resources to cut costs. * **Search boost window**: By limiting the number of days of [time series data](/solutions/search/ingest-for-search.md#elasticsearch-ingest-time-series-data) that are available for caching, you can reduce the number of search VCUs required. * **Machine learning trained model autoscaling:** [Trained model autoscaling](/deploy-manage/autoscaling/trained-model-autoscaling.md) is always enabled and cannot be disabled, ensuring efficient resource usage, reduced costs, and optimal performance without manual configuration. From a811e2f7d848a3c5925f15b90741f1f3070804ea Mon Sep 17 00:00:00 2001 From: lcawl Date: Wed, 27 Aug 2025 18:43:18 -0700 Subject: [PATCH 05/11] Move profile considerations --- .../billing/elasticsearch-billing-dimensions.md | 15 ++++++--------- 1 file changed, 6 insertions(+), 9 deletions(-) diff --git a/deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md b/deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md index 9ad41319af..abd0717243 100644 --- a/deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md +++ b/deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md @@ -24,11 +24,6 @@ The number of VCUs you need is determined by: * Search Power setting * Machine learning usage -The number of VCUs is also affected by your choice of configuration: - -* The general purpose configuration is suitable for all standard search and vector workloads. It is performant and less expensive than the vector optimized configuration for most use cases. -* The optimized for vectors configuration provides great performance on vector search for uncompressed vectors with high dimensionality. However, it uses approximately four times more VCUs than the general purpose configuration. - For detailed {{es-serverless}} project rates, refer to the [{{es-serverless}} pricing page](https://www.elastic.co/pricing/serverless-search). ## VCU types: search, indexing, and ML [elasticsearch-billing-information-about-the-vcu-types-search-ingest-and-ml] @@ -48,13 +43,15 @@ For detailed {{es-serverless}} project rates, refer to the [{{es-serverless}} pr You can control costs using the following strategies: -* **Search Power setting:** [Search Power](/deploy-manage/deploy/elastic-cloud/project-settings.md#elasticsearch-manage-project-search-power-settings) controls the speed of searches against your data. With Search Power, you can improve search performance by adding more resources for querying or you can reduce provisioned resources to cut costs. +* **Search Power setting**: [Search Power](/deploy-manage/deploy/elastic-cloud/project-settings.md#elasticsearch-manage-project-search-power-settings) controls the speed of searches against your data. With Search Power, you can improve search performance by adding more resources for querying or you can reduce provisioned resources to cut costs. * **Search boost window**: By limiting the number of days of [time series data](/solutions/search/ingest-for-search.md#elasticsearch-ingest-time-series-data) that are available for caching, you can reduce the number of search VCUs required. -* **Machine learning trained model autoscaling:** [Trained model autoscaling](/deploy-manage/autoscaling/trained-model-autoscaling.md) is always enabled and cannot be disabled, ensuring efficient resource usage, reduced costs, and optimal performance without manual configuration. +* **Machine learning trained model autoscaling**: [Trained model autoscaling](/deploy-manage/autoscaling/trained-model-autoscaling.md) is always enabled and cannot be disabled, ensuring efficient resource usage, reduced costs, and optimal performance without manual configuration. Trained model deployments automatically scale down to zero allocations after 24 hours without any inference requests. When they scale up again, they remain active for 5 minutes before they can scale down. During these cooldown periods, you will continue to be billed for the active resources. -* **Indexing strategies:** Consider your indexing strategies and how they might impact overall VCU usage and costs: - +* **Indexing strategies** Consider your indexing strategies and how they might impact overall VCU usage and costs: * To ensure optimal performance and cost-effectiveness for your project, it's important to consider how you structure your data. Consolidate small indices for better efficiency. In general, avoid a design where your project contains hundreds of very small indices, specifically those under 1GB each. * Why is this important? Every index in {{es}} has a certain amount of resource overhead. This is because {{es}} needs to maintain metadata for each index to keep it running smoothly. When you have a very large number of small indices, the combined overhead from all of them can consume more CPU resources than if the same data were stored in fewer, larger indices. This can lead to higher resource consumption and hence higher costs and potentially impact the overall performance of your project. * If your use case naturally generates many small, separate streams of data, the recommended approach is to implement a process to consolidate them into fewer, larger indices. This practice leads to more efficient resource utilization. By grouping your data into larger indices, you can ensure a more performant and cost-efficient experience with {{es-serverless}}. +* **Configuration profiles**: + * The general purpose profile offers a great performance for the price especially for most search use cases. It is the right profile for full-text search, semantic search using ELSER or sparse vector embeddings, sparse vectors, and dense vectors that use compression such as BBQ. It is recommended to use the general purpose profile for most search use cases. + * The vector optimized profile is recommended only for uncompressed dense vectors when you want better performance. Though the per VCU cost is the same for general purpose and vector optimized profiles, the latter provides a larger amount of RAM for searchable data. This leads to higher VCU consumption and is more expensive while providing significantly better performance for uncompressed vector data. \ No newline at end of file From 14ac990be5f2c29c00011051fac0506420c4bb5a Mon Sep 17 00:00:00 2001 From: Lisa Cawley Date: Thu, 28 Aug 2025 10:08:22 -0700 Subject: [PATCH 06/11] Update deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md Co-authored-by: shainaraskas <58563081+shainaraskas@users.noreply.github.com> --- .../billing/elasticsearch-billing-dimensions.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md b/deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md index abd0717243..c4a44b44b0 100644 --- a/deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md +++ b/deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md @@ -7,7 +7,7 @@ applies_to: elasticsearch: ga products: - id: cloud-serverless -description: The cost of {{es-serverless}} projects is affected by the type and number of virtual compute units used. +description: Learn about how costs for Elasticsearch Serverless projects are calculated, and strategies you can use to lower your costs. --- # {{es}} billing dimensions [elasticsearch-billing] From 12f7da2cbea325497200ae8e3aa66607ce96e591 Mon Sep 17 00:00:00 2001 From: Lisa Cawley Date: Thu, 28 Aug 2025 10:08:38 -0700 Subject: [PATCH 07/11] Update deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md Co-authored-by: shainaraskas <58563081+shainaraskas@users.noreply.github.com> --- .../billing/elasticsearch-billing-dimensions.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md b/deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md index c4a44b44b0..3fb88c56b7 100644 --- a/deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md +++ b/deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md @@ -28,7 +28,7 @@ For detailed {{es-serverless}} project rates, refer to the [{{es-serverless}} pr ## VCU types: search, indexing, and ML [elasticsearch-billing-information-about-the-vcu-types-search-ingest-and-ml] -{{es-serverless}} uses three VCU types: +{{es-serverless}} uses the following VCU types: * **Indexing:** The VCUs used to index incoming documents. Indexing VCUs account for compute resources consumed for ingestion. This is based on ingestion rate and amount of data ingested at any given time. Transforms and ingest pipelines also contribute to ingest VCU consumption. * **Search:** The VCUs used to return search results with the latency and queries per second (QPS) you require. Search VCUs are calculated as a factor of the compute resources needed to run search queries, search throughput, and latency. Search VCUs are not charged per search request. Instead, they are a factor of the compute resources that scale up and down based on amount of searchable data, search load (QPS), and performance (latency and availability). From 2e560c07f1e2291f668f44e9bf7ff79d62f724d2 Mon Sep 17 00:00:00 2001 From: lcawl Date: Thu, 28 Aug 2025 17:10:47 -0700 Subject: [PATCH 08/11] Remove nested bullets from indexing tips --- .../billing/elasticsearch-billing-dimensions.md | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md b/deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md index 3fb88c56b7..0e53003ea6 100644 --- a/deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md +++ b/deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md @@ -48,10 +48,17 @@ You can control costs using the following strategies: * **Machine learning trained model autoscaling**: [Trained model autoscaling](/deploy-manage/autoscaling/trained-model-autoscaling.md) is always enabled and cannot be disabled, ensuring efficient resource usage, reduced costs, and optimal performance without manual configuration. Trained model deployments automatically scale down to zero allocations after 24 hours without any inference requests. When they scale up again, they remain active for 5 minutes before they can scale down. During these cooldown periods, you will continue to be billed for the active resources. -* **Indexing strategies** Consider your indexing strategies and how they might impact overall VCU usage and costs: - * To ensure optimal performance and cost-effectiveness for your project, it's important to consider how you structure your data. Consolidate small indices for better efficiency. In general, avoid a design where your project contains hundreds of very small indices, specifically those under 1GB each. - * Why is this important? Every index in {{es}} has a certain amount of resource overhead. This is because {{es}} needs to maintain metadata for each index to keep it running smoothly. When you have a very large number of small indices, the combined overhead from all of them can consume more CPU resources than if the same data were stored in fewer, larger indices. This can lead to higher resource consumption and hence higher costs and potentially impact the overall performance of your project. - * If your use case naturally generates many small, separate streams of data, the recommended approach is to implement a process to consolidate them into fewer, larger indices. This practice leads to more efficient resource utilization. By grouping your data into larger indices, you can ensure a more performant and cost-efficient experience with {{es-serverless}}. +* **Indexing strategies** Consider your indexing strategies and how they might impact overall VCU usage and costs. + To ensure optimal performance and cost-effectiveness for your project, it's important to consider how you structure your data. + + Consolidate small indices for better efficiency. + In general, avoid a design where your project contains hundreds of very small indices, specifically those under 1GB each. + Why is this important? Every index in {{es}} has a certain amount of resource overhead. + This is because {{es}} needs to maintain metadata for each index to keep it running smoothly. + When you have a very large number of small indices, the combined overhead from all of them can consume more CPU resources than if the same data were stored in fewer, larger indices. + This can lead to higher resource consumption and hence higher costs and potentially impact the overall performance of your project. + + If your use case naturally generates many small, separate streams of data, the recommended approach is to implement a process to consolidate them into fewer, larger indices. This practice leads to more efficient resource utilization. By grouping your data into larger indices, you can ensure a more performant and cost-efficient experience with {{es-serverless}}. * **Configuration profiles**: * The general purpose profile offers a great performance for the price especially for most search use cases. It is the right profile for full-text search, semantic search using ELSER or sparse vector embeddings, sparse vectors, and dense vectors that use compression such as BBQ. It is recommended to use the general purpose profile for most search use cases. * The vector optimized profile is recommended only for uncompressed dense vectors when you want better performance. Though the per VCU cost is the same for general purpose and vector optimized profiles, the latter provides a larger amount of RAM for searchable data. This leads to higher VCU consumption and is more expensive while providing significantly better performance for uncompressed vector data. \ No newline at end of file From 895c4d90c0fca1eae202dc255f6d0bdb7b1323d4 Mon Sep 17 00:00:00 2001 From: lcawl Date: Thu, 28 Aug 2025 17:30:43 -0700 Subject: [PATCH 09/11] Improve profile tips --- .../billing/elasticsearch-billing-dimensions.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md b/deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md index 0e53003ea6..4ef602f743 100644 --- a/deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md +++ b/deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md @@ -59,6 +59,8 @@ You can control costs using the following strategies: This can lead to higher resource consumption and hence higher costs and potentially impact the overall performance of your project. If your use case naturally generates many small, separate streams of data, the recommended approach is to implement a process to consolidate them into fewer, larger indices. This practice leads to more efficient resource utilization. By grouping your data into larger indices, you can ensure a more performant and cost-efficient experience with {{es-serverless}}. -* **Configuration profiles**: - * The general purpose profile offers a great performance for the price especially for most search use cases. It is the right profile for full-text search, semantic search using ELSER or sparse vector embeddings, sparse vectors, and dense vectors that use compression such as BBQ. It is recommended to use the general purpose profile for most search use cases. - * The vector optimized profile is recommended only for uncompressed dense vectors when you want better performance. Though the per VCU cost is the same for general purpose and vector optimized profiles, the latter provides a larger amount of RAM for searchable data. This leads to higher VCU consumption and is more expensive while providing significantly better performance for uncompressed vector data. \ No newline at end of file +* **Configuration profiles**: When you create a project, the configuration profile also affects the VCU usage and costs. + + The **General purpose** profile is suitable for most search use cases. For example, it is the right profile for full-text search, sparse vectors, and dense vectors that use compression such as BBQ. + + The **Optimized for vectors** profile is recommended only for uncompressed dense vectors with high dimensionality. Though the per VCU cost is the same for general purpose and vector optimized profiles, the latter provides a larger amount of RAM for searchable data. This leads to higher VCU consumption in order to improve the performance for uncompressed vector data. From f00400cfe652ca75f66a160b73a7cce114de43d0 Mon Sep 17 00:00:00 2001 From: lcawl Date: Fri, 29 Aug 2025 08:16:01 -0700 Subject: [PATCH 10/11] Replace indexing question --- .../billing/elasticsearch-billing-dimensions.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md b/deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md index 4ef602f743..4649178899 100644 --- a/deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md +++ b/deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md @@ -53,10 +53,10 @@ You can control costs using the following strategies: Consolidate small indices for better efficiency. In general, avoid a design where your project contains hundreds of very small indices, specifically those under 1GB each. - Why is this important? Every index in {{es}} has a certain amount of resource overhead. - This is because {{es}} needs to maintain metadata for each index to keep it running smoothly. + Avoiding small indices is important because every index in {{es}} has a certain amount of resource overhead. + {{es}} needs to maintain metadata for each index to keep it running smoothly. When you have a very large number of small indices, the combined overhead from all of them can consume more CPU resources than if the same data were stored in fewer, larger indices. - This can lead to higher resource consumption and hence higher costs and potentially impact the overall performance of your project. + Higher resource consumption can lead to higher costs and potentially impact the overall performance of your project. If your use case naturally generates many small, separate streams of data, the recommended approach is to implement a process to consolidate them into fewer, larger indices. This practice leads to more efficient resource utilization. By grouping your data into larger indices, you can ensure a more performant and cost-efficient experience with {{es-serverless}}. * **Configuration profiles**: When you create a project, the configuration profile also affects the VCU usage and costs. From e6ab8bcb99a06088452f4e83142cfa15c0f5f71c Mon Sep 17 00:00:00 2001 From: lcawl Date: Fri, 29 Aug 2025 17:22:32 -0700 Subject: [PATCH 11/11] Remove profiles from billing --- .../billing/elasticsearch-billing-dimensions.md | 5 ----- solutions/search/vector/bring-own-vectors.md | 2 +- 2 files changed, 1 insertion(+), 6 deletions(-) diff --git a/deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md b/deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md index 4649178899..03279c5261 100644 --- a/deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md +++ b/deploy-manage/cloud-organization/billing/elasticsearch-billing-dimensions.md @@ -59,8 +59,3 @@ You can control costs using the following strategies: Higher resource consumption can lead to higher costs and potentially impact the overall performance of your project. If your use case naturally generates many small, separate streams of data, the recommended approach is to implement a process to consolidate them into fewer, larger indices. This practice leads to more efficient resource utilization. By grouping your data into larger indices, you can ensure a more performant and cost-efficient experience with {{es-serverless}}. -* **Configuration profiles**: When you create a project, the configuration profile also affects the VCU usage and costs. - - The **General purpose** profile is suitable for most search use cases. For example, it is the right profile for full-text search, sparse vectors, and dense vectors that use compression such as BBQ. - - The **Optimized for vectors** profile is recommended only for uncompressed dense vectors with high dimensionality. Though the per VCU cost is the same for general purpose and vector optimized profiles, the latter provides a larger amount of RAM for searchable data. This leads to higher VCU consumption in order to improve the performance for uncompressed vector data. diff --git a/solutions/search/vector/bring-own-vectors.md b/solutions/search/vector/bring-own-vectors.md index cb5150e516..85382bfcd6 100644 --- a/solutions/search/vector/bring-own-vectors.md +++ b/solutions/search/vector/bring-own-vectors.md @@ -20,7 +20,7 @@ You'll also learn the syntax for searching these documents using a [k-nearest ne ## Prerequisites -- If you're using {{es-serverless}}, create a project with the general purpose configuration. The optimized for vectors configuration is not required unless you choose to use uncompressed vectors with high dimensionality. To add the sample data, you must have a `developer` or `admin` predefined role or an equivalent custom role. +- If you're using {{es-serverless}}, create a project with the general purpose configuration. To add the sample data, you must have a `developer` or `admin` predefined role or an equivalent custom role. - If you're using {{ech}} or a self-managed cluster, start {{es}} and {{kib}}. The simplest method to complete the steps in this guide is to log in with a user that has the `superuser` built-in role. To learn about role-based access control, check out [](/deploy-manage/users-roles/cluster-or-deployment-auth/user-roles.md).