From 328e74f55876a531bfb73b101c0a8572e12ec67c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Istv=C3=A1n=20Zolt=C3=A1n=20Szab=C3=B3?= Date: Wed, 24 Sep 2025 14:18:03 +0200 Subject: [PATCH 1/6] [SEARCH] Adds EIS scenario to semantic search tutorial. --- .../semantic-search-semantic-text.md | 66 ++++++++++++++++--- 1 file changed, 58 insertions(+), 8 deletions(-) diff --git a/solutions/search/semantic-search/semantic-search-semantic-text.md b/solutions/search/semantic-search/semantic-search-semantic-text.md index a81512886f..27a6ddfb2d 100644 --- a/solutions/search/semantic-search/semantic-search-semantic-text.md +++ b/solutions/search/semantic-search/semantic-search-semantic-text.md @@ -27,7 +27,14 @@ This tutorial uses the `elasticsearch` service for demonstration, which is creat The mapping of the destination index - the index that contains the embeddings that the inference endpoint will generate based on your input text - must be created. The destination index must have a field with the [`semantic_text`](elasticsearch://reference/elasticsearch/mapping-reference/semantic-text.md) field type to index the output of the used inference endpoint. -```console +::::{tab-set} + +:::{tab-item} Using EIS on Serverless + +```{applies_to} +serverless: ga +``` + PUT semantic-embeddings { "mappings": { @@ -38,17 +45,64 @@ PUT semantic-embeddings } } } +::: + +1. The name of the field to contain the generated embeddings. +2. The field to contain the embeddings is a `semantic_text` field. Since no `inference_id` is provided, the default endpoint `.elser-2-elastic` for the `elasticsearch` service is used. This {{infer}} endpoint uses the [Elastic {{infer-cap}} Service (EIS)](/explore-analyze/elastic-inference/eis.md). + +:::{tab-item} Using EIS in Cloud + +```{applies_to} +stack: ga +deployment: + self: unavailable ``` +PUT semantic-embeddings +{ + "mappings": { + "properties": { + "content": { <1> + "type": "semantic_text", <2> + "inference_id": ".elser-2-elastic" <3> + } + } + } +} +::: + 1. The name of the field to contain the generated embeddings. -2. The field to contain the embeddings is a `semantic_text` field. Since no `inference_id` is provided, the default endpoint `.elser-2-elasticsearch` for the `elasticsearch` service is used. To use a different {{infer}} service, you must create an {{infer}} endpoint first using the [Create {{infer}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-inference-put) and then specify it in the `semantic_text` field mapping using the `inference_id` parameter. +2. The field to contain the embeddings is a `semantic_text` field. +3. The `.elser-2-elastic` preconfigured {{infer}} endpoint for the `elasticsearch` service is used. This {{infer}} endpoint uses the [Elastic {{infer-cap}} Service (EIS)](/explore-analyze/elastic-inference/eis.md). -::::{note} -If you’re using web crawlers or connectors to generate indices, you have to [update the index mappings](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-put-mapping) for these indices to include the `semantic_text` field. Once the mapping is updated, you’ll need to run a full web crawl or a full connector sync. This ensures that all existing documents are reprocessed and updated with the new semantic embeddings, enabling semantic search on the updated data. +:::{tab-item} Using ML-nodes + +```console +PUT semantic-embeddings +{ + "mappings": { + "properties": { + "content": { <1> + "type": "semantic_text", <2> + "inference_id": ".elser-2-elasticsearch" <3> + } + } + } +} +``` + +1. The name of the field to contain the generated embeddings. +2. The field to contain the embeddings is a `semantic_text` field. +3. The `.elser-2-elasticsearch` preconfigured {{infer}} endpoint for the `elasticsearch` service is used. To use a different {{infer}} service, you must create an {{infer}} endpoint first using the [Create {{infer}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-inference-put) and then specify it in the `semantic_text` field mapping using the `inference_id` parameter. + +::: :::: +::::{note} +If you’re using web crawlers or connectors to generate indices, you have to [update the index mappings](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-put-mapping) for these indices to include the `semantic_text` field. Once the mapping is updated, you’ll need to run a full web crawl or a full connector sync. This ensures that all existing documents are reprocessed and updated with the new semantic embeddings, enabling semantic search on the updated data. +:::: ## Load data [semantic-text-load-data] @@ -58,7 +112,6 @@ Use the `msmarco-passagetest2019-top1000` data set, which is a subset of the MS Download the file and upload it to your cluster using the [Data Visualizer](../../../manage-data/ingest/upload-data-files.md) in the {{ml-app}} UI. After your data is analyzed, click **Override settings**. Under **Edit field names**, assign `id` to the first column and `content` to the second. Click **Apply**, then **Import**. Name the index `test-data`, and click **Import**. After the upload is complete, you will see an index named `test-data` with 182,469 documents. - ## Reindex the data [semantic-text-reindex-data] Create the embeddings from the text by reindexing the data from the `test-data` index to the `semantic-embeddings` index. The data in the `content` field will be reindexed into the `content` semantic text field of the destination index. The reindexed data will be processed by the {{infer}} endpoint associated with the `content` semantic text field. @@ -68,7 +121,6 @@ This step uses the reindex API to simulate data ingestion. If you are working wi :::: - ```console POST _reindex?wait_for_completion=false { @@ -84,7 +136,6 @@ POST _reindex?wait_for_completion=false 1. The default batch size for reindexing is 1000. Reducing size to a smaller number makes the update of the reindexing process quicker which enables you to follow the progress closely and detect errors early. - The call returns a task ID to monitor the progress: ```console @@ -97,7 +148,6 @@ Reindexing large datasets can take a long time. You can test this workflow using POST _tasks//_cancel ``` - ## Semantic search [semantic-text-semantic-search] After the data has been indexed with the embeddings, you can query the data using semantic search. Choose between [Query DSL](/explore-analyze/query-filter/languages/querydsl.md) or [{{esql}}](elasticsearch://reference/query-languages/esql.md) syntax to execute the query. From 59c3b4218bc86763f8b06fa933b63cfbace17868 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Istv=C3=A1n=20Zolt=C3=A1n=20Szab=C3=B3?= Date: Wed, 24 Sep 2025 14:32:24 +0200 Subject: [PATCH 2/6] Updates markup. --- .../semantic-search-semantic-text.md | 22 ++++++++++++------- 1 file changed, 14 insertions(+), 8 deletions(-) diff --git a/solutions/search/semantic-search/semantic-search-semantic-text.md b/solutions/search/semantic-search/semantic-search-semantic-text.md index 27a6ddfb2d..69da666200 100644 --- a/solutions/search/semantic-search/semantic-search-semantic-text.md +++ b/solutions/search/semantic-search/semantic-search-semantic-text.md @@ -27,14 +27,15 @@ This tutorial uses the `elasticsearch` service for demonstration, which is creat The mapping of the destination index - the index that contains the embeddings that the inference endpoint will generate based on your input text - must be created. The destination index must have a field with the [`semantic_text`](elasticsearch://reference/elasticsearch/mapping-reference/semantic-text.md) field type to index the output of the used inference endpoint. -::::{tab-set} +:::::::{tab-set} -:::{tab-item} Using EIS on Serverless +::::::{tab-item} Using EIS on Serverless ```{applies_to} serverless: ga ``` +```console PUT semantic-embeddings { "mappings": { @@ -45,12 +46,14 @@ PUT semantic-embeddings } } } -::: +``` 1. The name of the field to contain the generated embeddings. 2. The field to contain the embeddings is a `semantic_text` field. Since no `inference_id` is provided, the default endpoint `.elser-2-elastic` for the `elasticsearch` service is used. This {{infer}} endpoint uses the [Elastic {{infer-cap}} Service (EIS)](/explore-analyze/elastic-inference/eis.md). -:::{tab-item} Using EIS in Cloud +:::::: + +::::::{tab-item} Using EIS in Cloud ```{applies_to} stack: ga @@ -58,6 +61,7 @@ deployment: self: unavailable ``` +```console PUT semantic-embeddings { "mappings": { @@ -69,13 +73,15 @@ PUT semantic-embeddings } } } -::: +``` 1. The name of the field to contain the generated embeddings. 2. The field to contain the embeddings is a `semantic_text` field. 3. The `.elser-2-elastic` preconfigured {{infer}} endpoint for the `elasticsearch` service is used. This {{infer}} endpoint uses the [Elastic {{infer-cap}} Service (EIS)](/explore-analyze/elastic-inference/eis.md). -:::{tab-item} Using ML-nodes +:::::: + +::::::{tab-item} Using ML-nodes ```console PUT semantic-embeddings @@ -95,9 +101,9 @@ PUT semantic-embeddings 2. The field to contain the embeddings is a `semantic_text` field. 3. The `.elser-2-elasticsearch` preconfigured {{infer}} endpoint for the `elasticsearch` service is used. To use a different {{infer}} service, you must create an {{infer}} endpoint first using the [Create {{infer}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-inference-put) and then specify it in the `semantic_text` field mapping using the `inference_id` parameter. -::: +:::::: -:::: +::::::: ::::{note} If you’re using web crawlers or connectors to generate indices, you have to [update the index mappings](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-put-mapping) for these indices to include the `semantic_text` field. Once the mapping is updated, you’ll need to run a full web crawl or a full connector sync. This ensures that all existing documents are reprocessed and updated with the new semantic embeddings, enabling semantic search on the updated data. From 2cf30ef437ab8274a16885436a7efbe689c4265a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Istv=C3=A1n=20Zolt=C3=A1n=20Szab=C3=B3?= Date: Wed, 24 Sep 2025 14:59:52 +0200 Subject: [PATCH 3/6] Adds explanatory sentence. --- .../search/semantic-search/semantic-search-semantic-text.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/solutions/search/semantic-search/semantic-search-semantic-text.md b/solutions/search/semantic-search/semantic-search-semantic-text.md index 69da666200..8e65405385 100644 --- a/solutions/search/semantic-search/semantic-search-semantic-text.md +++ b/solutions/search/semantic-search/semantic-search-semantic-text.md @@ -27,6 +27,8 @@ This tutorial uses the `elasticsearch` service for demonstration, which is creat The mapping of the destination index - the index that contains the embeddings that the inference endpoint will generate based on your input text - must be created. The destination index must have a field with the [`semantic_text`](elasticsearch://reference/elasticsearch/mapping-reference/semantic-text.md) field type to index the output of the used inference endpoint. +You can run {{infer}} either using the [Elastic {{infer-cap}} Service](/explore-analyze/elastic-inference/eis.md) or on your own ML-nodes. The following examples show you both scenarios. + :::::::{tab-set} ::::::{tab-item} Using EIS on Serverless From dee2d50e3796c516501ccf5cf1e855a9cc1a0495 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Istv=C3=A1n=20Zolt=C3=A1n=20Szab=C3=B3?= Date: Thu, 25 Sep 2025 12:34:44 +0200 Subject: [PATCH 4/6] Update solutions/search/semantic-search/semantic-search-semantic-text.md --- .../search/semantic-search/semantic-search-semantic-text.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/solutions/search/semantic-search/semantic-search-semantic-text.md b/solutions/search/semantic-search/semantic-search-semantic-text.md index 8e65405385..aa22c29fbe 100644 --- a/solutions/search/semantic-search/semantic-search-semantic-text.md +++ b/solutions/search/semantic-search/semantic-search-semantic-text.md @@ -51,7 +51,7 @@ PUT semantic-embeddings ``` 1. The name of the field to contain the generated embeddings. -2. The field to contain the embeddings is a `semantic_text` field. Since no `inference_id` is provided, the default endpoint `.elser-2-elastic` for the `elasticsearch` service is used. This {{infer}} endpoint uses the [Elastic {{infer-cap}} Service (EIS)](/explore-analyze/elastic-inference/eis.md). +2. The field to contain the embeddings is a `semantic_text` field. Since no `inference_id` is provided, the default endpoint `.elser-v2-elastic` for the `elasticsearch` service is used. This {{infer}} endpoint uses the [Elastic {{infer-cap}} Service (EIS)](/explore-analyze/elastic-inference/eis.md). :::::: From 73f16f22d278c74fbbc1ae11239ade62d38c32be Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Istv=C3=A1n=20Zolt=C3=A1n=20Szab=C3=B3?= Date: Thu, 25 Sep 2025 12:34:51 +0200 Subject: [PATCH 5/6] Update solutions/search/semantic-search/semantic-search-semantic-text.md --- .../search/semantic-search/semantic-search-semantic-text.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/solutions/search/semantic-search/semantic-search-semantic-text.md b/solutions/search/semantic-search/semantic-search-semantic-text.md index aa22c29fbe..6675f2f872 100644 --- a/solutions/search/semantic-search/semantic-search-semantic-text.md +++ b/solutions/search/semantic-search/semantic-search-semantic-text.md @@ -70,7 +70,7 @@ PUT semantic-embeddings "properties": { "content": { <1> "type": "semantic_text", <2> - "inference_id": ".elser-2-elastic" <3> + "inference_id": ".elser-v2-elastic" <3> } } } From bb71d2c3fe9e910bd3acdffc1baa7b919a4dab6d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Istv=C3=A1n=20Zolt=C3=A1n=20Szab=C3=B3?= Date: Thu, 25 Sep 2025 12:35:00 +0200 Subject: [PATCH 6/6] Update solutions/search/semantic-search/semantic-search-semantic-text.md --- .../search/semantic-search/semantic-search-semantic-text.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/solutions/search/semantic-search/semantic-search-semantic-text.md b/solutions/search/semantic-search/semantic-search-semantic-text.md index 6675f2f872..5880f28ebb 100644 --- a/solutions/search/semantic-search/semantic-search-semantic-text.md +++ b/solutions/search/semantic-search/semantic-search-semantic-text.md @@ -79,7 +79,7 @@ PUT semantic-embeddings 1. The name of the field to contain the generated embeddings. 2. The field to contain the embeddings is a `semantic_text` field. -3. The `.elser-2-elastic` preconfigured {{infer}} endpoint for the `elasticsearch` service is used. This {{infer}} endpoint uses the [Elastic {{infer-cap}} Service (EIS)](/explore-analyze/elastic-inference/eis.md). +3. The `.elser-v2-elastic` preconfigured {{infer}} endpoint for the `elasticsearch` service is used. This {{infer}} endpoint uses the [Elastic {{infer-cap}} Service (EIS)](/explore-analyze/elastic-inference/eis.md). ::::::