Skip to content
29 changes: 27 additions & 2 deletions solutions/search/esql-for-search.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,8 @@
| [DECAY function](#decay-function) | Calculate a relevance score that decays based on the distance of a field value from a target origin | 9.3 |
| [SCORE function](#score-function) | Score an expression using full-text functions; returns scores for all resulting documents | 9.3 |
| [TOP_SNIPPETS function](#top_snippets-function) | Extract the best snippets for a given query string from a text field | 9.3 |
| [MMR command](#result-diversification-with-mmr) | Reduce redundancy in results using Maximum Marginal Relevance diversification | 9.4 (preview) |
| [EMBEDDING function](#embedding-function) | Generate dense vector embeddings from multimodal input using inference endpoints | 9.5 (preview) |

## How search works in {{esql}}

Expand Down Expand Up @@ -123,6 +125,24 @@

Use `TEXT_EMBEDDING` to generate query vectors for KNN searches against your vectorized data or for other dense vector based operations.

### `EMBEDDING` function

The [`EMBEDDING` function](elasticsearch://reference/query-languages/esql/functions-operators/dense-vector-functions/embedding.md) generates dense vector embeddings from multimodal input (text, images) using a specified inference endpoint with the `embedding` task type.

Use `EMBEDDING` when you need to generate query vectors from non-text inputs, such as base64-encoded images, for multimodal search use cases.

### Vector similarity functions

{{esql}} provides functions for computing similarity and distance between dense vectors:

- [`V_COSINE`](elasticsearch://reference/query-languages/esql/functions-operators/dense-vector-functions/v_cosine.md) — cosine similarity
- [`V_DOT_PRODUCT`](elasticsearch://reference/query-languages/esql/functions-operators/dense-vector-functions/v_dot_product.md) — dot product
- [`V_L1_NORM`](elasticsearch://reference/query-languages/esql/functions-operators/dense-vector-functions/v_l1_norm.md) — Manhattan distance
- [`V_L2_NORM`](elasticsearch://reference/query-languages/esql/functions-operators/dense-vector-functions/v_l2_norm.md) — Euclidean distance
- [`V_HAMMING`](elasticsearch://reference/query-languages/esql/functions-operators/dense-vector-functions/v_hamming.md) — Hamming distance

These functions are useful for custom scoring logic, comparing query vectors against stored embeddings, or building custom reranking pipelines using `EVAL`.

### `DECAY` function

The [`DECAY` function](elasticsearch://reference/query-languages/esql/functions-operators/search-functions/decay.md) calculates a relevance score that decays based on the distance of a numeric, spatial, or date field value from a target origin, using configurable decay functions.
Expand Down Expand Up @@ -153,7 +173,7 @@

The [`FORK`](elasticsearch://reference/query-languages/esql/commands/fork.md) and [`FUSE`](elasticsearch://reference/query-languages/esql/commands/fuse.md) commands work together to enable hybrid search in {{esql}}.

`FORK` creates multiple execution branches that operate on the same input data. `FUSE` then combines and scores the results from these branches. Together, these commands allow you to execute different search strategies (such as lexical and semantic searches) in parallel and merge their results with proper relevance scoring.
`FORK` creates multiple execution branches that operate on the same input data. `FUSE` then combines and scores the results from these branches using either RRF (reciprocal rank fusion) or LINEAR (weighted score combination) methods. Together, these commands allow you to execute different search strategies (such as lexical and semantic searches) in parallel and merge their results with proper relevance scoring.

Check notice on line 176 in solutions/search/esql-for-search.md

View workflow job for this annotation

GitHub Actions / build / vale

Elastic.WordChoice: Consider using 'run, start' instead of 'execute', unless the term is in the UI.

Refer to [hybrid search with semantic_text](hybrid-semantic-text.md) for an example or follow the [tutorial](elasticsearch://reference/query-languages/esql/esql-search-tutorial.md).

Expand All @@ -166,7 +186,11 @@

### Semantic reranking with `RERANK`

Use the [`RERANK` command](elasticsearch://reference/query-languages/esql/commands/rerank.md) to re-score search results using inference models for improved relevance.
Use the [`RERANK` command](elasticsearch://reference/query-languages/esql/commands/rerank.md) to re-score search results using inference models for improved relevance. Refer to [semantic reranking](/solutions/search/ranking/semantic-reranking.md) for a comparison of the ES|QL and retriever approaches.

### Result diversification with `MMR`

The [`MMR` command](elasticsearch://reference/query-languages/esql/commands/mmr.md) uses Maximum Marginal Relevance to reduce redundancy in search results by removing documents that are too similar to each other. This is useful for ensuring variety in top results, particularly in RAG workflows where diverse context improves LLM output quality.

## Next steps [esql-for-search-next-steps]

Expand All @@ -177,6 +201,7 @@
### Technical reference [esql-for-search-reference]

- [Search functions](elasticsearch://reference/query-languages/esql/functions-operators/search-functions.md): Complete reference for all search functions
- [Dense vector functions](elasticsearch://reference/query-languages/esql/functions-operators/dense-vector-functions.md): Complete reference for vector embedding and similarity functions
- [Limitations](elasticsearch://reference/query-languages/esql/limitations.md#esql-limitations-full-text-search): Current limitations for search functions in {{esql}}

### Related blog posts [esql-for-search-blogs]
Expand Down
66 changes: 59 additions & 7 deletions solutions/search/querying-for-search.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,15 +19,67 @@ products:
This page is focused on the search use case. For an overview of Elastic query languages for every use case, refer to the [complete overview](/explore-analyze/query-filter/languages.md).
::::

Once you know which [search approaches](search-approaches.md) you need to use, you can start building and testing your search queries. {{es}} provides several query languages to help you express your search logic.
Once you know which [search approaches](search-approaches.md) you need to use, you can start building and testing your search queries. {{es}} provides several query interfaces to help you express your search logic.

| Interface | Endpoint | Description | Best for |
|-----------|----------|-------------|----------|
| [**Query DSL**](/explore-analyze/query-filter/languages/querydsl.md) | [`_search`](the-search-api.md) | Original, JSON-based query language native to Elasticsearch. Powerful but complex syntax. | Full-text and semantic search queries |
| [**ES\|QL**](esql-for-search.md) | `_query` | Fast, SQL-like language with piped syntax, built on new compute architecture. | Filtering, analysis, aggregations |
| [**Retrievers**](retrievers-overview.md) | `_search` | Modern `_search` API syntax focused on composability. | Building complex search pipelines, especially those using semantic search. Required for [semantic reranking](ranking/semantic-reranking.md). |
| Interface | Endpoint | Description |
|-----------|----------|-------------|
| [**Query DSL**](/explore-analyze/query-filter/languages/querydsl.md) | [`_search`](the-search-api.md) | Original, JSON-based query language native to {{es}}. Expressive and well-supported across all client libraries. |
| [**Retrievers**](retrievers-overview.md) | [`_search`](the-search-api.md) | Composable `_search` API syntax for building multi-stage retrieval pipelines in a single request. Built on top of Query DSL. |
| [**ES\|QL**](esql-for-search.md) | `_query` | Piped query language with SQL-like syntax, built on a new compute architecture. Supports full-text search, semantic search, hybrid search, reranking, and text generation. |

These query languages are complementary, not mutually exclusive. You can use different query languages for different parts of your application, based on your specific needs. This flexibility allows you to gradually adopt newer interfaces as your requirements evolve.
These query interfaces are complementary, not mutually exclusive. You can use different interfaces for different parts of your application, based on your specific needs. This flexibility allows you to gradually adopt newer interfaces as your requirements evolve.

## Choosing the right interface

Use the following guidance to decide which interface best fits your use case.

### Query DSL

Choose [Query DSL](/explore-analyze/query-filter/languages/querydsl.md) when you need:

- **Fine-grained control** over individual query clauses, scoring, and boosting
- **The widest ecosystem support** — all {{es}} clients, tools, and integrations work with Query DSL
- **A single-stage query** like a `match`, `bool`, or `term` query without multi-stage retrieval

Query DSL is the foundational query language and remains the right choice for straightforward search queries, especially when you're using a single retrieval strategy.

### Retrievers

Choose [retrievers](retrievers-overview.md) when you need to:

- **Compose multi-stage retrieval pipelines** in a single `_search` call (for example, retrieve → rerank → diversify)
- **Combine multiple retrieval strategies** using RRF or linear combination
- **Apply semantic reranking** with the `text_similarity_reranker` retriever
- **Use the multi-field query format** for simple hybrid search across lexical and semantic fields with automatic score normalization

Retrievers wrap Query DSL and add composability. If your search involves multiple retrieval stages — such as combining BM25 with vector search, or adding a reranking step — retrievers let you express the entire pipeline declaratively.

### ES|QL

Choose [ES|QL](esql-for-search.md) when you need to:

- **Transform or aggregate results** alongside your search (for example, filter → search → rerank → generate)
- **Build end-to-end search workflows** using piped syntax, including hybrid search with `FORK`/`FUSE`, reranking with `RERANK`, and text generation with `COMPLETION`
- **Explore data interactively** using a familiar SQL-like syntax in Kibana or the API
- **Combine search with analytics** such as aggregations, stats, or data transformations in a single query

ES|QL is a good fit when your workflow extends beyond retrieval — for example, when you want to search, rerank, and summarize results in a single piped query.

### Feature comparison

The following table summarizes which capabilities are available in each interface.

| Capability | Query DSL | Retrievers | ES\|QL |
|------------|:---------:|:----------:|:------:|
| Full-text search (BM25) | Yes | Yes | Yes |
| Semantic / vector search | Yes | Yes | Yes |
| Hybrid search (score combination) | — | Yes (RRF, linear) | Yes (FORK/FUSE) |
| Semantic reranking | — | Yes (`text_similarity_reranker`) | Yes (`RERANK`) |
| Result diversification (MMR) | — | Yes (`diversify`) | Yes (`MMR`) |
| Multi-field query format | — | Yes | — |
| Aggregations | Yes | Yes | Yes |
| Text generation (LLM) | — | — | Yes (`COMPLETION`) |
| Piped transformations | — | — | Yes |

::::{note}
You can use the [{{es}} REST APIs]({{es-apis}}) to search your data using any HTTP client, including the [{{es}} client libraries](site-or-app/clients.md), or directly in [Console](/explore-analyze/query-filter/tools/console.md). You can also run searches using [Discover](/explore-analyze/discover.md) in the UI.
Expand Down
5 changes: 5 additions & 0 deletions solutions/search/ranking.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,11 @@ When using the following advanced re-ranking pipelines, first-stage retrieval me

[*Semantic re-ranking*](ranking/semantic-reranking.md) uses machine learning models to reorder search results based on their semantic similarity to a query. Models can be hosted directly in your {{es}} cluster, or you can use [inference endpoints]({{es-apis}}group/endpoint-inference) to call models provided by third-party services. Semantic re-ranking enables out-of-the-box semantic search capabilities on existing full-text search indices.

{{es}} supports semantic re-ranking through two interfaces:

* The [`text_similarity_reranker` retriever](ranking/semantic-reranking.md) for composable retrieval pipelines via the `_search` API
* The [ES|QL `RERANK` command](ranking/semantic-reranking.md) for piped search workflows via the `_query` API
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is workflows an overloaded term here, considering Workflows is a product offered in Kibana?



#### Learning to Rank (LTR) [re-ranking-overview-ltr]

Expand Down
Loading
Loading