Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 36 additions & 2 deletions docs/reference/query-languages/esql/limitations.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,20 +10,54 @@

## Result set size limit [esql-max-rows]

By default, an {{esql}} query returns up to 1000 rows. You can increase the number of rows up to 10,000 using the [`LIMIT`](/reference/query-languages/esql/esql-commands.md#esql-limit) command. Queries do not return more than 10,000 rows, regardless of the `LIMIT` command’s value.
By default, an {{esql}} query returns up to 1,000 rows. You can increase the number of rows up to 10,000 using the [`LIMIT`](/reference/query-languages/esql/esql-commands.md#esql-limit) command.

This limit only applies to the number of rows that are retrieved by the query. Queries and aggregations run on the full data set.
For instance,
```esql
FROM index | WHERE field = "value"
```
is equivalent to:
```esql
FROM index | WHERE field = "value" | LIMIT 1000
```

Queries do not return more than 10,000 rows, regardless of the `LIMIT` command’s value. This is a configurable upper limit.

To overcome this limitation:

* Reduce the result set size by modifying the query to only return relevant data. Use [`WHERE`](/reference/query-languages/esql/esql-commands.md#esql-where) to select a smaller subset of the data.
* Shift any post-query processing to the query itself. You can use the {{esql}} [`STATS`](/reference/query-languages/esql/esql-commands.md#esql-stats-by) command to aggregate data in the query.

The upper limit only applies to the number of rows that are output by the query, not to the number of documents it processes: the query runs on the full data set.

Consider the following two queries:
```esql
FROM index | WHERE field0 == "value" | LIMIT 20000
```
and
```esql
FROM index | STATS AVG(field1) BY field2 | LIMIT 20000
```

In both cases, the filtering by `field0` in the first query or the grouping by `field2` in the second is applied over all the documents present in the `index`, irrespective of their number or indexes size. However, both queries will return at most 10,000 rows, even if there were more rows available to return.

The default and maximum limits can be changed using these dynamic cluster settings:

* `esql.query.result_truncation_default_size`
* `esql.query.result_truncation_max_size`

However, doing so involves trade-offs. A larger result-set involves a higher memory pressure and increased processing times; the internode traffic within and across clusters can also increase.

These limitations are similar to those enforced by the [search API for pagination](/reference/elasticsearch/rest-apis/paginate-search-results.md#paginate-search-results).

Check failure on line 51 in docs/reference/query-languages/esql/limitations.md

View workflow job for this annotation

GitHub Actions / docs-preview / build

`paginate-search-results` does not exist in paginate-search-results.md.

| Functionality | Search | {{esql}} |
|----------------------------------|-------------------------|-------------------------------------------|
| Results returned by default | 10 | 1.000 |
| Default upper limit | 10,000 | 10,000 |
| Specify number of results | `size` | `LIMIT` |
| Change default number of results | n/a | esql.query.result_truncation_default_size |
| Change default upper limit | index-max-result-window | esql.query.result_truncation_max_size |


## Field types [esql-supported-types]

Expand Down
Loading