Skip to content
This repository has been archived by the owner on Sep 28, 2022. It is now read-only.

Latest commit

 

History

History
40 lines (27 loc) · 1.68 KB

File metadata and controls

40 lines (27 loc) · 1.68 KB
code type title description
true
page
fetchNext
SearchResult:fetchNext

fetchNext

Fetches the next SearchResult, by triggering a new search/scroll request depending on the options and filters of the SearchResult.

If the previous request was a search or a scroll action which provided a scroll argument, fetchNext will use the scrollId retrieved from the current result to make a new scroll request.

If the previous request was a search action which provided size argument and sort filtering, fetchNext will use Elasticsearch's search_after mechanism, which can efficiently search through a large volume of documents, bypassing internal hard limits[1], but at the cost of reflecting the latest changes of the index, as opposed to using scroll.

If the previous request was a search action which provided from and size arguments, fetchNext will add size to from and make a new search request.


How to process every document of a collection

The safest way to process all documents in a collection is to fetch them as a batch in order to avoid memory exhaustion and possibly hitting some hard limits[1] on the database layer.

:::warning Make sure your first search request includes the size and scroll parameters :::

:::info [1] Elasticsearch limits the number of documents inside a single page to 10,000 by default. :::

Usage

<<< ./snippets/fetch-next-1.java

<<< ./snippets/fetch-next-2.java