From 3ce35e38a142be3b05332987777509b1809b1c49 Mon Sep 17 00:00:00 2001 From: Stef Nestor <26751266+stefnestor@users.noreply.github.com> Date: Tue, 16 Jul 2024 03:39:30 -0600 Subject: [PATCH] (Doc+) Error "number of documents in the index can't exceed" (#110449) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * (Doc+) Error "number of documents in the index can't exceed" 👋 howdy, team! This adds resolution outline for error ... which induces ongoing, lowkey support ``` Number of documents in the index can't exceed [2147483519] ``` * feedback * feedback Co-authored-by: David Turner * feedback Co-authored-by: David Turner Co-authored-by: Liam Thompson <32779855+leemthompo@users.noreply.github.com> * feedback * feedback * Test change to address docs check failure * Revert test change * Test docs check --------- Co-authored-by: David Turner Co-authored-by: Liam Thompson <32779855+leemthompo@users.noreply.github.com> --- .../how-to/size-your-shards.asciidoc | 33 +++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/docs/reference/how-to/size-your-shards.asciidoc b/docs/reference/how-to/size-your-shards.asciidoc index 56e5fbbf15c77..53f47fc88cdb2 100644 --- a/docs/reference/how-to/size-your-shards.asciidoc +++ b/docs/reference/how-to/size-your-shards.asciidoc @@ -544,3 +544,36 @@ PUT _cluster/settings } } ---- + +[discrete] +==== Number of documents in the shard cannot exceed [2147483519] + + +Elasticsearch shards reflect Lucene's underlying https://github.com/apache/lucene/issues/5176[index +`MAX_DOC` hard limit] of 2,147,483,519 (`(2^31)-129`) docs. This figure is +the sum of `docs.count` plus `docs.deleted` as reported by the <> +per shard. Exceeding this limit will result in errors like the following: + +[source,txt] +---- +Elasticsearch exception [type=illegal_argument_exception, reason=Number of documents in the shard cannot exceed [2147483519]] +---- + +TIP: This calculation may differ from the <> calculation, because the Count API does not include nested documents. + + +Try using the <> to clear deleted docs. For example: + +[source,console] +---- +POST my-index-000001/_forcemerge?only_expunge_deletes=true +---- +// TEST[setup:my_index] + +This will launch an asynchronous task which can be monitored via the <>. + +For a long-term solution try: + +* <> +* aligning the index to recommendations on this page by either +<> or <> the index