Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
Smarter default to `index.index_concurrency` #3546
A single shard, which is a Lucene level, has a limit on the number of concurrent threads that are allowed to perform indexing at the same time. It defaults to 8 in Lucene, but in ES, we allow to change it using
We should be smarter about setting the default value for it, specifically in cases where there is a single index being indexed into, with one shard on a node. We already have the index/bulk threads pools to protect and control concurrency, so we can increase that default value in most cases to something more relaxed.
On that other hand though, we don't want to default it to a too high value, cause it means each refresh/flush will end up creating more segments.
A safe default can be 65% of the number of bounded cores (bounded at 32), with a minimum of 8 (which is the default in Lucene).