Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Smarter default to `index.index_concurrency` #3546

Closed
kimchy opened this Issue Aug 20, 2013 · 0 comments

Comments

Projects
None yet
1 participant
@kimchy
Copy link
Member

commented Aug 20, 2013

A single shard, which is a Lucene level, has a limit on the number of concurrent threads that are allowed to perform indexing at the same time. It defaults to 8 in Lucene, but in ES, we allow to change it using index.index_concurrency.

We should be smarter about setting the default value for it, specifically in cases where there is a single index being indexed into, with one shard on a node. We already have the index/bulk threads pools to protect and control concurrency, so we can increase that default value in most cases to something more relaxed.

On that other hand though, we don't want to default it to a too high value, cause it means each refresh/flush will end up creating more segments.

A safe default can be 65% of the number of bounded cores (bounded at 32), with a minimum of 8 (which is the default in Lucene).

@kimchy kimchy closed this in b9c8ca8 Aug 20, 2013

kimchy added a commit that referenced this issue Aug 20, 2013

mute pushed a commit to mute/elasticsearch that referenced this issue Jul 29, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.