Skip to content

Commit

Permalink
Set queue sizes by default on bulk/index thread pools
Browse files Browse the repository at this point in the history
Now that we properly fixed the ability to set the queue size on the index / bulk thread pool, we should actually set them to a somehow reasonable value to protect from users potentially overflowing our system.

I suggest defaults to be 50 for bulk, and 200 for indexing.

Also, set the thread pool for get, which we should set (in a similar value to a "read" queue size we have today).
closes #3888
  • Loading branch information
kimchy committed Oct 12, 2013
1 parent 8a3df0c commit 420b339
Show file tree
Hide file tree
Showing 2 changed files with 14 additions and 4 deletions.
12 changes: 11 additions & 1 deletion docs/reference/modules/threadpool.asciidoc
Expand Up @@ -9,22 +9,32 @@ pools, but the important ones include:
`index`::
For index/delete operations, defaults to `fixed`,
size `# of available processors`.
queue_size `200`.

`search`::
For count/search operations, defaults to `fixed`,
size `3x # of available processors`.
queue_size `1000`.

`suggest`::
For suggest operations, defaults to `fixed`,
size `# of available processors`.
queue_size `1000`.

`get`::
For get operations, defaults to `fixed`
size `# of available processors`.
queue_size `1000`.

`bulk`::
`bulk`::
For bulk operations, defaults to `fixed`
size `# of available processors`.
queue_size `50`.

`percolate`::
For percolate operations, defaults to `fixed`
size `# of available processors`.
queue_size `1000`.

`warmer`::
For segment warm-up operations, defaults to `scaling`
Expand Down
6 changes: 3 additions & 3 deletions src/main/java/org/elasticsearch/threadpool/ThreadPool.java
Expand Up @@ -105,9 +105,9 @@ public ThreadPool(Settings settings, @Nullable NodeSettingsService nodeSettingsS
int halfProcMaxAt10 = Math.min(((availableProcessors + 1) / 2), 10);
defaultExecutorTypeSettings = ImmutableMap.<String, Settings>builder()
.put(Names.GENERIC, settingsBuilder().put("type", "cached").put("keep_alive", "30s").build())
.put(Names.INDEX, settingsBuilder().put("type", "fixed").put("size", availableProcessors).build())
.put(Names.BULK, settingsBuilder().put("type", "fixed").put("size", availableProcessors).build())
.put(Names.GET, settingsBuilder().put("type", "fixed").put("size", availableProcessors).build())
.put(Names.INDEX, settingsBuilder().put("type", "fixed").put("size", availableProcessors).put("queue_size", 200).build())
.put(Names.BULK, settingsBuilder().put("type", "fixed").put("size", availableProcessors).put("queue_size", 50).build())
.put(Names.GET, settingsBuilder().put("type", "fixed").put("size", availableProcessors).put("queue_size", 1000).build())
.put(Names.SEARCH, settingsBuilder().put("type", "fixed").put("size", availableProcessors * 3).put("queue_size", 1000).build())
.put(Names.SUGGEST, settingsBuilder().put("type", "fixed").put("size", availableProcessors).put("queue_size", 1000).build())
.put(Names.PERCOLATE, settingsBuilder().put("type", "fixed").put("size", availableProcessors).put("queue_size", 1000).build())
Expand Down

0 comments on commit 420b339

Please sign in to comment.