Queue size

[christopher5106]

I'm having the following problem :

"SearchPhaseExecutionException[Failed to execute phase [query], all shards failed; shardFailures {[rlrweRJAQJqaoKfFB8il5A][stt_prod][3]: EsRejectedExecutionException[rejected execution (queue capacity 1000) on org.elasticsearch.action.search.type.TransportSearchTypeAction

It sounds very strange; when I restarted the server, it worked fine again.

What could happen ?

Here is my configuration :

• ES version is 1.0.1
• I have 3 indexes, of respective size 2.5G, 1.7G and 250M, each one has 5 shards
• the cluster is one only one instance (solo)
• the state of the cluster says 15 successful shard, 0 failed shard and 30 total shards (where are the 15 shards missing ?)
• in my settings, mlockall is set to true
• I enabled script.disable_dynamic: false, installed plugins _head and action-updatebyquery
• ES heap size is set 50% which I can confirm using top command :
  5320 elastic+ 20 0 9.918g 4.788g 72980 S 7.6 65.3 29:49.42 java
• I'm using 30% of disk capacity

My traffic is not more than 125 requests per minutes :

So if I understand well, each request can live 30s, how come I have a queue of 1000 ?!
Can ES save the requests in the queue while the shards have failed ?
Why do the shards do not come back ?

Thanks for your help (I'm not using ES usually, more Solr or CloudSearch)

[markwalkom]

Issues here are mostly for code problems, given this is an operational one we will keep it on the mailing list thread you started here - https://groups.google.com/d/msgid/elasticsearch/33bfd4c7-4388-4e02-8e96-d8bb8cdd17ca%40googlegroups.com?utm_medium=email&utm_source=footer