Elasticsearch OutOfMemoryError

[kiaraho]

Getting the following error in Kibana a few weeks after upgrading to 2.3.40.

{"statusCode":503,"error":"Service Unavailable","message":"License is not available."}

Running so-status on the Manager node shows so-elasticsearch has an ERROR. Output from docker logs so-elasticsearch attached in elasticsearch.log. Manager hostname has been intentionally redacted and replaced with XXXXX.

Manager node has 16GB of memory and its minion file has esheap: 4063m configured. Is there a recommended sizing for this setting? We are running a distributed architecture with Logstash running on a separate Search node.

[trocade]

For ES you can increase the heap up to 31GB (over this and the it will get a performance slowdown). The only thing to really keep track of it that you have enough memory to support heap*2+OS (and other programs) to have file caches aswell.

If 4096 is not enough I would start by maybe increasing to 6GB or 8 GB and see how it performs.

[kiaraho]

Thanks for the advice. Manager seems to holding up pretty well at esheap: 6095m, based on your heap*2+OS recommendation. Memory usage has been hovering around 85% for last couple of days. Will keep monitoring it and adjust it accordingly.