elasticsearch index failures not log into log file

[jalogisch]

Expected Behavior

If Graylog is not able to ingest the messages to elasticsearch it should write a meaningful log message that contains something the admin can work with. That should contain what and why with as much information as possible.

The information / log should be found in the server.log as this would be the place where ppl check if anything is not working.

Current Behavior

In the MongoDB we have currently the following entry:

{
    "timestamp": "2017-09-17T13:32:19.000Z",
    "letter_id": "a11dbb91-9bac-11e7-9aa5-782bcb5376e0",
    "message": "{\"type\":\"unavailable_shards_exception\",\"reason\":\"[graylog_5728][0] Not enough active copies to meet write consistency of [QUORUM] (have 1, needed 2). Timeout: [1m], request: [BulkShardRequest to [graylog_5728] containing [113] requests]\"}",
    "index": "graylog_5728",
    "type": "message"
  },

But in the Logfile during that time is only visible that a deflector cycle had happened.

2017-09-17T13:32:18.964Z INFO  [MongoIndexSet] Pointing index alias <graylog_deflector> to new index <graylog_5728>.
2017-09-17T13:32:19.107Z INFO  [SystemJobManager] Submitted SystemJob <a0b62020-9bac-11e7-9aa5-782bcb5376e0> [org.graylog2.indexer.indices.jobs.SetIndexReadOnlyAndCalculateRangeJob]
2017-09-17T13:32:19.107Z INFO  [MongoIndexSet] Successfully pointed index alias <graylog_deflector> to index <graylog_5728>.
2017-09-17T13:32:49.296Z INFO  [SetIndexReadOnlyJob] Flushing old index <graylog_5727>.
2017-09-17T13:32:49.945Z INFO  [SetIndexReadOnlyJob] Setting old index <graylog_5727> to read-only.
2017-09-17T13:32:51.115Z INFO  [SystemJobManager] Submitted SystemJob <b3ca4fb6-9bac-11e7-9aa5-782bcb5376e0> [org.graylog2.indexer.indices.jobs.OptimizeIndexJob]

Context

In case something unexpected happen we could not shift between different location of possible log information. In fact the server.log should be the single source of information. To make investigation easier.

Additional the given Information should be with out a doubt useful and say what happened.

Your Environment

• Graylog Version: 2.3.1