Allow startup without working Elasticsearch cluster #1289
Conversation
The previous implementation of IndexerSetupService (and related classes) would should down Graylog if Elasticsearch wasn't available or the cluster health state was RED on startup. This made sense as long as Graylog didn't have a proper on-disk journal in which messages could be persistently buffered. The new implementation simply acknowledges the unavailability of Elasticsearch and generates an appropriate system notification (and log entries) but does not shutdown Graylog. Fixes #1136
| @@ -114,11 +111,10 @@ protected void startUp() throws Exception { | |||
| Tools.silenceUncaughtExceptionsInThisThread(); | |||
|
|
|||
| LOG.debug("Starting indexer"); | |||
| try { | |||
| node.start(); | |||
| node.start(); | |||
There was a problem hiding this comment.
are we sure this and the next line never throw exceptions?
if they do, we would miss them and not print any error courtesy of Tools.silenceUncaughtExceptionsInThisThread();
There was a problem hiding this comment.
The behavior is the same as before as the outer most catch block just re-threw the exception: https://github.com/Graylog2/graylog2-server/pull/1289/files#diff-72fd860130aebd46f43cb3fa1fb32d18L193
|
other than the comment above this looks really good 👍 |
| @@ -28,6 +28,8 @@ | |||
| MULTI_MASTER, | |||
| NO_MASTER, | |||
| ES_OPEN_FILES, | |||
| ES_CLUSTER_RED, | |||
| ES_UNAVAILABLE, | |||
| NO_INPUT_RUNNING, | |||
There was a problem hiding this comment.
This requires an update to the web interface as well, right?
There was a problem hiding this comment.
Yes, see graylog-labs/graylog2-web-interface#1516.
|
I tested the branch with Elasticsearch 1.5. Seems to work fine! I am getting the following message in the logs on a regular basis. Maybe we should have an "es alive" check in the |
|
@bernd Yes, thought about that as well. I would add this in another PR, though. |
Ack! LGTM 👍 |
Allow startup without working Elasticsearch cluster
The previous implementation of
IndexerSetupService(and related classes) would shutdown Graylog if Elasticsearch wasn't available or the cluster health state was RED on startup. This made sense as long as Graylog didn't have a proper on-disk journal in which messages could be persistently buffered.The new implementation simply acknowledges the unavailability of Elasticsearch and generates an appropriate system notification (and log entries) but does not shutdown Graylog.
Fixes #1136