Add support for new Elasticsearch versions

[Flole998]

As Elasticsearch is vulnerable to the log4j exploit it is necessary for secure operations to upgrade to version with a fix. However, according to the documentation Elasticsearch versions 7.11 or later should not be used with Graylog. In order to ensure secure operations please support newer, secure versions of Elasticsearch.

[loganmarchione]

Couple things:

1. Elastic says:

"Elasticsearch is not susceptible to remote code execution with this vulnerability due to our use of the Java Security Manager."

2. Graylog uses the open-source version of Elasticsearch, called elasticsearch-oss. This has a different license, which itself might not be compatible with Graylog's license. Also, it appears that elasticsearch-oss stopped development at v7.10.2 (at least according to their docker repos). My point is, upgrading Elasticsearch might not be an easy fix...

[Flole998]

Just because remote code execution is not possible it is still vulnerable to information disclosure caused by this vulnerability.

The license does not restrict how the server can be used/which software is allowed to call the endpoints. Also that change was introduced with 6.3 already, so if this hasn't been an issue since 3 years, why now?

You seem to only look at docker there, but just because no docker image exists that doesn't mean that the software doesn't exist.

[xoxys]

It's not only this special issue with log4j, every other bug or future security issue will not be back ported by Elastic and Graylog users are forced to Elastic 7.7.x-7.10.x which is a bit annoying...

Besides that, the documentation for the supported ES versions is confusing at all... The latest Graylog release it 4.2.x and not 4.0.x and there is absolutely nothing in the docs https://docs.graylog.org/v1/docs/elasticsearch

[xoxys]

"Elasticsearch is not susceptible to remote code execution with this vulnerability due to our use of the Java Security Manager."

That doesn't even make it better, especially as there is a permanent fix available in 7.16.1 which Graylog users can't apply....

[bendem]

A plan is needed (switch to something else, make it compatible, archive graylog and make it legacy), if the project decides to freeze elasticsearch to 7.10, it is basically a dead project, you cannot decide to stop updating a critical part of your infrastructure that's actively developed like it's no big deal.

The users have a right to know if they still have to invest time and energy in something that's not willing to work with up to date versions of its critical components, or if they are better off looking for alternatives.

It's been almost a year since the version was frozen, what's the plan?

[bendem]

Also, this is a duplicate of #11686

[kroepke]

We will announce a plan shortly with regards to Elasticsearch compatibility and support going forward.
Thanks for your patience.

[1tft]

@kroepke We are waiting for this information, because it decides should we upgrade to ES 7.10 or better wait with upgrade for official OpenSearch or Elasticsearch OSS support or anything else...
In times of Log4Shell and many more vulnerabilities we have extreme trouble to explain somebody, that we cant update our base components. We are using Graylog Enterprise, so it should act like enterprise software.

[KlavsKlavsen]

opensearch just released 1.2.1 which has updated log4j - and since it forked at v7.10 - it SHOULD "just work" - with graylog?

[kroepke]

OpenSearch works, but you might need to put it into compatibility mode to report itself as elasticsearch 7.10: https://opensearch.org/docs/latest/clients/agents-and-ingestion-tools/index/ and the current master (which will become Graylog 4.3) also will contain direct support for it: #11435

[AnonymousWP]

I very much support this request. Yesterday I tried using ElasticSearch 7.16.1 in combination with Graylog 4.2.3, but that may have caused issues, so I'm back at the last version of ElasticSearch-OSS again (which its latest release is from January 2021 if I recall correctly...) Couldn't really find a reason why Graylog chose ES-OSS to begin with, but probably because of some extra features (https://medium.com/codex/implementing-security-in-elasticsearch-oss-distribution-using-open-distro-security-plugin-d1d106e62ca6).