Beat via Kafka

[xuziheng1002]

we use graylog and sidecar to collector application logs,but graylog-server input has been the bottle-neck, so we think it will be better if let Beat input get messages from kafka.
Is there any schedule to develop this feature now?

[florianpopp]

We do not have anything scheduled yet, but this is a valid request.

[BenoitPoulet]

is it on the roadmap ?

You are pushing Filebeat to ingest logs, but we are unable to connect it to a queue.

[BenoitPoulet]

any news about this ?

[rkmbaxed]

News would be great, we are also waitung for this feature.

[waab76]

Is there a reason that the existing Kafka inputs don't work for this case? What additional functionality are we hoping to get from specifically building a Kafka input for Beats?

[bernd]

@waab76 We can reuse the Kafka transport infrastructure but need a new Kafka-based input with a Beats codec. We have to find out how the Beats payload looks like. If it's the same binary payload that is used in the lumberjack protocol, we can probably reuse the Beats input's codec.

[waab76]

Tentatively targeting this for 6.1