Beats input generates thousands of identical messages #15
Comments
This not remote Beat problem, because their version was not changing for months. |
Attached thread dump, when problem exist - dump.txt |
graylog-server restart helps to solve problem (for some time) |
last error logged before identical messages generation begins:
Actualy I have this message logged many times. About 50 per day and all for same Chinese server with bad internet. |
hc4
added a commit
to hc4/graylog-plugin-beats
that referenced
this issue
Oct 26, 2016
Fixes Graylog2#14 and Graylog2#15 Removed readable bytes checks (it will be checked by ReplayingDecoder) Added decodingLast implementation, that ignores all received data (we can't use it, because can't send ACK) and resets decoding state Fixed error message for bad protocol version
joschi
pushed a commit
that referenced
this issue
Oct 26, 2016
joschi
pushed a commit
that referenced
this issue
Oct 26, 2016
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Problem occured twice.
Graylog version: 2.1.1
Beats input: 1.1.2
Incoming messages raised to 100000+ /s

Graylog CPU usage 100%
From ganglia I can see exact moment, when problem occured.
Incomming traffic also increased

I suspect that root of problem is that Beats input produces messages without confirmation of Ack reveived by remote beat.
Possibly problem caused by #14
The text was updated successfully, but these errors were encountered: