-
Notifications
You must be signed in to change notification settings - Fork 15
Pipeline Processor plugin warning in Graylog 2.4.0 #236
Comments
Same issue on creating new pipelins - pipelines from Graylog 2.3 work fine. Graylog Version: 2.4.0 |
Actually, that's the case for the 2.3 to 2.4 systems I upgraded too... the pipeline processing configured as part of 2.3.x or earlier is still working in them now they're on 2.4.x. I'm not sure about what happens if I try to make changes there - I'm loathe to risk breaking them. |
tl;dr: Everything is fine. This message is the result of an unfortunate oversight after a refactoring which changed how updated rules, pipelines and pipeline stream connections are handled. The new state used to be posted on an event bus in the server(s) and then processed asynchronously by some other component. The posting still happens, but nothing is listening anymore because the pipeline state is accessed differently now, thus the event bus complains that some event is posted, but no one read it, because that typically indicates a bug or failure somewhere. We'll remove the warning in 2.4.1 and 3.0. Thanks for reporting this! |
@colin-stubbs Aside from the comment I just left, does processing work or not work for you at all? The first sentence implies that it doesn't, whereas the rest of the post makes it look to me as it does. |
@kroepke it definitely wasn't adding the test field.... but after changing a lot of things including reinstalling the graylog-server package, nuking all streams data caches, lookup tables etc related to the Threat Intel plugin that I was initially trying to use it's come good again. I'm 100% certain it wasn't working prior to that... pretty sure there's a larger issue there somewhere still. |
Unfortunately the issue is not reproducible in the latest release. Please re-open this issue if it still occurs, ideally with a set of reproducible steps. Thanks! |
@jalogisch I will re-open this because there is still an open task for 3.0 and 2.4.1. (#236 (comment)) I will also change the title for this. |
Problem description
Pipeline Processing does not work at all.
As below; almost all other plugins have been removed. The only error logged relates to the pipeline processs itself.
All pipelines, rules and stream/pipeline connections etc had been deleted to isolate the issue. graylog server restarted clean with ZERO pipeline configuration in GUI or in mongoDB.
Creating a simple rule as below,
rule "test"
when
has_field("message")
then
set_field("test","1234");
end
will result in the following error message,
Logs error "WARN [DeadEventLoggingListener] Received unhandled event of type <org.graylog.plugins.pipelineprocessor.processors.PipelineInterpreter.State> from event bus <AsyncEventBus{graylog-eventbus}>"
Deleting the same rule does not result in a log message.
Create a test pipeline results in the same log message.
Connecting the test pipeline to the "All Messages" stream results in the same log message.
Adding the test rule to the test pipeline results in the same log message.
Using the pipeline simulator with dummy message from "All Messages" stream shows that test pipeline rule should work, e.g.
Simulation results
These are the results of processing the loaded message. Processing took 660 µs.
Changes in original message de877dc1-f162-11e7-bc6d-000c29233450
Added fields
test
1234
2018-01-05T01:09:53.326+10:00 WARN [NettyTransport] receiveBufferSize (SO_RCVBUF) for input SyslogUDPInput{title=Cb-Protection-Syslog, type=org.graylog2.inputs.syslog.udp.SyslogUDPInput, nodeId=null} should be 262144 but is 212992.
2018-01-05T01:09:53.329+10:00 INFO [InputStateListener] Input [Syslog UDP/5a46d2be1d67a30519e24d88] is now RUNNING
2018-01-05T01:09:53.357+10:00 WARN [NettyTransport] receiveBufferSize (SO_RCVBUF) for input GELFTCPInput{title=Global-GELF-TCP-12201, type=org.graylog2.inputs.gelf.tcp.GELFTCPInput, nodeId=null} should be 1048576 but is 212992.
2018-01-05T01:09:53.359+10:00 INFO [InputStateListener] Input [GELF TCP/5a1cc21b1d67a3df2de91cfb] is now RUNNING
2018-01-05T01:10:47.418+10:00 WARN [DeadEventLoggingListener] Received unhandled event of type <org.graylog.plugins.pipelineprocessor.processors.PipelineInterpreter.State> from event bus <AsyncEventBus{graylog-eventbus}>
^C
[root@c01 plugin]# ls -l
total 11928
-rw-r--r--. 1 root root 6615813 Dec 22 22:29 graylog-plugin-map-widget-2.4.0.jar
-rw-r--r--. 1 root root 5592446 Dec 22 22:29 graylog-plugin-pipeline-processor-2.4.0.jar
[root@c01 plugin]# rpm -V graylog-server
SM5...GT. c /etc/graylog/server/server.conf
S.5....T. c /etc/sysconfig/graylog-server
missing /usr/share/graylog-server/plugin/graylog-plugin-aws-2.4.0.jar
missing /usr/share/graylog-server/plugin/graylog-plugin-beats-2.4.0.jar
missing /usr/share/graylog-server/plugin/graylog-plugin-cef-2.4.0.jar
missing /usr/share/graylog-server/plugin/graylog-plugin-collector-2.4.0.jar
missing /usr/share/graylog-server/plugin/graylog-plugin-enterprise-integration-2.4.0.jar
missing /usr/share/graylog-server/plugin/graylog-plugin-netflow-2.4.0.jar
missing /usr/share/graylog-server/plugin/graylog-plugin-threatintel-2.4.0.jar
[root@c01 plugin]# cat /etc/sysconfig/graylog-server
WARNING: This file is managed by Salt Stack
GRAYLOG_SERVER_ARGS=""
GRAYLOG_COMMAND_WRAPPER=""
JAVA="/usr/bin/java"
GRAYLOG_SERVER_JAVA_OPTS="-Xms4g -Xmx4g -XX:NewRatio=1 -server -XX:+ResizeTLAB -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled -XX:+UseParNewGC -XX:-OmitStackTraceInFastThrow -Djavax.net.ssl.trustStore=/etc/pki/java/cacerts"
EOF
[root@c01 plugin]# cat /etc/sysconfig/graylog-server graylog-plugin-^C
[root@c01 plugin]# rpm -qa | grep -i -e elasticsearch -e mongo -e graylog
mongodb-org-server-3.4.10-1.el7.x86_64
mongodb-org-shell-3.4.10-1.el7.x86_64
mongodb-org-tools-3.4.10-1.el7.x86_64
graylog-server-2.4.0-9.noarch
python-pymongo-2.5.2-4.el7.x86_64
elasticsearch-5.6.5-1.noarch
mongodb-org-mongos-3.4.10-1.el7.x86_64
mongodb-org-3.4.10-1.el7.x86_64
[root@c01 plugin]#
Steps to reproduce the problem
Environment
The text was updated successfully, but these errors were encountered: