New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rules.drl not always updating source field #2188

Closed
123dev opened this Issue May 4, 2016 · 3 comments

Comments

Projects
None yet
4 participants
@123dev

123dev commented May 4, 2016

Problem description

We have the following Drools code A and code B working in Graylog 1.3.x

Code A
// ==================================================================
// Rewrite source and remove .company.com
// ------------------------------------------------------------------
rule "Rewrite remove .company.com from source name"
        when
                m : Message( source matches "(.*)\\.company\\.com" )
        then
                Matcher matcher = Pattern.compile("(.*)\\.company.com",Pattern.CASE_INSENSITIVE).matcher(m.getField("source").toString());
                System.out.println("[.company.com]: Found " + m.getField("source"));
                if (matcher.find()) {
                        m.addField("source", matcher.group(1).toLowerCase());
                        System.out.println("[.company.com]: source updated " + matcher.group(1).toLowerCase());
                }
end
Code A Output:
[.company.com]: Found ABC.company.com
[.company.com]: source updated abc
Code B
// ==================================================================
// Rewrite ip-192-168-1-100 to vm17
// ------------------------------------------------------------------
rule "Rewrite ip-192-168-1-100 host"
        when
                m : Message( source == "ip-192-168-1-100" )
        then
                m.addField("source", "vm17" );
                System.out.println( "[Overwrite ip-192-168-1-100 rule fired] : " + m.toString() );
end

With Graylog 2.0,
Code B is still working fine, however Code A even though the System.out is printing the same messages as 1.3.x, the source field in Graylog is not updated
and is kept as ABC.company.com

quite confused as to why it would work for code B but not code A in Graylog 2.0

Steps to reproduce the problem

  • Enable Drools
  • include the Code A and B in /etc/graylog/server/rules.drl
  • Send Windows Event logs from ABC.company.com through nxlog

Environment

  • Graylog Version: 2.0
  • Elasticsearch Version: 2.3.2
  • MongoDB Version: 3.2.6
  • Operating System: Ubuntu 14.04
  • Browser version: Chrome
@alex-mamchenkov

This comment has been minimized.

alex-mamchenkov commented May 5, 2016

Have similar behavior with Graylog 2.0 drools. In my case the rule similar "Code A" sometimes replaces the source and sometimes does not.

One finding though: if I disable all streams and leave messages just in main input stream - it looks like the rule works everytime, but as soon as I start adding messages to different streams (via stream rules), the drools rule works 70-80% of times.

All same rules used to work 100% on graylog 1.3.4

@hc4

This comment has been minimized.

Contributor

hc4 commented May 5, 2016

Maybe the same problem as here?

@alex-mamchenkov

This comment has been minimized.

alex-mamchenkov commented May 5, 2016

Looks like in my case. Adjusted the server configuration and looks fine now. Will wait for a fix in that ticket.

@kroepke kroepke added the to-verify label May 9, 2016

@kroepke kroepke self-assigned this May 9, 2016

kroepke added a commit that referenced this issue May 13, 2016

MessageProcessors must be instantiated per processing thread
When introducing the MessageProcessor interface, the processing threads accidentally shared the instances (and by induction the MessageFilter instances as well).
That posed no problem for most of the filters, because they do not rely on shared state, but the Drools filter does and could skip messages (because of Drools itself returning early)

This change uses a Provider to get the OrderedMessageProcessor instances explicitly and those do not get shared across threads.

fixes #2119
fixes #2188

@kroepke kroepke added bug and removed to-verify labels May 13, 2016

@kroepke kroepke added this to the 2.0.2 milestone May 13, 2016

joschi added a commit that referenced this issue May 17, 2016

MessageProcessors must be instantiated per processing thread (#2231)
When introducing the MessageProcessor interface, the processing threads accidentally shared the instances (and by induction the MessageFilter instances as well).
That posed no problem for most of the filters, because they do not rely on shared state, but the Drools filter does and could skip messages (because of Drools itself returning early)

This change uses a Provider to get the OrderedMessageProcessor instances explicitly and those do not get shared across threads.

Fixes #2119, fixes #2188

@bernd bernd closed this in ca59a00 May 17, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment