Skip to content
This repository has been archived by the owner on Mar 21, 2023. It is now read-only.

NPE during preProcessArgs using Grok pattern #24

Closed
kroepke opened this issue May 4, 2016 · 8 comments
Closed

NPE during preProcessArgs using Grok pattern #24

kroepke opened this issue May 4, 2016 · 8 comments
Assignees
@kroepke
Labels
bug
Milestone
1.0.0

Comments

@kroepke
Copy link
Member

kroepke commented May 4, 2016

Problem description

Saving the rule below leads to a bad error message (cannot save rule "" with a 500) and this exception in the server log:

2016-05-04T10:58:12.307+02:00 ERROR [AnyExceptionClassMapper] Unhandled exception in REST resource
java.lang.NullPointerException
    at org.graylog.plugins.pipelineprocessor.ast.functions.Function.preprocessArgs(Function.java:55) ~[?:?]
    at org.graylog.plugins.pipelineprocessor.ast.expressions.FunctionExpression.<init>(FunctionExpression.java:40) ~[?:?]
    at org.graylog.plugins.pipelineprocessor.parser.PipelineRuleParser$RuleAstBuilder.exitFunctionCall(PipelineRuleParser.java:356) ~[?:?]
    at org.graylog.plugins.pipelineprocessor.parser.RuleLangParser$FunctionCallContext.exitRule(RuleLangParser.java:1323) ~[?:?]
    at org.antlr.v4.runtime.tree.ParseTreeWalker.exitRule(ParseTreeWalker.java:71) ~[?:?]
    at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:54) ~[?:?]
    at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:52) ~[?:?]
    at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:52) ~[?:?]
    at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:52) ~[?:?]
    at org.graylog.plugins.pipelineprocessor.parser.PipelineRuleParser.parseRule(PipelineRuleParser.java:131) ~[?:?]
    at org.graylog.plugins.pipelineprocessor.rest.RuleResource.createFromParser(RuleResource.java:84) ~[?:?]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_91]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_91]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_91]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_91]
    at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) ~[graylog.jar:?]
    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) ~[graylog.jar:?]
    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) ~[graylog.jar:?]
    at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205) ~[graylog.jar:?]
    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) ~[graylog.jar:?]
    at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) ~[graylog.jar:?]
    at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) ~[graylog.jar:?]
    at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) ~[graylog.jar:?]
    at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326) [graylog.jar:?]
    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [graylog.jar:?]
    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [graylog.jar:?]
    at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [graylog.jar:?]
    at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [graylog.jar:?]
    at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [graylog.jar:?]
    at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [graylog.jar:?]
    at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) [graylog.jar:?]
    at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) [graylog.jar:?]
    at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:384) [graylog.jar:?]
    at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224) [graylog.jar:?]
    at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) [graylog.jar:?]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_91]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_91]
    at java.lang.Thread.run(Thread.java:745) [?:1.8.0_91]

Steps to reproduce the problem

  1. Create a rule:
rule "extract_mac"
when
  contains(value: to_string($message.full_message), search: "DHCPREQUEST")
then
    let matches = grok(pattern: "DHCPREQUEST for %{IPV4:client_ip} from %{COMMONMAC:client_mac} \\(%{NOTSPACE:client_name}\\)", value: to_string($message.full_message));
    set_fields(matches);
end

Environment

  • Graylog Version: 2.0.0
  • Pipeline processor plugin version: 1.0.0-beta.2
  • Elasticsearch Version: n/a
  • MongoDB Version: n/a
  • Operating System: n/a
  • Browser version: n/a
@kroepke kroepke added the bug label May 4, 2016
@kroepke
Copy link
Member Author

kroepke commented May 4, 2016

The working extractor is:

"grok_pattern: DHCPREQUEST for %{IPV4:client_ip} from %{COMMONMAC:client_mac} \(%{NOTSPACE:client_name}\)"

@kroepke kroepke added this to the 1.0.0 milestone May 4, 2016
@kroepke kroepke self-assigned this May 4, 2016
@henrikjohansen
Copy link

henrikjohansen commented May 4, 2016 •
edited by kroepke 

2016-05-04T10:58:12.306+02:00 WARN  [Function] Unable to precompute argument value for pattern
java.lang.NullPointerException
    at org.graylog.plugins.pipelineprocessor.ast.functions.Function.preprocessArgs(Function.java:55) [graylog-plugin-pipeline-processor-1.0.0-beta.2.jar:?]
    at org.graylog.plugins.pipelineprocessor.ast.expressions.FunctionExpression.<init>(FunctionExpression.java:40) [graylog-plugin-pipeline-processor-1.0.0-beta.2.jar:?]
    at org.graylog.plugins.pipelineprocessor.parser.PipelineRuleParser$RuleAstBuilder.exitFunctionCall(PipelineRuleParser.java:356) [graylog-plugin-pipeline-processor-1.0.0-beta.2.jar:?]
    at org.graylog.plugins.pipelineprocessor.parser.RuleLangParser$FunctionCallContext.exitRule(RuleLangParser.java:1323) [graylog-plugin-pipeline-processor-1.0.0-beta.2.jar:?]
    at org.antlr.v4.runtime.tree.ParseTreeWalker.exitRule(ParseTreeWalker.java:71) [graylog-plugin-pipeline-processor-1.0.0-beta.2.jar:?]
    at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:54) [graylog-plugin-pipeline-processor-1.0.0-beta.2.jar:?]
    at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:52) [graylog-plugin-pipeline-processor-1.0.0-beta.2.jar:?]
    at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:52) [graylog-plugin-pipeline-processor-1.0.0-beta.2.jar:?]
    at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:52) [graylog-plugin-pipeline-processor-1.0.0-beta.2.jar:?]
    at org.graylog.plugins.pipelineprocessor.parser.PipelineRuleParser.parseRule(PipelineRuleParser.java:131) [graylog-plugin-pipeline-processor-1.0.0-beta.2.jar:?]
    at org.graylog.plugins.pipelineprocessor.rest.RuleResource.createFromParser(RuleResource.java:84) [graylog-plugin-pipeline-processor-1.0.0-beta.2.jar:?]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_91]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_91]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_91]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_91]
    at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) [graylog.jar:?]
    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) [graylog.jar:?]
    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) [graylog.jar:?]
    at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205) [graylog.jar:?]
    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) [graylog.jar:?]
    at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) [graylog.jar:?]
    at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) [graylog.jar:?]
    at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) [graylog.jar:?]
    at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326) [graylog.jar:?]
    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [graylog.jar:?]
    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [graylog.jar:?]
    at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [graylog.jar:?]
    at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [graylog.jar:?]
    at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [graylog.jar:?]
    at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [graylog.jar:?]
    at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) [graylog.jar:?]
    at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) [graylog.jar:?]
    at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:384) [graylog.jar:?]
    at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224) [graylog.jar:?]
    at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) [graylog.jar:?]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_91]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_91]
    at java.lang.Thread.run(Thread.java:745) [?:1.8.0_91]

@kroepke
Copy link
Member Author

kroepke commented May 4, 2016

It's very simple: the grok function is in the code, but not actually exposed...

This uncovered two more problems, though, which I'll fix as part of this fix, will create more issues for them and link them here.

kroepke added a commit that referenced this issue May 4, 2016
@kroepke
actually bind the GrokMatch function so it is available in rules
e67ebb1 
fix issue #24
@kroepke kroepke mentioned this issue May 4, 2016
Closed
@kroepke
Copy link
Member Author

kroepke commented May 4, 2016

@henrikjohansen can you give https://drive.google.com/file/d/0B5NDLgIbCaSvM19FT2ZHcEYwZFU/view?usp=sharing a try, please?

@kroepke kroepke mentioned this issue May 4, 2016
Merged
@henrikjohansen
Copy link

@kroepke I'll take it for a spin this evening :)

@henrikjohansen
Copy link

@kroepke works like a charm now 👍

joschi pushed a commit that referenced this issue May 9, 2016
@kroepke @joschi
Bind the GrokMatch function so it is available in rules (#26)
446826c 
Fixes #24
@bernd bernd closed this as completed in 1ba3c63 May 11, 2016
@akiontke
Copy link

akiontke commented Sep 27, 2017 •
edited

@kroepke Is it possible, that this bug still occurs in 2.3.1?

I'm not able to save the rule if there is only a single \

@joschi
Copy link
Contributor

joschi commented Sep 27, 2017

@akiontke Please open a new bug report for your issue and include all necessary details, such as the complete error message, the complete rule, and an example message.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@kroepke kroepke

Labels
bug
Projects
None yet
Milestone
1.0.0
Development

No branches or pull requests
4 participants
@joschi @kroepke @henrikjohansen @akiontke