Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NPE during preProcessArgs using Grok pattern #24

Closed
kroepke opened this issue May 4, 2016 · 8 comments
Closed

NPE during preProcessArgs using Grok pattern #24

kroepke opened this issue May 4, 2016 · 8 comments
Assignees
Labels
bug
Milestone

Comments

@kroepke
Copy link
Member

@kroepke kroepke commented May 4, 2016

Problem description

Saving the rule below leads to a bad error message (cannot save rule "" with a 500) and this exception in the server log:

2016-05-04T10:58:12.307+02:00 ERROR [AnyExceptionClassMapper] Unhandled exception in REST resource
java.lang.NullPointerException
    at org.graylog.plugins.pipelineprocessor.ast.functions.Function.preprocessArgs(Function.java:55) ~[?:?]
    at org.graylog.plugins.pipelineprocessor.ast.expressions.FunctionExpression.<init>(FunctionExpression.java:40) ~[?:?]
    at org.graylog.plugins.pipelineprocessor.parser.PipelineRuleParser$RuleAstBuilder.exitFunctionCall(PipelineRuleParser.java:356) ~[?:?]
    at org.graylog.plugins.pipelineprocessor.parser.RuleLangParser$FunctionCallContext.exitRule(RuleLangParser.java:1323) ~[?:?]
    at org.antlr.v4.runtime.tree.ParseTreeWalker.exitRule(ParseTreeWalker.java:71) ~[?:?]
    at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:54) ~[?:?]
    at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:52) ~[?:?]
    at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:52) ~[?:?]
    at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:52) ~[?:?]
    at org.graylog.plugins.pipelineprocessor.parser.PipelineRuleParser.parseRule(PipelineRuleParser.java:131) ~[?:?]
    at org.graylog.plugins.pipelineprocessor.rest.RuleResource.createFromParser(RuleResource.java:84) ~[?:?]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_91]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_91]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_91]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_91]
    at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) ~[graylog.jar:?]
    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) ~[graylog.jar:?]
    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) ~[graylog.jar:?]
    at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205) ~[graylog.jar:?]
    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) ~[graylog.jar:?]
    at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) ~[graylog.jar:?]
    at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) ~[graylog.jar:?]
    at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) ~[graylog.jar:?]
    at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326) [graylog.jar:?]
    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [graylog.jar:?]
    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [graylog.jar:?]
    at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [graylog.jar:?]
    at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [graylog.jar:?]
    at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [graylog.jar:?]
    at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [graylog.jar:?]
    at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) [graylog.jar:?]
    at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) [graylog.jar:?]
    at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:384) [graylog.jar:?]
    at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224) [graylog.jar:?]
    at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) [graylog.jar:?]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_91]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_91]
    at java.lang.Thread.run(Thread.java:745) [?:1.8.0_91]

Steps to reproduce the problem

  1. Create a rule:
rule "extract_mac"
when
  contains(value: to_string($message.full_message), search: "DHCPREQUEST")
then
    let matches = grok(pattern: "DHCPREQUEST for %{IPV4:client_ip} from %{COMMONMAC:client_mac} \\(%{NOTSPACE:client_name}\\)", value: to_string($message.full_message));
    set_fields(matches);
end

Environment

  • Graylog Version: 2.0.0
  • Pipeline processor plugin version: 1.0.0-beta.2
  • Elasticsearch Version: n/a
  • MongoDB Version: n/a
  • Operating System: n/a
  • Browser version: n/a
@kroepke kroepke added the bug label May 4, 2016
@kroepke
Copy link
Member Author

@kroepke kroepke commented May 4, 2016

The working extractor is:

"grok_pattern: DHCPREQUEST for %{IPV4:client_ip} from %{COMMONMAC:client_mac} \(%{NOTSPACE:client_name}\)"
@kroepke kroepke added this to the 1.0.0 milestone May 4, 2016
@kroepke kroepke self-assigned this May 4, 2016
@henrikjohansen
Copy link

@henrikjohansen henrikjohansen commented May 4, 2016

2016-05-04T10:58:12.306+02:00 WARN  [Function] Unable to precompute argument value for pattern
java.lang.NullPointerException
    at org.graylog.plugins.pipelineprocessor.ast.functions.Function.preprocessArgs(Function.java:55) [graylog-plugin-pipeline-processor-1.0.0-beta.2.jar:?]
    at org.graylog.plugins.pipelineprocessor.ast.expressions.FunctionExpression.<init>(FunctionExpression.java:40) [graylog-plugin-pipeline-processor-1.0.0-beta.2.jar:?]
    at org.graylog.plugins.pipelineprocessor.parser.PipelineRuleParser$RuleAstBuilder.exitFunctionCall(PipelineRuleParser.java:356) [graylog-plugin-pipeline-processor-1.0.0-beta.2.jar:?]
    at org.graylog.plugins.pipelineprocessor.parser.RuleLangParser$FunctionCallContext.exitRule(RuleLangParser.java:1323) [graylog-plugin-pipeline-processor-1.0.0-beta.2.jar:?]
    at org.antlr.v4.runtime.tree.ParseTreeWalker.exitRule(ParseTreeWalker.java:71) [graylog-plugin-pipeline-processor-1.0.0-beta.2.jar:?]
    at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:54) [graylog-plugin-pipeline-processor-1.0.0-beta.2.jar:?]
    at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:52) [graylog-plugin-pipeline-processor-1.0.0-beta.2.jar:?]
    at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:52) [graylog-plugin-pipeline-processor-1.0.0-beta.2.jar:?]
    at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:52) [graylog-plugin-pipeline-processor-1.0.0-beta.2.jar:?]
    at org.graylog.plugins.pipelineprocessor.parser.PipelineRuleParser.parseRule(PipelineRuleParser.java:131) [graylog-plugin-pipeline-processor-1.0.0-beta.2.jar:?]
    at org.graylog.plugins.pipelineprocessor.rest.RuleResource.createFromParser(RuleResource.java:84) [graylog-plugin-pipeline-processor-1.0.0-beta.2.jar:?]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_91]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_91]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_91]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_91]
    at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) [graylog.jar:?]
    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) [graylog.jar:?]
    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) [graylog.jar:?]
    at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205) [graylog.jar:?]
    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) [graylog.jar:?]
    at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) [graylog.jar:?]
    at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) [graylog.jar:?]
    at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) [graylog.jar:?]
    at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326) [graylog.jar:?]
    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [graylog.jar:?]
    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [graylog.jar:?]
    at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [graylog.jar:?]
    at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [graylog.jar:?]
    at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [graylog.jar:?]
    at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [graylog.jar:?]
    at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) [graylog.jar:?]
    at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) [graylog.jar:?]
    at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:384) [graylog.jar:?]
    at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224) [graylog.jar:?]
    at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) [graylog.jar:?]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_91]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_91]
    at java.lang.Thread.run(Thread.java:745) [?:1.8.0_91]
@kroepke
Copy link
Member Author

@kroepke kroepke commented May 4, 2016

It's very simple: the grok function is in the code, but not actually exposed...

This uncovered two more problems, though, which I'll fix as part of this fix, will create more issues for them and link them here.

@henrikjohansen
Copy link

@henrikjohansen henrikjohansen commented May 4, 2016

@kroepke I'll take it for a spin this evening :)

@henrikjohansen
Copy link

@henrikjohansen henrikjohansen commented May 4, 2016

@kroepke works like a charm now 👍

@akiontke
Copy link

@akiontke akiontke commented Sep 27, 2017

@kroepke Is it possible, that this bug still occurs in 2.3.1?

I'm not able to save the rule if there is only a single \

@joschi
Copy link
Contributor

@joschi joschi commented Sep 27, 2017

@akiontke Please open a new bug report for your issue and include all necessary details, such as the complete error message, the complete rule, and an example message.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.