Problem like #24

[akiontke]

Problem description

Can't save a rule with ( in grok pattern

Steps to reproduce the problem

1. Create Rule

rule "extract_mac"
when
  contains(value: to_string($message.full_message), search: "DHCPREQUEST")
then
    let matches = grok(pattern: "DHCPREQUEST for %{IPV4:client_ip} from %{COMMONMAC:client_mac} \\(%{NOTSPACE:client_name}\\)", value: to_string($message.full_message));
    set_fields(matches);
end

2. Try to save

Environment

• Graylog Version: 2.3.1
• Pipeline Processor plugin version: 2.3.1
• Elasticsearch Version:
• MongoDB Version:
• Operating System: Debian 9
• Browser version: Google Chrome Version 61.0.3163.100 (Official Build) (64-bit)

[joschi]

@akiontke Please check the logs of your Graylog node(s) and attach the complete error messages which occur when trying to save the rule.

[akiontke]

@joschi I get the following error message with the mentioned example.

2017-09-28T10:56:33.600+02:00 ERROR [AnyExceptionClassMapper] Unhandled exception in REST resource
java.lang.NullPointerException: null
        at org.graylog.plugins.pipelineprocessor.ast.functions.FunctionArgs.lambda$getConstantArgs$0(FunctionArgs.java:57) ~[?:?]
        at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:174) ~[?:1.8.0_102]
        at java.util.HashMap$EntrySpliterator.forEachRemaining(HashMap.java:1691) ~[?:1.8.0_102]
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) ~[?:1.8.0_102]
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) ~[?:1.8.0_102]
        at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) ~[?:1.8.0_102]
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:1.8.0_102]
        at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499) ~[?:1.8.0_102]
        at org.graylog.plugins.pipelineprocessor.ast.functions.FunctionArgs.getConstantArgs(FunctionArgs.java:59) ~[?:?]
        at org.graylog.plugins.pipelineprocessor.ast.functions.Function.preprocessArgs(Function.java:54) ~[?:?]
        at org.graylog.plugins.pipelineprocessor.ast.expressions.FunctionExpression.<init>(FunctionExpression.java:44) ~[?:?]
        at org.graylog.plugins.pipelineprocessor.parser.PipelineRuleParser$RuleAstBuilder.exitFunctionCall(PipelineRuleParser.java:411) ~[?:?]
        at org.graylog.plugins.pipelineprocessor.parser.RuleLangParser$FunctionCallContext.exitRule(RuleLangParser.java:1434) ~[?:?]
        at org.antlr.v4.runtime.tree.ParseTreeWalker.exitRule(ParseTreeWalker.java:71) ~[?:?]
        at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:54) ~[?:?]
        at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:52) ~[?:?]
        at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:52) ~[?:?]
        at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:52) ~[?:?]
        at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:52) ~[?:?]
        at org.graylog.plugins.pipelineprocessor.parser.PipelineRuleParser.parseRule(PipelineRuleParser.java:170) ~[?:?]
        at org.graylog.plugins.pipelineprocessor.parser.PipelineRuleParser.parseRule(PipelineRuleParser.java:135) ~[?:?]
        at org.graylog.plugins.pipelineprocessor.rest.RuleResource.update(RuleResource.java:174) ~[?:?]
        at sun.reflect.GeneratedMethodAccessor415.invoke(Unknown Source) ~[?:?]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_102]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_102]
        at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) ~[graylog.jar:?]
        at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [graylog.jar:?]
        at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [graylog.jar:?]
        at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) [graylog.jar:?]
        at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) [graylog.jar:?]
        at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:384) [graylog.jar:?]
        at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224) [graylog.jar:?]
        at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) [graylog.jar:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_102]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_102]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_102]

[joschi]

@akiontke I just tried to reproduce this issue on Graylog 2.3.1 (Docker image) but everything worked as expected.

Please post the contents of the System / Nodes / Details page ("Installed plugins" specifically). Maybe you're running an old/incompatible version of the Pipeline Processor Plugin?

Also check the contents of the System / Grok Patterns page and make sure that all referenced Grok patterns in your rule (IPV4, COMMONMAC, etc.) do exist.

For reference:
docker-compose.yml

version: '2'
services:
  mongodb:
    image: mongo:3
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:5.6.2
    environment:
      - http.host=0.0.0.0
      - transport.host=localhost
      - network.host=0.0.0.0
      - xpack.security.enabled=false
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    mem_limit: 1g
  graylog:
    image: graylog/graylog:2.3.1-2
    mem_limit: 4g
    environment:
      - GRAYLOG_PASSWORD_SECRET=somepasswordpepper
      # Password: admin
      - GRAYLOG_ROOT_PASSWORD_SHA2=8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918
      - GRAYLOG_WEB_ENDPOINT_URI=http://127.0.0.1:9000/api
    links:
      - mongodb:mongo
      - elasticsearch
    depends_on:
      - mongodb
      - elasticsearch
    ports:
      - 9000:9000

[akiontke]

These are the plugins in use

Name	Version	Author	Description
Anonymous Usage Statistics	2.3.1	Graylog, Inc.	A plugin for collecting anonymous usages statistics about Graylog nodes and clusters.  Website
Collector	2.3.1	Graylog, Inc.	Collectors plugin  Website
Elastic Beats Input	2.3.1	Graylog, Inc.	Input plugin for Elastic Beats (Beats/Lumberjack protocol).  Website
Enterprise Integration Plugin	2.3.1	Graylog, Inc	Provides basic integration with Graylog Enterprise  Website
Internal Metrics InfluxDB Reporter	1.4.0	Graylog, Inc.	A plugin for reporting internal Graylog metrics to InfluxDB.  Website
MapWidgetPlugin	2.3.1	Graylog, Inc.	Map widget for Graylog  Website
NetFlow Plugin	2.3.0-rc.5	Graylog, Inc.	Provides NetFlow inputs  Website
Pipeline Processor Plugin	2.3.1	Graylog, Inc	Pluggable pipeline processing framework  Website
Slack	2.4.0	Graylog, Inc.	Slack plugin to forward messages or write alarms to Slack chat rooms.  Website

All mentioned grok patterns are configured