JSON Extractor fails to convert to fields when there are fields with null value

[gvagenas]

Working with the latest 1.2.1.

Even though when testing with "Try" I can see all the fields, even the ones with null value, the extractor doesn't convert the string to fields.

I believe the fields with null value should be extracted with string "null" instead of null value

[joschi]

@gvagenas Could you please elaborate on the problem and maybe add a concrete example to the issue?

[gvagenas]

@joschi see bellow a json message that contains null values:

{  
  "1xx-processed":0,
  "1xx-sent":0,
  "2xx-processed":0,
  "2xx-sent":0,
  "3xx-processed":0,
  "3xx-sent":0,
  "4xx-processed":0,
  "4xx-sent":0,
  "5xx-processed":0,
  "5xx-sent":0,
  "6xx-processed":0,
  "6xx-sent":0,
  "7xx-processed":0,
  "7xx-sent":0,
  "8xx-processed":0,
  "8xx-sent":0,
  "9xx-processed":0,
  "9xx-sent":0,
  "ack-processed":0,
  "ack-sent":0,
  "additional-parameterable-headers":null,
  "app-dispatcher-class":"org.mobicents.servlet.sip.core.SipApplicationDispatcherImpl",
  "application-router":"configuration/dars/mobicents-dar.properties",
  "back-to-normal-memory-threshold":75,
  "base-timer-interval":500,
  "bye-processed":0,
  "bye-sent":0,
  "cancel-processed":0,
  "cancel-sent":0,
  "canceled-timer-tasks-purge-period":-1,
  "concurrency-control-mode":"SipApplicationSession",
  "congestion-control-interval":"-1",
  "congestion-control-policy":"ErrorResponse",
  "dialog-pending-request-checking":false,
  "dns-resolver-class":null,
  "dns-server-locator-class":null,
  "dns-timeout":1,
  "gather-statistics":false,
  "info-processed":0,
  "info-sent":0,
  "instance-id":null,
  "invite-processed":0,
  "invite-sent":0,
  "memory-threshold":85,
  "message-processed":0,
  "message-sent":0,
  "notify-processed":0,
  "notify-sent":0,
  "number-client-transaction":0,
  "number-dialog":0,
  "number-early-dialog":0,
  "number-server-transaction":0,
  "options-processed":0,
  "options-sent":0,
  "outbound-proxy":null,
  "path-name":"org.mobicents.ext",
  "prack-processed":0,
  "prack-sent":0,
  "publish-processed":0,
  "publish-sent":0,
  "refer-processed":0,
  "refer-sent":0,
  "register-processed":0,
  "register-sent":0,
  "stack-properties":"configuration/mss-sip-stack.properties",
  "subscribe-processed":0,
  "subscribe-sent":0,
  "t2-interval":4000,
  "t4-interval":5000,
  "timer-d-interval":32000,
  "update-processed":0,
  "update-sent":0,
  "use-pretty-encoding":true,
  "connector":{  
    "sip-udp":null,
    "sip-tcp":null,
    "sip-tls":null,
    "sip-ws":null
  }
}

When trying to extract fields with a JSON extractor the "Try!" button gives me all the fields as expected and the ones with null values comes with just the key:

cancel-processed
    0
*** connector.sip-udp ***
application-router
    configuration/dars/mobicents-dar.properties
message-sent
    0
path-name
    org.mobicents.ext
subscribe-processed
    0
base-timer-interval
    500
update-sent
    0
...

Noticed the property "connector" : { "sip-udp" : null => connector.sip-udp ?

Now even like this when I save the extractor, I see no extracted fields, not even the one that have values.

After applying the patch #1476 it will come with:
connector.sip-udp = null and all the fields will be available.

Thanks
George

[joschi]

Mapping a JSON null value to null is intentional and mapping null to "null" would change this semantic.

Please check your Graylog server logs for error messages (maybe a NullPointerException) around the time you've received those messages.

[gvagenas]

@joschi, there are no NPE at server log.
I changed to DEBUG log level and created a gist, please check here:
https://gist.github.com/gvagenas/7903010bbe4912ca2d7f

George

[gvagenas]

Also, I agree on the semantics of null, the patch I provided was a quick hack that maybe can provide hints for where is the problem

[chiefy]

Currently evaluating GL as we're not happy with CloudWatch logging & JSON, and this is kind of a blocker. 😞

[bernd]

The issue has been fixed and will be in the upcoming Graylog 1.2.2 release.

[gvagenas]

Thanks @bernd . Do you have a ETA for the 1.2.2 release? I have a blog post ready for how to use Graylog with Restcomm (www.restcomm.com) and waiting the 1.2.2 release.

Regards
George

[joschi]

@gvagenas Graylog 1.2.2 is being released as we speak.

[gvagenas]

Thats great :) Congrats guys