Prevent select_jsonpath queries return empty list values.

[miwent]

Expected Behavior

To have the option to remove empty list results from the paths output of select_jsonpath.

Current Behavior

Empty list values are returned from all select_jsonpath path queries when there are no query results.

Possible Solution

Add an optional setting for the select_jsonpath function to remove empty list values.

Steps to Reproduce (for bugs)

Use the following rule in the pipeline simulator with any message:

rule "testing select_jsonpath"
when
    true
then
    let json_msg = "{ \"json_array\":[{\"name\":\"obj_a\",\"value\":\"val_a\"}], \"blah\": \"blah\" }";
    
    let json_parsed = parse_json ( to_string ( json_msg ) );
        
    let paths = select_jsonpath (
        json: json_parsed,
        paths: {
            example_a: "$.blah",
            example_b: "$.nothing",
            exmaple_c: "$.[?(@.json_array)]..name",
            example_d: "$.[?(@.nothing)]..value"
            }
        );

    set_fields (
        fields: paths
        );

end

The results will be:

There is no "example_b" field (there is no root key by that name) but that example_d exists and is an empty list.

Context

Processing JSON event log messages from complex sources such as Office 365 and Okta leads to using the select_jsonpath with many path options, using path queries, to account for many different conditions.

Currently this leads to adding a stage of processing with multiple rules just to identify and remove empty query results. While it may sometimes be desirable to have these empty list values, the majority of the time this just leads to a large number of additional rules that exist only to remove the empty list vales in certain conditions.

Your Environment

• Graylog Version: 5.1.6
• Java Version: openjdk version "17.0.9" 2023-10-17
• OpenSearch Version:
• MongoDB Version:
• Operating System: Ubuntu 20.04.6 LTS
• Browser version: Firefox 120.0