Splunk HEC only accepts flat JSON object (not nested)

[hvaghani221]

Component(s)

exporter/splunkhec, receiver/splunkhec

What happened?

Description

According to the Splunk HEC doc,
fields key specifies a JSON object that contains a flat (not nested) list of explicit custom fields to be defined at index time.

Steps to Reproduce

Execute the following curl request

curl -k -H "Authorization: Splunk 00000000-0000-0000-0000-000000000000" https://splunk:8088/services/collector -d '{"fields":{"keyArray":["A", [1]]}, "event": "hello world"}'

curl -k -H "Authorization: Splunk 00000000-0000-0000-0000-000000000000" https://splunk:8088/services/collector -d '{"fields":{"keyMap":{"field": "value"}}, "event": "hello world"}'

It throws this error

{"text":"Error in handling indexed fields","code":15,"invalid-event-number":0}

The implementation of the splunkhec exporter and receiver doesn't handle it explicitly. It can lead to unexpected failure/invalid behaviour in both the exporter/receiver

Collector version

v0.67.0

Environment information

No response

OpenTelemetry Collector configuration

No response

Log output

No response

Additional context

No response

[hvaghani221]

CC: @dmitryax @atoulme

[github-actions]

Pinging code owners for receiver/splunkhec: @atoulme @keitwb. See Adding Labels via Comments if you do not have permissions to add labels yourself.

[github-actions]

Pinging code owners for exporter/splunkhec: @atoulme @dmitryax. See Adding Labels via Comments if you do not have permissions to add labels yourself.

[atoulme]

Thank you for finding this bug!

[hvaghani221]

@atoulme, I am thinking of fixing this bug later this week. For the receiver, it should return the same error as an actual Splunk HEC does.
But, I am not sure about the exporter. I can think of 3 ways to handle this:

1. Drop the entire event (unreasonable)
2. Drop the nested field
3. Flatten the nested map and drop the nested array (I prefer this)

Please share your thoughts.

[fatsheep9146]

can this issue be assigned to you? @atoulme

[atoulme]

3. seems ok with me. Serialize the map to a string?

@fatsheep9146 please feel free to assign to me, but it looks like @harshit-splunk is the one doing the work. It should probably go to him.

[fatsheep9146]

ok, I assigned this to @harshit-splunk

[hvaghani221]

Serialize the map to a string?

We can serialise nested arrays, but not sure about the map. AFAIK, Splunk will not populate search-time fields from the indexed field if the indexed field contains any serialised map. User has to write their own field extraction logic in splunk.

[atoulme]

Could we flatten the map, prefixing with the top key?

[hvaghani221]

Could we flatten the map, prefixing it with the top key?

Yes, that's what I am going to do.

[hvaghani221]

Closing as fixed.