I was directed here from the community forum, as this behaviour appears to be a bug (https://community.graylog.org/t/alert-processing-query/5964).
I have a set of data with key value pairs separated by a colon and a whitespace ": ". The whitespace is being ignored, resulting in data being split on a colon. This breaks the data, as it contains timestamps (e.g. 12:34:34).
I have a data set coming into graylog, into a pipeline and then using a rule to extract the field names and data values.
email: firstname.lastname@example.org, event: user login, time: 2018-07-18T08:02:19Z, user_id: 12345
My processing rule looks like this:
This breaks, as it splits on the timestamp. I changed kv_delimiters to this:
kv_delimiters: ": "
This includes the whitespace, but the behaviour does not change. The split is done exclusively on the colon, breaking timestamps
I tried escaping the whitespace with \ but that results in an error.
The separator is always a ": " but : can occur in a data field, as can whitespace. Being able to split on ": " would be extremely useful.
This is Graylog 2.4.5.
The text was updated successfully, but these errors were encountered:
Hello, I'm trying to simulate the bug and noticed another bug.
Using the message in your example:
"email: email@example.com, event: user login, time: 2018-07-18T08:02:19Z, user_id: 12345"
Not only the timestamp gets truncated in the first occurrence of the colon inside the timestamp but also the event gets truncated in the space character, so the event gets mapped to just "user" and the "login" gets lost.
Continuing with this, the problem resides in the class:
Graylog is using CharMatcher from Guava library, that is specially built to work with single characters, not strings.
another approach would be to fix current behaviour to only apply the inner splitter on the first occurrence of the kv_delimiter char, so next occurrences inside the value string would not be treated.
This way it can still working with the CharMatcher.
So, how should I try to fix it?