Description
Component(s)
extension/encoding/jsonlogencoding
Is your feature request related to a problem? Please describe.
Currently, the extension expects input to be a JSON array. This requirement is enforced by the unmarshaler implementation with []map[string]any mapping.
However, when it comes to JSON log data, many such logs occur in new line delimited JSON (ndjson) format. Further, such logs do not get collected with array formatting expected by this encoder.
For example, a sample ndjson log,
{"time": "timestamp" , "message":"example valid log"}
{"time": "timestamp" , "message":"example valid log"}
The expected format of by current implementation
[
{"time": "timestamp" , "message":"example valid log"}
{"time": "timestamp" , "message":"example valid log"},
]
Describe the solution you'd like
Keeping the current behaviour as default, the extension should support a new configuration option to enable multiline JSON marshaling and unmarshaling.
json_log_encoding:
multiline: trueWhen multiline is true, []byte input is parsed through a scanner. And plogs are derived from each token-delimited scan read.
scanner := bufio.NewScanner(bytes.NewReader(buf))
for scanner.Scan() {
// unmarshal the content of the scanner.Text()
}Describe alternatives you've considered
Alternative is to parse content before forwarding to this extension. For example, extracting lines and then forward. But this is not efficient