Add option to split log messages on newlines - fixes #404

[metacubed]

This PR adds an option to output the "message" field as a JSON array, instead of a JSON string.

For example, with either of these logging statements:

logger.error("first line{}second line", System.lineSeparator());
// or
logger.error("first line\nsecond line");
// or
logger.error("first line\r\nsecond line");

The output is consistent:

{
    "message": [
        "first line",
        "second line"
    ],
    ...
}

This provides an unambiguous parsing format on the target system instead of relying on the source platform's newline representation.

[philsttr]

Thanks for the PR!

In addition to addressing my inline comments, please update the README.md with documentation for your change. Specifically, the message provider description/properties needs to be updated here, and a new section for "Splitting Message into an Array" could probably be added after the "Customizing Timestamp" section.

[philsttr]

Hmm. The Encoder's lineSeparator property is the string to output between events. Coupling the separator to output between events with the regex to use to split the message string does not seem ideal. I think the regex to use to split the message string should be separate from the separator string to use between events.

[philsttr]

It's not clear that the "lineSeparator" is only used as a regex to split the message into an array if and only if "writeMultiLineMessage" is true.

Additionally, the term "writeMultiLineMessage" does not convey that the message field will be written as an array. A message with embedded line separators could also be considered a "multi line message".

Perhaps, we can combine these two fields into a single String field named "messageSplitRegex". When null/empty, the message will not be split, and will be written as a JSON string. When non-null, the message will be split, and written as a JSON array.

[philsttr]

Same comment as LogstashFormatter regarding naming of these fields. Combine into a single "messageSplitRegex" field.

Also add some javadoc for this field.

[metacubed]

@philsttr, thanks for the review. I'll make the changes.

I can use the existing SeparatorParser class to handle the special strings "SYSTEM", "UNIX", "WINDOWS", etc. Any other strings would be treated as raw regex patterns.

[philsttr]

I just had another thought. Do we really need regexes for splitting the message? regexes can be computationally expensive. I wonder if just searching for specific strings would be faster, maybe just using a StringTokenizer.

[metacubed]

A couple of problems with StringTokenizer:

• It does not support multi-character delimiters. For example, specifying \r\n as the delimiter will split by \r and by \n, not the combination.

• It does not preserve consecutive delimiters. For example, "Hello\n\nWorld" will return [ "Hello", "World" ], instead of the expected [ "Hello", "", "World" ].

Also note the Javadoc comment on StringTokenizer:

StringTokenizer is a legacy class that is retained for compatibility reasons although its use is discouraged in new code. It is recommended that anyone seeking this functionality use the split method of String or the java.util.regex package instead.

Both of those are regex-based.

[philsttr]

Sounds good. It's a shame there isn't an easy way to split a string that doesn't involve regexes.

[metacubed]

The Apache Commons StringTokenizer might be worth exploring, but it adds a new dependency on commons-text. Should I try it out?

[philsttr]

Just use a regex. I'd rather not introduce another dependency.

[metacubed]

@philsttr, all right, looks like I've addressed all the review comments.

[philsttr]

Beautiful!