Add support for multi-line logs

[otisg]

e.g. Java exception stack traces

Inspiration for how (not to) handle this could be taken from Logstash or rsyslog.

[megastef]

Multi-line logs need a delimiter for the text block (e.g. the date). Each pattern with multi-line support must have the property "blockStart" (a regular expression).

Example:

INFO [main ] 2012-02-10 09:15:33,112 DatabaseDescriptor.java  (line 495 )
     Found table data in data directories. Consider using the CLI to define your schema.
INFO  [main ] 2012-02-10 09:15:33,135 CommitLog.java  (line 166 )
     No commitlog files found; skipping replay

and the related pattern definition:

- # Apache Cassandra
  type: apache_cassandra
  sourceName: !!js/regexp cassandra
  # multi-line, start sequence
  blockStart: !!js/regexp ^\S+\s\[.+\]\s\d{4}-\d{2}-\d{2}\s[\d|\:]+,\d+\s
  match:
    - regex: !!js/regexp (\S+)\s\[(.+)\]\s(\d{4}-\d{2}-\d{2}\s[\d|\:]+,\d+)\s(.+.java)\s+\(line\s(\d+).+\n\s+(.*)
      fields: [severity,module,ts,java_file,code_line,message]
      dateFormat: YYYY-MM-DD HH:mm:ss,SS

Results in:

severity:   INFO
module:     main 
ts:         2012-02-10 09:15:33,112
java_file:  DatabaseDescriptor.java
code_line:  495
message:    Found table data in data directories. Consider using the CLI to define your schema.
@timestamp: Fri Feb 10 2012 09:15:33 GMT+0100 (CET)