parsers.md

Parsers

Parser operators are used to isolate values from a string. There are two classes of parsers, simple and complex.

Simple Parsers

Simple parsers perform a very specific operation where the result is assigned to a predetermined field in the log data model. For example, the time_parser will parse a timestamp and assign it to the a log record's Timestamp field.

List of simple parsers:

• time_parser
• severity_parser
• trace_parser
• scope_name_parser

Complex Parsers

Complex parsers differ from simple parsers in several ways.

1. Parsing produces multiple key/value pairs, where the values may be strings, ints, arrays, or objects.
2. By default, these key/value pairs will be added to log entry's attributes field. In the case of collisions between keys, the newly parsed value will override the existing value. Alternately, the field to which the object is written can be configured using the parse_to setting.
3. The configuration may "embed" certain followup operations. Generally, these operations correlate to the simple parsers listed above. Embedded operations are applied immediately after the primary parsing operation, and only if the primary operation was successful. Each embedded operation is executed independently of the others. (e.g. failure to parse a timestamp will not prevent an attempt to parse severity.)

The following examples illustrate how a json_parser may embed timestamp and severity parsers.

Consider a simple json log: {"message":"foo", "ts":"2022-08-10", "sev":"INFO"}

# Standalone json parser
- type: json_parser

# Regex parser with embedded timestamp and severity parsers
- type: json_parser
  timestamp:
    parse_from: attributes.ts
    layout_type: strptime
    layout: '%Y-%m-%d'
  severity:
    parse_from: attributes.sev

Note that when configuring embedded operations, it is typically necessary to reference a field that was set by the primary operation. In the above example, the values specified in the parse_from fields have taken into account that the json_parser will write key/value pairs to attributes.

List of complex parsers:

• json_parser
• regex_parser
• csv_parser
• key_value_parser
• uri_parser
• syslog_parser

List of embeddable operations:

• timestamp
• severity
• trace
• scope_name
• body: A field that should be assigned to a the log body.