Batch formatters

The batch formatter is responsible for batching a sequence of log events into a single payload that can be sent over the network. It does not care about individual log event formatting but instead focuses on how these events are serialized together into a single HTTP request.

The sink comes pre-loaded with one JSON based batch formatter named ArrayBatchFormatter. You can decide to configure your sink to use this batch formatter, or write your own by implementing IBatchFormatter.

/// <summary>
/// Formats batches of log events into payloads that can be sent over the network.
/// </summary>
public interface IBatchFormatter
{
  /// <summary>
  /// Format the log events into a payload.
  /// </summary>
  /// <param name="logEvents">
  /// The events to format.
  /// </param>
  /// <param name="formatter">
  /// The formatter turning the log events into a textual representation.
  /// </param>
  /// <param name="output">
  /// The payload to send over the network.
  /// </param>
  void Format(IEnumerable<LogEvent> logEvents, ITextFormatter formatter, TextWriter output);

  /// <summary>
  /// Format the log events into a payload.
  /// </summary>
  /// <param name="logEvents">
  /// The events to format.
  /// </param>
  /// <param name="output">
  /// The payload to send over the network.
  /// </param>
  void Format(IEnumerable<string> logEvents, TextWriter output);
}

Formatter types

• Batch formatter - ArrayBatchFormatter
• Custom formatter 1
• Custom formatter 2 - DefaultBatchFormatter

Batch formatter - ArrayBatchFormatter

A sequence of log events are stored as a JSON array. This formatter is compatible with the Logstash HTTP input plugin configured to use the JSON codec.

[
  { event n },
  { event n+1 }
]

Custom formatter 1

You can always implement your own custom formatter assuming ArrayBatchFormatter isn't to your satisfaction. Let's implement a formatter that will produce the following JSON object.

{ event n }, { event n+1 }

As stated in the beginning of the document, we need to implement the IBatchFormatter interface. Let's also add some error handling given any arguments are null or the sequence of log events is empty. Our new formatter would look something like this.

public class MyBatchFormatter : IBatchFormatter
{
  public void Format(IEnumerable<LogEvent> logEvents, ITextFormatter formatter, TextWriter output)
  {
    if (logEvents == null) throw new ArgumentNullException(nameof(logEvents));
    if (formatter == null) throw new ArgumentNullException(nameof(formatter));

    IEnumerable<string> formattedLogEvents = logEvents.Select(
      logEvent =>
      {
        var writer = new StringWriter();
        formatter.Format(logEvent, writer);
        return writer.ToString();
      });

    Format(formattedLogEvents, output);
  }

  public void Format(IEnumerable<string> logEvents, TextWriter output)
  {
    if (logEvents == null) throw new ArgumentNullException(nameof(logEvents));
    if (output == null) throw new ArgumentNullException(nameof(output));

    // Abort if sequence of log events is empty
    if (!logEvents.Any())
    {
      return;
    }

    var delimStart = string.Empty;

    foreach (var logEvent in logEvents)
    {
      if (string.IsNullOrWhiteSpace(logEvent))
      {
        continue;
      }

      output.Write(delimStart);
      output.Write(logEvent);
      delimStart = ",";
    }
  }
}

Custom formatter 2 - DefaultBatchFormatter

A breaking change in v8 was the removal of DefaultBatchFormatter. Applications that relied on this batch formatter had to keep a copy of the class in their own codebase. This chapter, for future reference, contains the complete implementation of DefaultBatchFormatter, as it was defined in its latest official version.

A sequence of log events are stored in a JSON object under a property called events. This formatter is inspired by the Seq sink.

{
  "events": [
    { event n },
    { event n+1 }
  ]
}

public class DefaultBatchFormatter : IBatchFormatter
{
    public void Format(IEnumerable<string> logEvents, TextWriter output)
    {
        if (logEvents == null) throw new ArgumentNullException(nameof(logEvents));
        if (output == null) throw new ArgumentNullException(nameof(output));

        // Abort if sequence of log events is empty
        if (!logEvents.Any())
        {
            return;
        }

        output.Write("{\"events\":[");

        var delimStart = string.Empty;

        foreach (var logEvent in logEvents)
        {
            if (string.IsNullOrWhiteSpace(logEvent))
            {
                continue;
            }

            output.Write(delimStart);
            output.Write(logEvent);
            delimStart = ",";
        }

        output.Write("]}");
    }
}