Find file History


AMQP/RabbitMQ Log4J Appender

Managing log files in the cloud can be a real pain. SSH terminal sessions work okay, but what if you want to aggregate files from multiple server instances because you're troubleshooting a problem and you can't reasonably watch 12 individual log files at once?

This is why I wrote this appender. It's pretty simple. There's not many moving parts. Here's an example configuration:

  <appender name="cloud" class="com.jbrisbin.vcloud.logging.RabbitMQAppender">
    <param name="AppenderId" value="${instance.replyTo}"/>
    <param name="Host" value=""/>
    <param name="User" value="guest"/>
    <param name="Password" value="guest"/>
    <param name="VirtualHost" value="/"/>
    <param name="Exchange" value=""/>
    <param name="QueueNameFormatString" value="%1s.%2s"/>
    <layout class="org.apache.log4j.PatternLayout">
      <param name="ConversionPattern" value="%d %-5p %c{1} %m%n"/>

  <category name="com.jbrisbin.vcloud.logging">
    <level value="DEBUG"/>
    <appender-ref ref="console"/>
  <category name="com.jbrisbin.vcloud">
    <level value="DEBUG"/>
    <appender-ref ref="cloud"/>

The important stuff is the "AppenderId", which you need to set so your log aggregator can distinguish logging events from one server versus another; the RabbitMQ server info (host, username, password, virtual host, port isn't listed); and the topic exchange to which your logging events will be published.

How Events are Published

Logging events get published to the configured exchange using a routing key that is a combination of Level and Category. For example, at DEBUG level, logging for category com.jbrisbin.vcloud would, in the configuration above, go to a queue named:

This is configurable, so if you'd rather have the level on the end, you can invert the queueNameFormatString to "%2$s.%1$s". Check the Javadoc for java.util.Formatter for all the options when creating formatting strings.

The correlationId property is a concatenation of appenderId and System.currentTimeMillis(). This isn't configurable.

Aggregating Events

Here's some simple Groovy code using my RabbitMQ DSL to watch this exchange and aggregate logging events: "", durable: true, autoDelete: false) {
  queue name: "vcloud.logging.test", routingKey: "#", {
    consume onmessage: { msg ->
      print "${} ${msg.envelope.routingKey} ${msg.bodyAsString}"
      return true